Document of The World Bank

Public Disclosure Authorized Report No:ICR0000343

IMPLEMENTATION COMPLETION AND RESULTS REPORT (CIDA-52996, IDA-H0360)

ON AN

IDA GRANT Public Disclosure Authorized

IN THE AMOUNT OF SDR 3.3 MILLION (US$ 4.5 MILLION EQUIVALENT)

TO

KOSOVO

FOR AN

EDUCATION PARTICIPATION IMPROVEMENT PROJECT Public Disclosure Authorized

June 29, 2007

Human Development Sector Unit South East Europe Country Unit

Public Disclosure Authorized Europe and Central Asia Region

CURRENCY EQUIVALENTS

(Exchange Rate Effective 28/02/2007)

Currency Unit = Euro Euro 1.00 = US$ 1.32 US$ 1.00 = Euro 0.76

FISCAL YEAR [January 1 – December 31]

ABBREVIATIONS AND ACRONYMS

CIDA Canadian International Development Agency NER Net Enrollment Rate DEST Department of Education, Science and Technology NGO Non-governmental Organization DO Development Objective NPV Net Present Value EHP Education and Health Project PAD Project Appraisal Document EMIS Education Management Information System PCU Project Coordination Unit EOI Expression of Interest PDO Project Development Objective EPIP Education Participation Improvement Project PISG Provisional Institutions of Self Government ERR Economic Rate of Return PL Poverty Line FPL Food Poverty Line PMU Project Management Unit FRY Former Republic of Yugoslavia POM Project Operating Manual GER Gross Enrollment Rate QAE Quality at Entry Assessment ICR Implementation Completion Results Report QER Quality Enhancement Review IDA International Development Association QAG Quality Assessment Group INT Institutional Integrity Office QAS Quality of Supervision Assessment ISR Implementation REO Regional Education Office KEC Kosovo Education Center ROSO Regional Outreach School Officer LSMS Living Standard Measurement Survey SDG School Development Grant M&E Monitoring and Evaluation SGAB School Grants Approval Board MDG Millennium Development Goals SIL Specific Investment Loan MED Municipality Education Director SRSG Special Representative of the UN Secretary MEST Ministry of Education, Science and Technology General MSGT Municipal School Grant Team TSS Transitional Support Strategy MTR Mid-Term Review UIS UNESCO Institute of Statistics NATO North Atlantic Treaty Organization UNMIK United Nations Interim Administration Mission in Kosovo

Vice President: Shigeo Katsu Country Director: Orsalia Kalantzopoulos Sector Manager: Mamta Murthi Project Team Leader: Keiko Miwa ICR Team Leader: Sachiko Kataoka

Kosovo Education Participation Improvement Project

CONTENTS

Data Sheet A. Basic Information B. Key Dates C. Ratings Summary D. Sector and Theme Codes E. Bank Staff F. Results Framework Analysis G. Ratings of Project Performance in ISRs H. Restructuring I. Disbursement Graph

1. Project Context, Development Objectives and Design...... 1 2. Key Factors Affecting Implementation and Outcomes ...... 5 3. Assessment of Outcomes...... 11 4. Assessment of Risk to Development Outcome...... 18 5. Assessment of Bank and Borrower Performance ...... 19 6. Lessons Learned ...... 21 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners ...... 22 Annex 1. Project Costs and Financing...... 23 Annex 2. Outputs by Component ...... 24 Annex 3. Economic and Financial Analysis...... 26 Annex 4. Bank Lending and Implementation Support/Supervision Processes ...... 27 Annex 5. Beneficiary Survey Results...... 29 Annex 6. Stakeholder Workshop Report and Results...... 30 Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR...... 33 Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders...... 37 Annex 9. List of Supporting Documents ...... 38

MAP - IBRD 35406

A. Basic Information Education Participation Country: Kosovo Project Name: Improvement Project CIDA-52996,IDA- Project ID: P079260 L/C/TF Number(s): H0360 ICR Date: 06/29/2007 ICR Type: Core ICR Lending Instrument: SIL Borrower: UNMIK Original Total XDR 3.3M Disbursed Amount: XDR 3.2M Commitment: Environmental Category: C Implementing Agencies: Ministry of Education, Science and Technology Cofinanciers and Other External Partners: Canadian International Development Agency (CIDA)

B. Key Dates Revised / Actual Process Date Process Original Date Date(s) Concept Review: 08/30/2002 Effectiveness: 07/23/2003 07/23/2003 Appraisal: 12/09/2002 Restructuring(s): Approval: 05/13/2003 Mid-term Review: 02/14/2005 02/14/2005 Closing: 12/31/2006 12/31/2006

C. Ratings Summary C.1 Performance Rating by ICR Outcomes: Moderately Satisfactory Risk to Development Outcome: Moderate Bank Performance: Moderately Satisfactory Borrower Performance: Moderately Satisfactory

C.2 Detailed Ratings of Bank and Borrower Performance (by ICR) Bank Ratings Borrower Ratings Quality at Entry: Moderately Satisfactory Government: Moderately Satisfactory Implementing Quality of Supervision: Satisfactory Satisfactory Agency/Agencies: Overall Bank Overall Borrower Moderately Satisfactory Moderately Satisfactory Performance: Performance:

i C.3 Quality at Entry and Implementation Performance Indicators Implementation QAG Assessments Indicators Rating Performance (if any) Potential Problem Project Quality at Entry No None at any time (Yes/No): (QEA): Problem Project at any Quality of Yes None time (Yes/No): Supervision (QSA): DO rating before Satisfactory Closing/Inactive status:

D. Sector and Theme Codes Original Actual Sector Code (as % of total Bank financing) Central government administration 5 5 Primary education 38 38 Secondary education 38 38 Sub-national government administration 19 19

Theme Code (Primary/Secondary) Education for all Primary Primary Gender Secondary Secondary Municipal governance and institution building Secondary Secondary Participation and civic engagement Primary Primary Vulnerability assessment and monitoring Primary Primary

E. Bank Staff Positions At ICR At Approval Vice President: Shigeo Katsu Johannes F. Linn Country Director: Orsalia Kalantzopoulos Orsalia Kalantzopoulos Sector Manager: Mamta Murthi Maureen Anne McLaughlin Project Team Leader: Keiko Miwa Peter Darvas ICR Team Leader: Sachiko Kataoka ICR Primary Author: Sachiko Kataoka

F. Results Framework Analysis

Project Development Objectives (from Project Appraisal Document) The objective of the project is to improve educational attainment at primary and secondary education levels and to enhance access of vulnerable groups to education.

ii

Revised Project Development Objectives (as approved by original approving authority) The original PDO was not changed. However the PDO and intermediate outcome indicators were revised and made more specific and measurable after the Mid Term Review (MTR) in February 2005 because the original ones did not have specific target values and some of the indicators such as net and gross enrolment rates and completion rates were not measurable due to lack of population census data (the last census in Kosovo was in 1991, and the next one is planned in 2009/2010). See Section 1.2 in the main text for a detailed description of changes in the POD indicators.

During the Mid-Term Review, the Ministry of Science, Technology and Education (MEST) and the Bank team agreed on a set of new, more specific PDO and intermediate outcome monitoring indicators and/or their target values (Kosovo Education Participation Improvement Project: Aide Memoire, Mid-term Review, February 24, 2004).

(a) PDO Indicator(s)

Original Target Formally Actual Value Values (from Revised Achieved at Indicator Baseline Value approval Target Completion or documents) Values Target Years Original: Reduction of dropout rate in the targeted schools. Indicator 1 : Revised: 25 percent reduction of the dropout rate among the ethnic minorities in 75 percent of the targeted schools. Dropout of Roma, Dropout of Roma, Value Ashkalia, Egyptians Ashkalia, Egyptians in quantitative or Reduction 25% reduction in Round 1 schools Round 1 schools Qualitative) 2003/2004: 14% 2002/2003: 15% 2004/2005: 8% Date achieved 08/31/2004 08/31/2004 02/24/2005 12/31/2006 Comments ACHIEVED: The dropout rate was reduced by 43% from 2002/03 (15%) to (incl. % 2004/05 (8%), hence the achievement exceeded the target. achievement) Original: Improvement in the survival rate and completion rate at primary level in targeted schools. Indicator 2 : Revised: 25 % improvement in the survival rate in targeted primary/secondary schools among ethnic minorities in 75 % of the targeted schools. Survival rates of Survival rates of Roma, Roma, Ashkalia, Ashkalia, Egyptians Egyptians 03/04 to 2002/03 to 2003/04: 04/05 (change from Gr 1-2 109%; 02/03 to 03/04): Value Gr 2-3 101%; 25% Gr 1-2 154% quantitative or Gr 3-4 93%; Improvement improvement (+42%); Qualitative) Gr 4-5 90%; Gr 2-3 96% (-5%); Gr 5-6 79%; Gr 3-4 113% Gr 6-7 95%; (+20%); Gr 7-8 96%; Gr 4-5 70% (-20%); Gr 8-9 53% Gr 5-6 148%

iii (69%); Gr 6-7 90% (-5%); Gr 7-8 92% (-4%); Gr 8-9 92% (+39%) Date achieved 08/31/2004 08/31/2004 02/24/2005 12/31/2006 Comments Changes in the survival rates are generally positive, but mixed in some grades. (incl. % These mixed results might be due partially to the high mobility of achievement) students/families in the post-conflict environment and the reliability of data. Original: Improved net and gross enrolment rates to primary education in targeted non-Serb and non-Albanian communities Indicator 3 : Revised: Dropped because there was no population census data to calculate NER/GER Value quantitative or Not collected Improvement Dropped Not applicable Qualitative) Date achieved 08/31/2004 08/31/2004 02/24/2005 02/24/2005 Comments It was agreed to drop this indicator during the MTR due to the lack of recent (incl. % population census data (last census was in 1991). achievement) Original: Higher level transition to and attainment at secondary level especially by girls in the targeted communities Indicator 4 : Revised: 30% of improvement of girls survival rate between grades in secondary education Survival rate of Survival rate of girls in girls in secondary Value secondary schools 30% schools quantitative or 2002/03 to 2003/04 Improvement improvement 2003/04 to 2004/05 Qualitative) Grade 10-11: 95% Grade 10-11: 82% Grade 11-12: 96% Grade 11-12: 82% Date achieved 08/31/2004 08/31/2004 02/24/2005 12/31/2006 Comments The survival rate of girls (and boys) appears to have deteriorated in participating (incl. % secondary schools which could be due to the migration of the population or the achievement) poor quality of data. (See discussion in section 3.2)

(b) Intermediate Outcome Indicator(s)

Original Target Actual Value Formally Values (from Achieved at Indicator Baseline Value Revised approval Completion or Target Values documents) Target Years Indicator 1 : School Development Plans completed in about 300 schools. 514 schools Value 300 schools successfully (quantitative 0 school complete school completed School or Qualitative) development plan. Development Plans. Date achieved 07/31/2003 07/31/2003 12/31/2006 Comments FULLY ACHIEVED: The achievement exceeded the original target by more (incl. % than 70%. According to external evaluation, the implementation resulted in achievement) various achievements including increased public awareness and community

iv participation in education reform. Original: School grant process is implemented in a timely manner. Indicator 2 : Revised: 270 schools implement successfully initiatives to improve infrastructure, heating, hygiene, computers and others. 514 schools 270 schools successfully implement implemented the initiatives to school development Value improve initiatives. Out of (quantitative 0 school Not specified infrastructure, which, 91 schools or Qualitative) heating and implemented the hygiene, second school computers development successfully. initiatives. Date achieved 07/31/2003 07/31/2003 02/24/2005 12/31/2006 Comments FULLY ACHIEVED: The achievement was almost double the target. The (incl. % contributions by communities (Euro 1.2 million, 22% of total school grants achievement) amount) were much higher than expected (5% of total school grants amount). Original: EMIS has established data on dropout, attendance and completion (9 years) is available Indicator 3 : Revised: EMIS has established data on enrollment, gender and other key school indicators EMIS was piloted in 30 schools, its EMIS design was produces Database was established EMIS produces finalized, and the Value enrolment, under the Education dropout, installation in (quantitative gender and Health Project, but attendance and municipalities or Qualitative) other key needed to be improved. completion rates completed. It is in school the process of indicators installation for all schools. Date achieved 07/31/2003 07/31/2003 02/24/2005 12/31/2006 Comments PARTIALLY ACHIEVED: EMIS was redesigned after the Mid-Term Review (incl. % and the new EMIS system can produce data on 19 key indicators. achievement) Agreed and added at MTR: Stakeholders have actively participated in all stages Indicator 4 : of the development of education strategy, and the education strategy is approved by Government. Strategy for Higher Education (2005- Development 15) and Strategy for Value and approval No strategy education Pre-University (quantitative of the existed. Education (2007- or Qualitative) education 17) completed and strategy. approved by the government. Date achieved 02/24/2005 02/24/2005 12/31/2006 Comments FULLY ACHIEVED: The government involved various stakeholders in the

v (incl. % process of developing strategies to reflect their views. The strategies were achievement) approved by the Government, and are used for the preparation of the medium- term expenditure framework.

G. Ratings of Project Performance in ISRs

Actual Date ISR No. DO IP Disbursements Archived (USD millions) 1 08/05/2003 Satisfactory Satisfactory 0.00 2 12/17/2003 Satisfactory Satisfactory 0.12 3 06/22/2004 Satisfactory Satisfactory 0.31 4 12/21/2004 Satisfactory Satisfactory 1.34 5 05/16/2005 Satisfactory Unsatisfactory 1.89 Moderately 6 06/29/2005 Satisfactory 2.01 Unsatisfactory 7 01/25/2006 Moderately Satisfactory Moderately Satisfactory 3.13 8 08/23/2006 Satisfactory Moderately Satisfactory 3.77 9 03/19/2007 Moderately Satisfactory Moderately Satisfactory 4.87

H. Restructuring (if any) Not Applicable

I. Disbursement Profile

vi

1. Project Context, Development Objectives and Design

1.1 Context at Appraisal

Country Background: While a number of sub-regions of the Former Republic of Yugoslavia (FRY) were gaining independence in the 1990s, Kosovo, which also strived for independence from the FRY, faced escalating conflicts with Serbia, resulting in air strikes by NATO between March and June 1999. Following NATO’s attack, UN Security Council Resolution 1244 was adopted in June 1999, and Regulation No. 2001/9 on a Constitutional Framework for Provisional Self-Government in Kosovo created in May 2001 the post-conflict political and constitutional framework with the Provisional Institutions of Self-Government (PISG) that included the President of Kosovo, the Assembly and the Government. While the PISG had progressively more responsibility in governing Kosovo, the Special Representative of the UN Secretary General (SRSG) remains the final legal authority.

During the 1990s, Kosovo’s economy experienced a significant economic decline, firstly due to massive decline of investment and neglect, and secondly due to the disruption and destruction of infrastructure and productive capacity caused by the 1999 conflict. After an initial post-conflict acceleration in 2000, economic growth has weakened in line with declining donor resources, and Kosovo has continued facing many of the same structural issues as other poor, rural, transition economies in an environment of declining donor funding. The Bank's Poverty Assessment 2001, based on the Living Standard Measurement Survey (LSMS), found that over half of the population was living below the poverty line (PL) and around 12 percent below the food poverty line (FPL).1 Typically, the households in extreme poverty were those of Albanian ethnicity, working in agriculture in a rural area with a high dependency ratio,2 and headed by a person with a low level of education.

Sector Background: In Kosovo, approximately 88 percent of the total population of about 2 million is Albanians.3 However, the centralization of the education system by Belgrade, the Serbian capital, during the 1990s compelled the Albanians in Kosovo to create an unofficial, or so-called ‘underground’ education system in the Albanian language supported by communities, in parallel to the formal system in the Serbian language. During this period, funding and the availability of educational inputs to the community- based Albanian system were very limited, and teachers were unable to update their teaching skills and methodologies. The 1999 conflict further disrupted both the formal and

1 PL allows for the household’s need to consume goods other than food (DM 3.499, at 2001 price) per adult per day. FPL is defined as the cost of the minimum food basket to provide 2,100 calories per adult (DM 1.8539 per adult, per day). A household is considered 'extremely poor,' if its consumption is below FPL. 2 The dependency ratio is equal to the number of individuals aged below 15 or above 64 divided by the number of individuals aged 15 to 64, expressed as a percentage.

3 According to government estimates, the population of Kosovo consists of, approximately, 88 percent Albanians, 7 percent Serbs, 1.9 percent Bosniaks, 1.7 percent Roma, 1 percent Turks, and 0.5 percent Gorani.

1

informal education systems. Moreover, after the war, schools were required by law to establish functional school boards, but most schools were unable to mobilize parents and communities and to establish a board, which limited their capacity to efficiently and effectively manage themselves and to provide good quality education for children.

The overall education attainment (defined as the highest grade achieved for adults who have left school) in Kosovo was low, compared to neighboring countries. In addition, educational attainment and enrolment rates varied widely between ethnic groups, gender, geographical locations and income levels. According to the LSMS 2001, the primary net enrolment rate for the entire population of Kosovo was around 97 percent with little gender gap, but only 77 percent for non-Serb ethnic minorities which recorded a wide gender gap (85 percent for boys and 69 percent for girls). At the secondary education level, the gender gap was wide for all ethnic groups: in 2001, the net enrolment rates were 74 percent for boys and 56 percent for girls among Albanians, 87 percent for boys and 93 percent for girls among Serbs, and 54 percent for boys and 40 percent for girls among non-Serb ethnic minorities.

Education administration, like the rest of public administration, was (and still is) at transition. In September 2002, the responsibility for provision of education was transferred from UNMIK to the PISG Ministry of Education, Science and Technology (MEST), with UNMIK playing a supportive, capacity building role. In May 2003, the Ministry of Education, Science and Technology (MEST) developed the Strategic Plan for the Development of Education in Kosova (2003-2007), which aimed at moving the Kosovar education from its emergency phase to consolidation.

Rationale for Bank Assistance: The Bank’s Transitional Support Strategy (TSS) was prepared in 2002. The program for CY2002-2003 aimed to support post-conflict reconstruction, economic recovery, poverty alleviation and human development, fiscal sustainability, good governance, peace-building and stabilization of the region as part of a broader international effort led by the United Nations Interim Administration Mission in Kosovo (UNMIK). Within this overall support strategy, it was considered indispensable that the Bank builds on and continues supporting the process of institutional building, decentralizing education financing and administration and strengthening and reinforcing the sustainability of the Education Management Information System (EMIS) initiated by the first phase of support in education through the Education and Health Project (EHP) (May 2000- June 2004). While good progress was made under the EHP, continuous support for the institutional building of the education sector to address the issue of low educational attainment was considered critical in Kosovo. The second phase of assistance through the Education Participation Improvement Project (EPIP) aimed to tackle these issues by providing schools with school development grants accompanied by training on school-based planning and management, strengthening the role of school boards, participation of municipalities in the school grant selection process, and improvement of the EMIS for monitoring and evaluation.

2

1.2 Original Project Development Objectives (PDO) and Key Indicators (as approved)

The original PDO was to improve educational attainment at primary and secondary education levels and to enhance access of vulnerable groups to education. The original outcome indicators were: (i) reduction of dropout rate in the targeted schools; (ii) improvement in the survival rate between grades and completion rate at primary level (currently nine years) in the targeted schools; (iii) improved net and gross enrollment rates for primary education in targeted non-Serb and non-Albanian communities; and (iv) higher level transition to and attainment at secondary level especially by girls in the targeted communities. The original output indicators were: (i) School Development Plan completed in about 300 schools, (ii) school grant process is implemented in a timely manner; (iii) EMIS has established data on dropout, attendance and completion (9 years) is available; and (iv) enrollment rates, dropout rates, average number of years in the education system.

1.3 Revised PDO (as approved by original approving authority) and Key Indicators, and reasons/justification

The original PDO was not changed, but the monitoring indicators (both outcome and output indicators) were revised and made more specific and measurable during the Mid Term Review (MTR) in February 2005 because the original ones did not have specific target values and some of the indicators such as net and gross enrolment rates and completion rates were not measurable due to the lack of population census data (the last census in Kosovo was in 1991, and the next one is planned in 2009/2010). The revised outcome indicators were: (i) 25 percent reduction of the dropout rate among the ethnic minorities in 75 percent of the targeted schools; (ii) 25 percent improvement in the survival rate in targeted primary and secondary schools among the ethnic minorities between the grades (where feasible until Grade 9) in 75 percent of the targeted schools by the beginning of the next academic year following the completion of the school based planning and grant activities; and (iii) 30 percent of improvement of girls survival rate between grades in secondary education. The revised output indicators were: (i) School Development Plan completed in about 300 schools; (ii) 270 schools implement successfully initiatives to improve infrastructure, heating, hygiene, computers and others; (iii) EMIS has established data on enrollment, gender and other school key indicators; and (iv) stakeholders have actively participated in all stages of development and then the strategy is approved by Government.

1.4 Main Beneficiaries

ƒ Through the improvement of access and attainment, the project was to primarily benefit the poorest, those who lived in rural areas, girls, and non-Albanian, non-Serb minorities, who, in general, had more limited access and lower attainment; ƒ Through the improvement of effectiveness, quality and relevance, the whole student population in the selected schools was to benefit; and ƒ Through the improved planning, management and strengthened participation, local stakeholders were to benefit from the project.

3

1.5 Original Components (as approved)

Component I: School Development Grants (SDGs) (Total est. cost: US$ 3.79 million) The objective of the SDG component was to provide funding to schools to help increase enrollment, attendance, and retention in primary and secondary education. Through the SDG process, approximately US$ 3.4 million in grant finding was to be awarded in targeted small grant contracts to approximately half of the 550 primary schools and 110 secondary schools in Kosovo's 30 municipalities. Primary schools were to be eligible for grants up to US$10,000 and secondary schools were to be eligible for grants up to US$ 15,000. The grants were to be awarded on the basis of school development plans and grant proposals, which were to be evaluated by Municipal School Grant Teams (MSGTs) and approved by a central School Grants Approval Board (SGAB). All schools selected for the program were to receive technical assistance and training in the areas of school development planning and proposal writing. Schools were to be selected for the program by municipal education officials on the basis of documented access and retention problems in the school's catchment area. Selection criteria were to focus largely on enrollment, dropout, and attendance statistics. Grant proposals, prepared by the selected schools, were to be evaluated by the MEST on the basis of how effectively they address access and retention problems. The grants were to finance training, community outreach activities, educational materials and school supplies, and some small works aimed at improving security and the educational environment. Random special purpose audit of the school grants was to be carried on a yearly basis to ensure that the school grants were being used for the purposes intended.

Component II: Improving the Education Management Information System (Total est. cost: US$ 0.30 million) The objective of this component was to help strengthen and ensure the sustainability of the existing EMIS, which had been designed and initiated under the EHP. More specifically, the EMIS was to help monitor enrollment, attendance, repetition, dropout and completion rates, as well as try to account for children in the targeted municipalities, who were not participating in the school system. The Project was to finance local and international technical assistance, external training, and limited equipment and furniture of central level and municipal offices.

Component III: Project Management (Total est. cost: US$ 0.51 million) The objective of this component was the effective implementation, monitoring and evaluation of the Project. The implementing agency was the MEST. A small team of professional staff was to form a Project Management Unit (PMU) for the implementation of the Project. The PMU was to be headed by a Project Director who had been hired under the EHP as Deputy Director. He was receiving on-the-job training from an international Project Director of the education component of the EHP. The Project Director recruited for the proposed Project had a background and experience in financial management. Most of the PMU staff had already been contracted under the EHP and were involved in the preparation of the Project. The team that worked on the EHP for implementation of the Project was to be maintained to ensure continuity and preserve the skills and knowledge gained under the ongoing project. The PMU was also to be responsible for monitoring and evaluation of

4

results under the school development grants component. The component was to finance technical assistance, limited computer and office equipment, and training of staff.

1.6 Revised Components There was no change to the original components.

1.7 Other significant changes It was agreed during the MTR in February 2005 that the PMU would be restructured because its management and procurement capacities were inappropriate. Between June and November 2005, the MEST re-advertised the staff positions in the unit, and transformed it as the Project Coordination Unit (PCU).4 As the name suggests, the new PCU became more a coordinating rather than managing unit, closely supervised by the MEST. The MEST also extended the contract with the facilitating agency for the SDG component, Kosovo Education Center (KEC), and strengthened the role of Regional Outreach School Officers (ROSOs) in the grant implementation and monitoring. The management responsibility for the EMIS component was also transferred from the PMU to the MEST.

2. Key Factors Affecting Implementation and Outcomes

2.1 Project Preparation, Design and Quality at Entry

Project Preparation Soundness of the background analysis: The education chapter of the Medium-Term Public Expenditure Review (October 2002) and the experience of the Education and Health Project (EHP) among others provided a good analytical base for the project preparation and design. The former identified low educational attainment (the grade level completed) especially at the secondary level and the gaps in attainment depending upon the household income level, ethnicity and gender, as one of the key issues to be addressed in Kosovo. It also pointed out the importance of local participation and accountability in education. The Education and Health Project (EHP) supported the government to implement the new Law on Primary and Secondary Education in Kosovo, No. 2002/19, to establish the principles of block grant funding for municipalities based on a per student funding formula, and to start developing EMIS.

Given the institutional and legal framework developed under the EHP, it was an appropriate decision that the EPIP aimed to sustain the achievements of the EHP and further strengthen capacities of schools and the monitoring and evaluation (M&E) system as the second phase of the development of the education sector. In order to develop a model for the new school grant scheme, the EHP piloted a School Grant Project in 10 selected schools, and established the institutional framework for the scheme, i.e., the School Grants Approval Board (SGAB) at the central level and the Municipal School Grants Team (MSGT) at the municipality level.

4 The Acting Project Coordinator was appointed in June 2005, who was formally appointed for the position in November 2005. Other staff, including Financial Management officer, education specialist, and IT expert, were also appointed by November 2005.

5

This preparation work was appropriate and effective for the final design of the EPIP SDG component. The capacity of the EMIS Section of the MEST was also assessed with the assistance of a local IT consultancy firm, which demonstrated MEST’s ability to collate credible formatted information from municipalities and schools and to develop software tools, and that the system was able to produce data required for school selection.

Overall Project Design: The EPIP was designed at an uncertain and transitional time in Kosovo, and such a context was fully taken into consideration. The project was appraised just after the Department of Education, Science and Technology (DEST) of the UNMIK was transformed into the Ministry of Education, Science and Technology (MEST) in December 2002. The 2001 Constitutional Framework for the PISG had not clearly specified the division of responsibilities between the UNMIK and MEST, however. Neither was sufficient capacity building support provided from the UNMIK to the MEST prior to the transfer of responsibilities. Given the weak capacity of the MEST and the government in general and the needs for the second phase of the development of the education sector as discussed above, it was highly appropriate to design a simple project focusing on the strengthening of school management capacities with 85 percent of the investment allocated to the School Development Grants component and the rest to supporting activities under the EMIS and Project Management components. This simple and focused project design contributed to the timely completion of the project without any extension. In addition, community participation in post-conflict reconstruction of education (or in other sectors) is well recognized as critical and instrumental,5 and therefore, was another appropriate element in project design.

Lessons from the School Grant Pilot Project: In principle, the recommendations made in an evaluation report of the School Grant pilot project6 were reflected in the design of the SDG component. The flexible and demand-driven design allowed schools to analyze their financial, physical and pedagogical situations, to identify and prioritize their needs, and to prepare a school development plan and a grant proposal on their own for the first time. It was also crucial that community participation was encouraged by requiring schools to mobilize financial and/or in-kind contributions from parents and communities (minimum five percent of the school grant). The design of SDG could have been further strengthened if the following recommendations were more vigorously reflected: (i) to include procurement and monitoring training for schools in the initial training program (the issue of procurement training was addressed after one year of project implementation); and (ii) to encourage the participation of local NGOs at the municipality level in the preparation and implementation of SDG activities (when/if competent local NGOs at municipal level are available).

Assessment of Risks and Mitigation Measures: Critical risks were identified and assessed adequately. While some of the mitigation measures were appropriate and indeed minimized risks, others were not sufficient or adequate to address the identified risks. Participation of schools, parents, communities and municipalities in the planning of school grant process was

5 Reshaping the Future: Education and Postconflict Reconstruction. The World Bank 2005.

6 School Grant Pilot Project Report: Final Report (February 2004)

6

encouraged and ensured by the provision of sufficient training in school-based planning capacity to prepare and implement a proposal and required involvement of municipality and school members in the process and the agreement for the grant proposal. Although weak monitoring of enrollment and attendance by municipalities and schools was recognized as a potential risk, training in and dissemination of data collection and analysis were not sufficiently incorporated in the project design.

In sum, a simple, focused and demand-driven project design promoting the participation of school boards and communities took into account various key factors in Kosovo then: (i) the post-conflict uncertain and transitional context of Kosovo, (ii) the weak capacity of the government and the newly established MEST, (iii) the opportunity to capitalize upon communities’ enthusiasm for education, (iii) achievements and lessons from the EHP including findings from the pilot project for the SDG component. The quality at entry could have been further strengthened if lessons learned under the EHP were reflected more vigorously in the detailed project design, and some risk mitigation measures were more rigorous.

2.2 Implementation

Communication Strategy and Information about the Project Activities: Communication and information sharing with the key stakeholders about the project objective and activities was effective. While the Project was going to cover half of the primary and secondary schools in Kosovo, the Regional Outreach School Officers (ROSOs) presented the objectives of the EPIP to all school directors in all regions before the school selection process started. This wide dissemination of the project information for the broader audience as well as the establishment of the Municipal School Grants Team (MSGT) and clear presentation of selection criteria at the municipality level contributed to gaining a good understanding and ownership of the project. There was little, if any, negative response to the school selection even from non-selected schools because key stakeholders understood the objectives of this project, and agreed to prioritize poor schools first.

The MEST invited all of the Municipality Education Directors (MEDs) to a dissemination seminar for the SDG activities and provided them with the Project Operating Manuals, including procurement guidelines and SDG manual. Some MEDs were unable to participate in the seminar, and other key municipal officers such as the Financial Officer of the Municipality Education Directorate and the Procurement Officer of the Procurement Department were not well informed about the procurement rules under the project at the outset. This sometimes resulted in poor cooperation from procurement officers, particularly for the introduction of school-based procurement. Stronger engagement of these key municipal officials from the early stage of the project implementation would/could have helped in smoother implementation at the beginning. Nevertheless, this issue was raised during the first phase of project implementation, and the procurement training was provided to MEDs and communication with finance officers was improved before the MTR.

Education Management Information System (EMIS): When the MSGTs tried to collect data required for school selection, it was found out that the EMIS, which had been designed

7

and initiated under the EHP, was unable to produce necessary school statistics and population data, or to monitor enrollment, attendance, repetition, dropout and completion rates. In order to substitute the EMIS data, the PMU/PCU and KEC collected quantitative data already available such as statistical reports on attainment and attendance, Regional Education Office (REO) reports on dropouts, and reports on poverty from the Center for Social Work (Social Welfare Officer) for school selection. The MSGTs also involved various stakeholders, including civil society members and parents, REO, ROSO, and MED, in the school selection process to compensate for the lack of reliable population and poverty data. For monitoring, the PMU/PCU and KEC also collected qualitative evidence from project schools.

Mid Term Review (MTR): From Project Management Unit (PMU) to Project Coordination Unit (PCU): Even though the PMU did not function well in the beginning because of a lack of management capacity, the MEST showed stronger ownership over the Project and made an important decision to restructure the PMU as the PCU, as discussed above. The new PCU members, in closer coordination with the MEST, were committed and competent in implementing the project. The redefinition of the unit’s roles and responsibilities and the extension of the Kosovo Education Center’s (KEC’s) role successfully contributed to the revitalization of project implementation.

EMIS: The September 2004 Aide Memoire commented that the EMIS implementation was unsatisfactory as the EMIS team was unable to produce reports and data were unreliable. This status was confirmed during the first phase of the MTR mission in December 2004. While the then-TTL and the PMU decided to revise the monitoring indicators, and tasked the KEC to collect necessary data from schools through ROSOs, the MEST took over the responsibility for the EMIS from the PMU, established a Working Group on the EMIS and started a reform process in January 2005. Owing to these efforts, the EMIS started to make considerable progress after the MTR. The new EMIS system was designed to produces 19 key indicators,7 eight of which were in compliance with the international ranking defined by the UNESCO Institute of Statistics (UIS). The pilot test of the new system was completed and nationwide installation and training of schools and municipalities began towards the end of the project implementation period, but is not complete yet.

CIDA Funds: Because of the successful implementation of the SDG component of the Project and its positive impacts on school development, the Canadian International Development Agency (CIDA) signed a grant agreement with the Bank to co-finance the EPIP at the end of 2004. The total amount was US$900,000, consisting of $150,000 for

7 Includes students per class, repetition rate, percentage of repeaters, survival (promotion) rate, attendance rate, drop-out rate, percentage of female students, transition rate to tertiary education, transition rate to secondary education, student teacher ratio, class size, share of students attending school in different shifts, share of unqualified teachers, share of teachers without qualifications for the subject they teach, share of teachers needing specific pedagogical or subject training, share of teachers receiving specific pedagogical or subject training, percentage of female teachers, percentage of school space requiring immediate renovation and per student expenditure.

8

consulting services and $750,000 for school grants. CIDA’s contribution enabled the Project to provide the SDG for about 100 additional schools.

In sum, the wide dissemination of the project information helped gain consensus on school selection at the municipal level, and built a strong ownership of the project by key stakeholders. The success was recognized by a development partner, and mobilized additional financing. The MTR successfully encouraged the government to take stronger ownership over the project implementation. The project completed within the original time frame, and reached more number of schools than originally planned.

2.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization

M&E design: The M&E design is discussed in terms of: (i) the original monitoring indicators, (ii) the revised monitoring indicators after the MTR, and (iii) a linkage from the inputs to the outcomes. First, as illustrated in the Data Sheet, “F. Results Framework Analysis” above, the original monitoring indicators were not well specified in terms of target values, and some indicators such as net and gross enrolment rates were immeasurable because relevant population data was not available in the post-conflict situation. Second, the revised monitoring indicators were better specified, but some of the target values might have been ambitious given a short time frame of the project (a little over three years). Third, even with the revised indicators, it would still have been difficult to isolate the impact of the project alone and measure short-term and long-term outcomes of the SDG without conducting a rigorous impact evaluation study because the SDG has provided a relatively small amount of one-time grants for each school, and many other factors could have influenced the outcomes.

M&E implementation: PMU/PCU took three key actions to address the inability of EMIS to produce some of the quantitative data (especially the ones which require population census to estimate the ratio). First, PMU/PCU and the facilitation NGO, KEC adopted collection methods of quantitative data which do not depend on EMIS. The KEC started collecting data to produce indicators from schools which were participating in the SDG activities. The statistics used in F. Results Framework Analysis above are based on this data source. The project team’s flexible and practical response to the constraints of EMIS is highly commendable. Primarily due to high mobility and migration of families and students, it has been extremely difficult to track enrolment, attendance, drop-out and completion rates of each school. In addition, the absolute number of minority students or students in targeted schools is rather small that capturing changes/trends in percentage terms could possibly be misleading.

Second, given the difficulties in collecting reliable quantitative data, the PMU/PCU and KEC also collected qualitative data from schools participating in SDG activities as part of their regular monitoring activities. They interviewed school principals, teachers, students and parents and observed the impacts of SDG projects in school. The PMU/PCU regularly produced a narrative report twice a year, which described achievements through the SDG activities.

9

Third, the MEST took the initiative to carry out an independent external evaluation of the project to: (i) assess the main outcomes of the EPIP, in particular those of the SDG component, and (ii) to draw a number lessons relevant for education policy and future initiatives (see Annex 6 for key findings).8 An international consultant, who is familiar with the education systems in the region and Kosovo, together with two national consultants carried out the evaluation through: (i) administration of survey questionnaires with a sample of participating schools, (ii) field visit to schools, (iii) interviews and focus-group interviews of key stakeholders (officials of MEST MED, REO, ROSOs, school principals, teachers parents and students), and (iv) desk review of existing reports and data.. The comprehensive analysis of the SDG component by the external evaluators provides deeper understanding of the quality of project implementation and lessons for future project designs.

M&E utilization: In addition to monitoring and evaluation of the quantitative indicators, the PMU/PCU together with the ROSOs visited project sites and provided technical support for schools and municipalities regularly. As discussed in the previous section, findings from these visits were documented in progress reports, and used to improve project implementation. As discussed above, the government conducted an external evaluation of the SDG component. These monitoring and evaluation exercises informed the government of the effectiveness of the SDG and the ways forward to improve the program.

2.4 Safeguard and Fiduciary Compliance

Safeguard management performance: The environmental category for this project is C, and therefore, no environmental assessment was necessary.

Procurement: Taking into account the weak capacity within the PCU/MEST, an international procurement advisor was hired. This significantly improved and expedited the procurement under the project. Overall, the procurement under the project was rated moderately satisfactory

Two post reviews were conducted in December 2003 and June 2006. The review observed that schools impressively carried out the procurement in line with the agreed procedures in the procurement handbook, and with the participation of a representative from parents, teachers and school management. The proper application of procurement procedures was attributable to ROSOs who according to the schools were highly committed to provide timely advice. The filing system of the procurement documents could have been improved for audits or reviews by the Bank, however.

Audit Reports: Project financial management was handled satisfactorily. The financial monitoring reports were prepared and submitted on time. Project financial statements were audited on an annual basis by independent auditors and all audit reports had unqualified (clean) opinions. Financial management supervision was carried out in two main ways: (i)

8 Crisan, Alexandru, Rexhaj, X. and Saqipi Blerim. External Evaluation: School Development Grants (SDG) Component (December 2006).

10

review of the project’s quarterly financial monitoring reports as well as the project’s annual audited financial statements and auditor’s management letters; and (ii) during the Bank’s supervision missions, review of the project’s financial management and disbursement arrangements to ensure compliance with the Bank's minimum requirements. During the implementation of the project, the PIU was trained in all aspects of disbursement and financial management after taking over the responsibilities that were originally carried out by the Grant Unit in the Ministry of Economy and Finance.

2.5 Post-completion Operation/Next Phase

Building on the success of the SDG, the MEST and stakeholders who were involved in the development of the Strategy for the Development of Pre-University Education in Kosovo: 2007-2017 decided to include SDG as one of the priority activities in the education sector. Such “mainstreaming” in the education strategy is expected to contribute to the sustainability of SDGs.

Recognizing the critical importance of SDGs and education in the new status of Kosovo, CIDA approved a follow-up education project, EPIP-II (CA$ 500,000) in March 2007, with a renewed and stronger focus on quality improvement and social inclusion. It is designed to provide SDG to schools with a high proportion of students from minority groups and poor families, strengthen the EMIS, and finance the Project Coordination Team (PCT) which will manage the implementation of SDG and EMIS components. In addition, the Post-Conflict Fund approved a contribution of US$704,000 in May 2007 to provide SDG for more schools, and to strengthen school services at the municipality level. To enhance the sustainability, the amount of grant to each school is smaller than the EPIP-I, and communities are expected to contribute more when they can. The Bank is currently supporting the MEST to prepare a new education project which builds on the EPIP and to support the implementation of the Pre-University Education Strategy 2007-2017.

3. Assessment of Outcomes

3.1 Relevance of Objectives, Design and Implementation

Objectives: The direction of the original Project Development Objective (PDO), aiming higher educational attainment and improved access of vulnerable groups, was appropriate and relevant at the time of project preparation because the attainment level, especially at secondary level, in Kosovo was lower compared to other neighboring countries. While the emergency reconstruction of school facilities had made some good progress immediately after the war, the government still needed to ensure that all students have access to primary and secondary education. The PDO remains relevant today because enrollment in secondary education, particularly among ethnic minorities and girls, remains as an issue. The PDO indicators were rather ambitious for a 3-year project because attainment (defined as the highest grade achieved for adults who have left school) includes the entire adult population which is not covered by the project, and schools alone cannot easily tackle attainment and access issues for vulnerable groups even with the improved management capacity at the school level.

11

Design: As discussed above, the simple and focused project design was extremely relevant when the capacity of the Implementing Agency was still in a flux in a post-conflict environment. The strong emphasis on participation of school boards and community was also instrumental in the post-conflict context of Kosovo where the enthusiasm and demand for education by communities was high. Requiring a minimum of five percent contribution to school grants by communities was an effective way to increase their ownership. As will be discussed later, the contribution from the communities exceeded the original expectation. The demand-driven SDG component was particularly appropriate to make use of the limited education budgets by developing the capacity of schools to better manage themselves with the help of communities and parents. The project design included capacity building of school directors and municipal officials in planning and prioritization decision-making, establishment of the MSGT and SGAB, creation of functional communication and interaction between municipalities and schools, and institutionalization of the EMIS functions. This contributed to strengthening the foundation of the education system, which was crucial for Kosovo’s education system to focus more on quality improvement in education.

Implementation: Because it was the first time for the government to introduce the SDG scheme, the school selection process was carefully designed to ensure that intended beneficiaries would be selected. First, after the MSGT pre-selected beneficiary schools, the list of schools was approved by the SGAB. Upon SGAB’s approval of the pre-selection, the selected schools started preparing school development plans, which had to be approved first by the MSGT, and then by the SGAB. The Regional Outreach School Officers (ROSOs), officials of the Municipality Education Directorate, and PCU members among others pointed out that the procedures for school selection and proposal approval were too long and repetitive, and caused delays in the implementation of SDG activities especially during the first year. For instance, the ROSOs made best efforts to train schools to prepare proposals on time, but delays in proposal approval and disbursement of grants resulted in loss of momentum among schools and communities. This issue was addressed during project implementation, and the approval process became more timely from the Round 3 onwards.

Under the regular government budgetary system, schools are not allowed to have a bank account or a sub-account under the MED for the management of government budgets. Teachers and other staff receive their salary directly to their personal bank account. In other words, schools generally do not know how much budget they are allocated for teacher salaries, educational materials, utilities, repair and maintenance, and other recurrent costs. As a special arrangement under the EPIP, SDG-recipient schools were allowed to open a sub- account under the MED to ensure that the grants were allocated to and spent by recipient schools. This sub-account system gave schools more flexibility and transparency of the budget than the regular system.

3.2 Achievement of Project Development Objectives

The PDO was to improve educational attainment at primary and secondary education levels and to enhance access of vulnerable groups to education. In terms of attainment, while the dropout rates for ethnic minorities decreased by 43 percent from 2002/03 (15 percent) to

12

2004/05 (8 percent). Survival rates for ethnic minorities were mainly positive, but varied among different grades. Survival rates for girls (and boys) in secondary schools seem to have deteriorated. The mixed results on survival rates may be due to a lack of reliable data and the high mobility of students in the post-conflict environment. In terms of access, some schools experienced a 400 percent increase in enrolment in one year, but it does not necessarily mean that a large number of out-of-school children started enrolling in those schools, but there might have been a large inflow of families into this neighborhood. Likewise, negative increases in enrolment do not necessarily mean that more children are dropping out of school. Hence, without carefully tracking each individual student, it is extremely difficult to measure enrolment, attendance, drop-out and transition rates in a highly mobile society particularly after the war.

As the Project covered more than half the schools in Kosovo, the overall reduction in drop- out rates from 1.67 percent in 2002/2003 to 1.17 percent in 2004/2005 for primary and lower secondary, and from 2.32 percent to 2.07 percent for upper secondary, as shown in Table 1, may suggest that the Project might have contributed to this reduction. Given that the project covers more than half the schools in the country, national-level data may be more suitable to assess its overall impact. Moreover, national-level data is less affected by internal migration compared to school level data.

Table 1: Primary and Lower Secondary Education Number of Students and Drop-outs

No. of Drop-outs students Total % Boys % Girls % Primary and Lower Secondary 2002/03 340,257 5,682 1.67 3,075 54 2,607 46 2003/04 354,918 5,703 1.61 2,654 47 3,049 53 2004/05 355,339 4,141 1.17 1,932 47 2,209 53 Upper Secondary 2002/03 66,968 1,554 2.32 992 64 562 36 2003/04 68,361 2,433 3.56 1,570 65 863 35 2004/05 68,976 1,428 2.07 980 67 446 33 Source: Progress Report July-December 2005

The external evaluation report argues that SDG had a very important impact on institution building of the education system and other positive and unintended effects for the sector, which is expected to contribute to the achievement of the PDO, increasing education attainment including for vulnerable groups.9 This assessment is supported by the

9 Specific examples include: introduction of computers and other electronic equipment such as language learning facilities and radio station made schools a more interesting place to be and students are more attracted to these classes and school in general; and in one of the 10 selected pilot schools, which introduced a transportation service, attendance of pupils increased by 7 percent and 5 pupils who had dropped out returned to complete their education. Moreover, even though many of the SDG activities

13

implementation completion report prepared by the MEST, PMU/PCU’s progress reports, and discussions with school principals, government officials, UNMIK officials and other international donors who almost unanimously praised this project as a success during the ICR mission. The project has certainly built capacities of schools to prioritize, plan and implement school projects and to mobilize community participation. It has also contributed to “a thorough public awareness raising concerning the importance of education reform for any dynamic society; the development of a system of functional and valuable grass-roots processes that can be considered a prerequisite for any further educational decentralization; a number of relevant evolutions in the functioning of central, municipal and school administration; and a general increase of public and community participation in education.”10

Targeting: Once the resources were allocated to municipalities, the MSGTs prioritized schools in poor conditions and those with a higher proportion of vulnerable groups such as ethnic minorities, poor, girls and disadvantaged children as grant recipients. Therefore, even though not all SDG activities directly targeted the disadvantaged children, a higher proportion of those children in each municipality benefited from the project due to the school selection favorable to them. The project also successfully benefited other targeted beneficiaries, including all children and local stakeholders in the selected schools through the improvement of planning, management and strengthened participation for improved school environment and teaching and learning quality.

Coverage: The Project originally targeted to provide grants for approximately 275 primary and 55 secondary schools. After the target was met by Round 4, the SGAB discussed whether successful schools in the earlier Rounds should be awarded grants for the second time, or to include a new set of schools in Round 5. The decision was made to cover more schools in Round 5, but also successful repeaters in Round 6. As summarized in Table 2, a total of 604 projects were carried out and 90 schools implemented the second project in the 6th round. The total number of students who benefited from the Project was around 300,000. The number of total recipients was much more than the original target. The types of project included (i) improvement in learning environment - infrastructure, (ii) computers, musical instruments, and equipment, (iii) textbooks and clothes, (iv) teacher training, and (v) transportation. Many of the improvements in learning environment concerned basic and critical infrastructural needs such as sanitation facilities, water supply, and roofs. This was inevitable as many schools in Kosovo still lack basic infrastructure. After guidance from the SGAB, the percentage of infrastructure improvement slightly declined in Rounds 4 and 5, and again increased in Round 6.

might not have explicitly targeted girls as main beneficiaries, some activities benefited girls more than boys. For instance, a school director succeeded in bringing back female drop-outs in secondary school by encouraging them to attend computer courses. A primary school set up fences for school boundary and improved security condition, which encouraged some parents to send their daughters to school. Transportation service and provision of textbooks and winter clothes enabled children in rural areas and/or from poor families to attend school.

10 Crisan, Alexandru, Rexhaj, X. and Saqipi Blerim. External Evaluation: School Development Grants (SDG) Component (December 2006).

14

Table 2: Percentage of SDG projects

Round Round Round Round Round Round 6 Total 1 2 3 4 5 Total no. of schools 96 107 102 108 101 90 604 Improvement in learning environment 58.2 70.2 62.2 38.0 50.0 62.0 55.8 (infrastructure)1 (%) Computers, musical instruments, equipment 27.6 13.8 38.8 38.0 38.6 20.8 29.6 (%) Textbooks and clothes (%) 16.3 7.4 5.1 23.1 18.8 14.2 14.1 Teacher training (%) 6.1 1.1 2.2 0.9 5.0 0.0 2.6 Transportation (%) 3.1 7.4 3.1 0.0 1.0 2.0 2.7 Source: PCU, Project Monitoring and Evaluation (June and November 2006) 1 Included sanitation facilities, fences, roofs, water supply, sports grounds and classroom renovation.

The expansion of school coverage rather than repeating the school grants in disadvantaged schools had a positive impact on the entire school system. Because this Project was seen by then as a successful and important project to help schools build their management capacity and mobilize community for school development, the MEST wished to provide this opportunity to more schools. The wider coverage with a smaller amount for each recipient was a better decision from a sustainability point of view because it prevented a limited number of schools to depend on the government support too much.

3.3 Efficiency

Net Present Value/Economic rate of return: The nature of the Project, in which 85 percent of the total project budget was provided for public schools as grants, makes conventional measures of efficiency in terms of NPV/ERR not applicable. Instead, what can be assessed is cost effectiveness – whether the school grant scheme was a cost effective way of improving student participation in comparison with other alternative approaches as debated in the PAD.

First, cash transfers to compensate for out-of-pocket expenditures for poor children was considered but rejected, because other factors such as gender and location were more important reasons for dropouts than poverty. Some schools indeed succeeded in retaining girls by building school fence and others improved attendance by providing transportation facilities, textbooks or winter clothes. The school grant scheme, therefore, allowed schools to flexibly adopt effective measures suitable for their children.

Second, the provision of more physical access such as roads and transportation and/or more schools and classrooms is considered not suitable for this project. Given the size of grants and objectives of the project, it was a right decision not to invest in new school buildings, and some schools were able to arrange small scale transportation facility within the school grant scheme.

15

Third, the school grant played a catalytic role, and mobilized Euro 1.2 million as contributions from parents and communities in addition to Euro 3.8 million provided by the Project. While the requirement by the project operational manual was a minimum of 5 percent contribution by communities, communities mobilized as much as 22 percent of the total SDG program. The school grant approach was effective in enabling schools to develop their own strategies to help children participate in school. While the allocation formula could have been more flexible to take into account the level of school needs, the simple and transparent formula was perhaps the best way when introducing a new system with the weak management capacity at the central and municipal levels.

Financial rate of return:

No financial rate of return was calculated for this project.

3.4 Justification of Overall Outcome Rating Rating: Moderately Satisfactory

The overall outcome is rated moderately satisfactory for the following reasons. First, one of the key PDOs – reduction of dropout among ethnic minorities – has been achieved, and most of the intermediate outcome indicators have been achieved. SDGs have considerably contributed to institution-building in the sector, especially at the school to think about and address the issue of educational attainment. As the independent evaluation recognizes, the project contributed to institutional changes, including strengthening the foundation of the education system through capacity building of school directors and municipality officials in planning and prioritization decision-making, establishment of the MSGT and SGAB, creation of functional communication and interaction between municipalities and schools, and institutionalization of the EMIS functions. These institutional changes and positive achievements are extremely important outcomes for achieving the PDO and further development of the education system in Kosovo.

Second, the project aimed to address the critical issue of educational attainment especially for low-income groups, ethnic minorities, and girls with a simple and focused project design which was appropriate for the post-conflict and uncertain context of Kosovo. The project objective remains relevant today since human capital (better educated population) is considered to contribute to the post-conflict reconstruction, poverty reduction and growth in Kosovo. Indeed, the government recognizes education as one of their top priorities and plans to increase its public expenditure, and education is likely to be the focus of the next Bank’s interim support strategy.

Third, the project achieved all the intermediate objectives, and often exceeded the targets, without any additional resources. In addition, the project mobilized more resources from communities than originally anticipated. In some cases, the amount of contributions made by communities for SDGs was more than the grant allocated by the project.

In sum, despite some weaknesses in the quantitative evidence, the overall outcome of the project is rated moderately satisfactory because the PDO was and still is relevant, the project

16

achieved various positive results especially in institutional building which will contribute to the achievement of the PDO, and most of the intermediate outcome indicators were achieved in an efficient manner.

3.5 Overarching Themes, Other Outcomes and Impacts (a) Poverty Impacts, Gender Aspects, and Social Development

As discussed in Section 3.2 above, schools in poor conditions and those with a higher proportion of vulnerable groups such as ethnic minorities, poor, girls and disadvantaged children were prioritized for the SDG program. Therefore, even though not all SDG activities directly targeted the disadvantaged children, a higher proportion of those children in each municipality benefited from the project due to a school selection process favorable to them. In addition, the community participation which is expected to contribute to social development at local level was a hallmark of this project.

(b) Institutional Change/Strengthening

Schools developed planning and management capacities under the SDG component of the project. Through monitoring and supervision activities, a PCU member assessed that about 60 percent of the total SDG-funded schools are now able to produce school development plan (SDP) on their own, while others still need further assistance to develop their capacities. Furthermore, about 30 percent of the SDG-funded schools have succeeded in obtaining other funds by using their newly acquired skills to prepare project proposals. In some municipalities, e.g., Dresden, the MED organized training on SDP and proposal preparation to meet demand from non-selected schools. Hence, the Project has strengthened the capacity of schools – both project supported and non-supported schools – to develop themselves.

At the municipal level, relevant officials received training in EMIS, who then trained schools. They have gained operational skills but are yet to improve analytical skills to make the best use of the collected data.

(c) Other Unintended Outcomes and Impacts (positive or negative)

The School Development Grants program reached almost twice as many schools as originally planned and created unexpectedly strong public awareness for educational development. As a result of strong appreciation for the SDG activities among schools, parents and communities, the government has included the SDG as one of the priority areas in the new Strategy for the Development of Pre-University Education in Kosovo: 2007-2017.

3.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops

An independent consultant team conducted an external evaluation of the School Development Grants component based on interviews and questionnaires with stakeholders. See Annexes 5 and 6 for details.

17

4. Assessment of Risk to Development Outcome Rating: Moderate

SDG: More than half of schools in the entire country have received training on school development planning and implementation. Although the SDG was a one-time investment, many schools have built up foundations for school management by school directors, school boards, and the community. Some schools indeed took the initiative to prepare a second- phase of school development plan and mobilized resources from local donors. In addition, building on the experience of this project, the government decided to mainstream the School Development Plans in its education strategy; the Pre-university Education Strategy2007-201711 aims for all schools to develop the capacity to prepare an SDP and MEDs to monitor and help schools in preparing SDP by 2008.

EMIS: As the EMIS was redesigned towards the end of the project implementation period, the installation of the new system and actual data collection and analysis could not be completed in all schools before the project completion date. In order to sustain the system, two measures have been taken. First, the EMIS contractor is obligated to provide technical support during the one-year warranty period, and the EMIS section of the MEST will continue data collection and training activities with support from the contractor. The PCU, together with the contractor is in the process of preparing an Action Plan for monitoring and supporting schools for hardware and software implementation, data collection and analysis. Second, the MEST is preparing a budgeted action plan to institutionalize the EMIS activities after 2008, including an appointment of IT expert/contractor. Indeed, the Pre-university Education Strategy includes “establishment of the education data processing infrastructure” under the Objective: Qualitative and efficient governance and management of the education system.

The institutions built under the EPIP – a practice of developing and implementing school development plans, active school boards, community participation, a fund flow mechanism to schools, partnerships with municipalities and schools, the involvement of local NGO – are likely to continue contributing to the achievement of the PDO. Recognizing SDG and EMIS as a critical activity in the Pre-University Education is a good indication of the government’s ownership of the initiative, and commitment to address the issue of educational attainment, especially for the vulnerable groups. The recently approved CIDA and the Post-Conflict Fund grants will continue supporting the SDG and EMIS components for the next two years. The Bank is also currently supporting the MEST to prepare a new project which would build on the EPIP and help the government implement the education strategy in Kosovo.

The continued uncertainty on Kosovo’s Status is not likely to have a significant impact on the sustainability of development objectives achieved under the EPIP because the Project has substantially contributed to the capacity building at school level, improved public awareness in education and the importance of monitoring and evaluation based on quantitative and qualitative data among school principals and local education officers. This local capacity

11 MEST, 2007, Strategy for the Development of Pre-University Education in Kosovo 2007-2017.

18

will be maintained, whatever the outcome is on Status.

5. Assessment of Bank and Borrower Performance

5.1 Bank Performance (a) Bank Performance in Ensuring Quality at Entry Rating: Moderately Satisfactory

As mentioned earlier, it was a wise and appropriate decision by the Bank Team to design a simple and focused project when the government’s institutional capacity was still weak. The overall direction of the PDO and monitoring indicators was appropriate, but more specific indicators were necessary at the time of project preparation. The Bank team had IT experts assess the capacity of EMIS established under the EHP, even though the assessment could not fully capture the weaknesses that the initial design had.

(b) Quality of Supervision (including of fiduciary and safeguards policies) Rating: Satisfactory

Since the project inception in July 2003, the Bank Team supported the MEST and PMU/PCU to implement the project through supervision missions. The MTR took place in December 2004-February 2005, and identified various problems with the EMIS and Project Management components. The MTR triggered the MEST to solve those problems and revitalize the project implementation. The Bank team took the initiative to mobilize additional funding from a development partner (CIDA) who was interested in the initial achievement of the SDG. The Bank team also saw the opportunity to support the MEST in developing the pre-university education strategy, and the project was able to finance technical assistance and the team emphasized the importance of public consultation. After the MTR, the supervision and implementation support missions became even more frequent (every 2-4 months), and video and audio conferences with the PCU were held in between to troubleshoot the implementation bottlenecks and keep the momentum of project implementation.

(c) Justification of Rating for Overall Bank Performance Rating: Moderately Satisfactory

The assessment of the implementation capacity of the government and the needs of the education sector was appropriate, which was reflected in the simple project design directly impacting the improvement of schools. The Bank Team provided close supervision throughout and especially towards the second half of the project implementation, which helped the timely completion of the project with almost 100 percent disbursement. However, the ambitious choice of target values for monitoring indicators and the weak assessment of the EMIS capacity at the entry stage caused problems for the progress of the SDG component and the M&E activities during the implementation.

5.2 Borrower Performance

19

(a) Government Performance Rating: Moderately Satisfactory

At the time of project preparation and commencement, the Department of Education, Science and Technology (DEST) of the UNMIK had just been transformed into the Ministry of Education, Science and Technology (MEST). Unfortunately, the Constitutional Framework for Provisional Self-Government, adopted in May 2001, did not clearly specify the division of responsibilities between the UNMIK and the MEST. Neither was there sufficient capacity building support for the MEST prior to the transfer of responsibilities. The weak government capacity resulted in the initial difficulties in the implementation of the project, but it turned to be an opportunity to diminish the role of UNMIK and strengthen the MEST's ownership over the Project.

(b) Implementing Agency or Agencies Performance Rating: Satisfactory

Implementing Agency Performance The Project implementation stagnated at the beginning in the absence of the MEST's strong leadership to improve the PMU, which did not have a clear division of individual responsibilities or monthly work plan for project implementation. Given that the MEST just took over the new responsibilities from the DEST of the UNMIK, the slow start was probably Ministry of Education, understandable. Science and Technology (MEST), and the Project Despite the initial difficulties with the PMU and the EMIS Coordination Unit within the component, the MEST demonstrated stronger leadership and MEST ownership of the project after the MTR. The PCU/MEST provided close support to schools and municipalities through frequent visits and communication, which led to the successful implementation and completion of the project. The participatory and timely process for finalizing the Pre- University Education Strategy 2007-2017 was also commendable.

(c) Justification of Rating for Overall Borrower Performance Rating: Moderately Satisfactory

Given the transitional period when the government just began building its own capacities, the initial difficulties in the project implementation are understandable. The MEST has demonstrated the stronger leadership towards the second half of the project implementation and commitment to sustain project achievement after the project completion, which led the MEST/PCU to successfully complete the project in time with almost 100 percent disbursement of IDA funds as well as additional funding from CIDA which was not originally planned.

20

6. Lessons Learned

Simple and focused project design is appropriate for a post-conflict context and accompanying weak capacity of the government. As repeatedly discussed, the simple and focused project design was one of the most important factors for the successful implementation and completion of this project in post-conflict Kosovo. It is crucial to take into account the institutional capacities of the government and the Implementing Agency when designing a project.

Community participation is instrumental in rebuilding the education system in a post- conflict environment. A strong emphasis on community participation and school boards was particularly relevant for post-conflict environment in Kosovo where the enthusiasm and demand for education was high, which needs to be captured and capitalized upon. As mentioned above, the contribution by communities for SDG greatly exceeded the “requirement.” This is likely to contribute to the sustainability of SDG. Community participation is also critical for the current uncertain status of Kosovo; schools and communities could play an important role in sustaining and improving the quality of educational services at local level.

PDO indicators need to be carefully selected, and target values should be set in a realistic manner. It is crucial that PDO indicators are reasonable for the length of the project, and measurable monitoring indicators are selected for which baseline data can be collected at the time of project preparation. It is important to assess the feasibility of data collection at the outset and select indicators which can be collected with certain reliability.

Capacity building in procurement procedures at the outset of the project is critical. Capacity building in the Bank’s procurement procedures by providing user-friendly training for all stakeholders from the MEST, PMU, REO, MED and schools in the very initial stage of the project implementation is critical. Also, while the central and local administration is building their own capacity in administering schools, it is important to strengthen the capacities of school boards and communities in assisting schools in managing themselves because active and close involvement of school board members and communities in school management help motivate schools to improve the quality of education that they provide for children.

Having a local NGO as a facilitation agency is an effective approach for school grants and management. Having a competent local NGO (Kosovo Education Center) that is well received by and closely tied with the MEST/PCU as a facilitation agency helped the MEST/PCU overcome the initial difficulties and make persistent efforts to make progresses during the project implementation. It is important to increase participation of NGOs in the project activities for these types of projects, not only at the central level, but also at the municipality level, to help advance cooperation between school and community.

Defining the roles and responsibility of each stakeholders at the outset is key for successful implementation. This is perhaps a common sense for any project, but not always easy to put into practice. It is important to ensure that Project Operating Manuals, in a user friendly

21

manner, clearly define objectives, responsibilities, and expected outputs for each stakeholder involved in the implementation, and is distributed to and understood by all schools, municipalities and regions. In particular, the supporting mechanism by the MED and REO for schools needs to be clearly defined, including teacher training, provision of transportation and security, and M&E activities. Continuous consultations and workshops at various levels would facilitate communication and enhances the clear understanding of roles and responsibility.

7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners (a) Borrower/implementing agencies

The MEST and UNMIK reviewed the draft ICR and had no comment (the letter is attached in Annex 7).

(b) Cofinanciers

Co-financing partner, CIDA, also reviewed the draft ICR, and agreed with the assessment.

(c) Other partners and stakeholders (e.g. NGOs/private sector/civil society)

22

Annex 1. Project Costs and Financing

(a) Project Cost by Component (in USD Million equivalent) Appraisal Estimate Actual/Latest Percentage of Components (USD M) Estimate (USD M) Appraisal 1. SCHOOL PLANNING AND SCHOOL DEVELOPMENT 3.79 3.79 100.00 GRANTS 2. EDUCATION MANAGEMENT 0.30 0.30 100.00 INFORMATION SYSTEM 3. PROJECT MANAGEMENT 0.51 0.51 100.00 Total Baseline Cost 4.60 4.60 Physical Contingencies 0.00 Price Contingencies 0.00 Total Project Costs 4.60 4.60 100.00 CIDA for SDG 0.90 0.90 Total Financing Required 5.50 5.50 100.00

(b) Financing Appraisal Type of Actual/Latest Percentage of Source of Funds Estimate (USD Cofinancing Estimate (USD M) Appraisal M) Borrower 0.10 0.10 100.00 CANADA: Canadian International Development 0.90 0.90 100.00 Agency (CIDA) IDA GRANT FOR POST- 4.50 4.50 100.00 CONFLICT

23

Annex 2. Outputs by Component

Component I: School Development Grants (SDGs)

The Municipal School Grant Teams (MSGTs) and the School Grants Approval Board (SGAB) were established as the central and municipal bodies to evaluate and approve the SDG proposals. The original implementation plan was to cover four rounds (about 80 schools each round, or 330 schools in total) by June 2005. However, due to some delays in the initial project activities and availability of extra funding, six rounds were organized in the period October 2003-December 2006. Each round consisted of the following major activities: (i) selection of approximately 100 schools from seven regions; (ii) training for representatives of selected schools in school development planning (approximately 7 training sessions on regional level, 2-days each); (iii) follow-up meetings on school development plans/Experience sharing (approximately 7 meetings on regional level, 1-day each); (iv) training for representatives of selected schools in school project planning (approximately 7 training sessions on regional level, 2-days each); (v) outreach support to schools in preparing school grant proposals; and (vi) submission of school grant proposals to the PMU.

As summarized in Table A-1, a total of 604 projects were carried out and 90 schools implemented the second project in the 6th round. The total number of students who benefited from the Project was around 300,000. The number of total recipients was much more than the original target. The types of project included (i) infrastructure improvement, (ii) computer cabinets, musical instruments, and craft equipments, (iii) textbooks and clothes, (iv) transportation, and (v) teacher training. Despite the wide range of potential activities, the flexible design of SDG resulted in a large percentage of investment in infrastructure. These infrastructure investments concerned mostly basic infrastructural needs such as sanitation facilities, water supply, and roofs. After guidance from the SGAB, the percentage of infrastructure improvement slightly decline in Rounds 4 and 5, and again increased in Round 6.

Table A-1: Percentage of SDG projects

Round Round Round Round Round Round 6 Total 1 2 3 4 5 Total no. of schools 96 107 102 108 101 90 604 Infrastructure 58.2 70.2 62.2 38.0 50.0 62.0 55.8 improvement1 (%) Computer cabinets, musical instruments, craft 27.6 13.8 38.8 38.0 38.6 20.8 29.6 equipments (%) Textbooks and clothes (%) 16.3 7.4 5.1 23.1 18.8 14.2 14.1 Transportation (%) 3.1 7.4 3.1 0.0 1.0 2.0 2.76 Teacher training (%) 6.1 1.1 2.2 0.9 5.0 0.0 2.55 Source: PCU, Project Monitoring and Evaluation (June and November 2006) 1 Included sanitation facilities, fences, roofs, water supply, sports grounds and classroom renovation.

24

Component II: Improving the Education Management Information System

The newly developed EMIS has been piloted in 30 schools and is in the process of nation- wide installation. MEDs officers have been trained.

The EMIS unit in the MEST has taken over the responsibility for EMIS from the PMU/PCU. The EMIS Unit/PCU, together with the contractor has prepared an Action Plan for software installation in the remaining schools, and an Action Plan for monitoring and supporting schools for hardware and software implementation, data collection and analysis.

The objective of this component was to help strengthen and ensure the sustainability of the EMIS, which had been designed and initiated under the EHP. However, as soon as the EPIP commenced and the MSGTs tried to collect EMIS data required for school selection, it was discovered that the EMIS was unable to produce necessary school statistics and population data. Therefore, the EMIS was not able to help monitor enrollment, attendance, repetition, dropout and completion. The September 2004 Aide Memoire commented that the EMIS implementation was unsatisfactory as the EMIS team was unable to produce reports and the data were unreliable. This status was confirmed during the MTR mission in December 2004. While the then-TTL and the PMU decided to revise monitoring indicators, and tasked the Kosovo Education Center (KEC) (the local NGO facilitating the SDG component) to collect necessary data from schools through ROSOs, the MEST took over the responsibility for the EMIS from the PMU, established a Working Group on the EMIS and started a reform process in January 2005. Due to these efforts, the EMIS finally started to make considerable progress after the MTR.

After it was recognized that the EMIS developed under the EHP was unable to produce M&E indicators as expected, the system had to be redesigned. The pilot test of the new system was completed and nation-wide installation and training of schools and municipalities began towards the end of the project implementation period. The new system produces 19 indicators, eight of which are in compliance with the international ranking defined by the UNESCO Institute of Statistics (UIS).

Component III: Project Management

The Project Management Unit (PMU) was established under the MEST for the implementation, monitoring and evaluation of the EPIP. Most of the initial staff had already been contracted under the EHP and were involved in the preparation of the EPIP. However, from the very beginning of the project implementation, it was clear that the PMU was not functioning well. The PMU was restructured as the Project Coordination Unit (PCU) after the Mid Term Review in the mid to late 2005. The new PCU was more a coordinating, rather than managing, unit closely supervised by the MEST. The management responsibility for the EMIS component was also transferred from the PMU to the EMIS unit of the MEST. The new PCU members, in closer coordination with the MEST, contributed to revitalization of the project implementation and the nearly 100 percent of disbursement of the project budgets.

25

Annex 3. Economic and Financial Analysis

As discussed under Section 3.3 Efficiency in the main text.

26

Annex 4. Bank Lending and Implementation Support/Supervision Processes

(a) Task Team Members Names Title Unit Responsibility/Specialty Lending Peter Darvas Sr. Education Economist AFTH2 Task Team Leader Peter Buckland Sr. Education Specialist MNSHD Education Specialist Jim Stevens Sr. Operations Officer HDNED Operations Shobhana Sosale Operations Officer HDNED Operations Jeanine Braithwaite Sr. Social Projection Economist HDNSP Social Protection Maria V. Vannari Sr. Procurement Specialist ECSPS Procurement Mark Walker Lead Counsel ECSPS Legal Hiran Herat Consultant AFTNL Financial Management Joe Formoso Lead Finance Officer LOAG1 Disbursement Anna Goodman Program Assistant ECSHD Program Assistant

Supervision/ICR Sr. Financial Management Olav Rex Christensen ECSPS Financial Management Specialist Elona Guika Financial management Analyst ECSPS Financial Management Plamen Stoyanov Procurement Specialist ECSPS Procurement Kirov Belita Manka Procurement Analyst ECSPS Procurement Maria V. Vannari Sr. Procurement Specialist ECSPS Procurement Keiko Miwa Sr. Education Economist ECSPS Task Team Leader Sachiko Kataoka Education Economist ECSHD ICR Author Lorena Kostallari Sr. Operations Officer ECSHD Operations Gentjana Sula Operations Officer ECSHD Operations Kseniia Malenko Finance Analyst LOAG1 Disbursement Anahit Poghosyan Office Manager ECSHD Program Assistant Svetlana Georgieva Program Assistant ECSHD Program Assistant Raykova Carmen Laurente Sr. Program Assistant ECSHD Program Assistant

27

b) Staff Time and Cost Staff Time and Cost (Bank Budget Only) Stage of Project Cycle USD Thousands No. of staff weeks (including travel and consultant costs)

Lending FY03 45 219.18 Total: 45 219.18

Supervision/ICR FY03 Not Available 0.26 FY04 15 84.44 FY05 25 108.18 FY06 39 108.25 FY07 22 60.30 Total: 101 361.43

28

Annex 5. Beneficiary Survey Results

The government’s Project Closing Report summarizes ‘Opinions and Comments of Project Beneficiaries’ based on interviews and questionnaires as follows:

According to the interviews and respondents from the questionnaire successful impact in fulfilling the objectives (attendance and reduction of drop-out rate) of the project have had particularly schools that have provided students’ transport and schools that developed projects on supplying students with school books and warm clothes.

- “The children are happier and feeling safe after we have set up the fence. Since the school is close to a highway, there were a number of accidents every year while children were playing or crossing the road. Many of them have endured serious injuries or were even killed. Now we also have a group of five Roma students who are attending our school because they feel safer here, although they can attend another school which is nearer to their place where they live”. - “This grant has helped us to implement a very important project for our school. In order to fulfill the requirements for this project we had to design school development plan which helped us in further implementation of our plans. Having set up a development plan we have attracted other donors so now we are in the process of implementation of two other projects with an approximate fund of 200.000 Euros”. - “EPIP project has initiated and now with MED’s support we managed to ensure transportation for our children, which is crucial for them especially in winter when the weather conditions deteriorate. Without this grant, school attendance would have been very low”. - “Grants offered by the World Bank are the only financial support that we received. It is a great support for our municipality since our budget is very low. I will personally be engaged in managing of these grants by schools in order to fulfill project requirements”. - “The children are now actively involved in using computers rather than watching television or wondering around the village”. - “The essential component in making a difference in the classroom is that teachers know how to teach. Once teachers have these skills, then improved infrastructure or transport can improve the process. There is no value exhorting teachers to do better, work harder or provide initiatives, if they do not know how to facilitate effective and meaningful learning”. - Attendance has improved, the success has increased since the classes are cleaner and there is no interruption during the classes for lightening the fire and there is no dust”. - Definitely the lessons have become more attractive for most of the pupils and they likely spend their free time in school being engaged with different activities”. - “Small grants big effect, EPIP is one of the most successful projects in education field after the war”.

29

Annex 6. Stakeholder Workshop Report and Results

The aims of the external evaluation were: (i) to assess the main outcomes of the EPIP project in terms of effectiveness, efficiency, relevance, and sustainability; and (ii) to draw a number of education policy lessons that could offer a thorough frame of reference for future similar or new initiatives. A number of quantitative and qualitative methods were used for data collection and analysis. Below are extracts from the evaluation report.12

Key Findings: In terms of general findings, from the point of view of EPIP development objective, the conclusion of the evaluation is the following: at this point in time, it is difficult to offer concrete project-related statistical data in order to quantitatively prove that the Project development objective has been fulfilled. However, in terms of general outcomes - also confirmed by the qualitative data collected throughout the evaluation process - the results of EPIP prove to be positive. More than that, in a paradoxical way, EPIP displays an important number of ‘unexpected results’ (see more on this aspect in Chapter 7 – Conclusions and recommendations). These results have not been planned or foreseen in the initial project design. Nor the initial development objective or the body of specific objectives in EPIP made reference to them. These “unexpected results” – extremely important for an emerging educational system - are: a thorough public awareness raising concerning the importance of education reform for any dynamic society; the development of a system of functional and valuable grass-roots processes that can be considered a prerequisite for any further educational decentralization; a number of relevant evolutions in the functioning of central, municipal and school administration; a general increase of public and community participation in education. These are important outcomes and any further development in Kosovo education should take them into account. Therefore, there is a clear feeling according to which the project was more than successful in addressing Kosovo education systemic needs. Acting at grass-roots level, EPIP had without any doubt a systemic impact.

As for the general evaluation questions this approach has showed that the project was effective, efficient, and relevant; it also can be considered sustainable. In terms of effectiveness, based on the evidences collected during the evaluation approach one can conclude the following: (a) as compared to the period in which the Project was launched, the current trend of dropout, attendance and completion rates by gender and ethnic groups is a positive one in Kosovo (and here the Project had an important contribution alongside other interventions in the system); (b) the selection of schools for EPIP was open, fair and transparent; (c) the project has contributed to a better involvement of school boards, communities, parents, students and other stakeholders in school management; (d) the project and the school grants contributed to the improvement of teaching and learning environment at the school level.

In terms of efficiency, (a) the project has contributed to an increase of expenditure per child and the increase is even more relevant when one refer to smaller schools from remote areas;

12 Crisan, Alexandru, Rexhaj, X. and Saqipi Blerim. External Evaluation: School Development Grants (SDG) Component (December 2006).

30

(b) savings have been made from the International Competitive Bidding of computers and the amounts have been reallocated to school grants.

In terms of relevance all stakeholders and participants agreed that EPIP fully responded to schools’ needs, national strategies and priorities (first of all those related to decentralization and increasing school autonomy), as well as international partners’ priorities (see the involvement in the Project of KEPD).

As for the sustainability criteria (a) the project appeared to strengthen or add value to the reform effort in the education system in Kosovo (awareness raising, developing school-based processes, fostering ownership and school autonomous thinking and action, networking, etc.); (b) the project fostered capacity building in education (professional development of the staff, better management at all levels, improving planning, strategizing and prioritizing capacities etc.); (c) ‘partnering’ and networking have been developed in order to foster the quality of education; (d) the project has complemented and created linkages as well as synergetic or multiplier effect with education projects implemented by other partners (KEPD, UNICEF); (e) the project has fostered and consolidated the political will to bring the project to scale and mainstream the concept and principles of the project in the education system.

Conclusions and Recommendations: The new Education Strategy 2007-2017 and its underlying action plans recently designed by Kosovo educational stakeholders clearly show: (a) the need for consolidating and strengthening the changes that have already taken place and (b) the need for a paradigm shift from extensive and centralized development to an intensive and decentralized type of development; more exactly, to the development and implementation of appropriate mechanisms that better ensure and monitor the quality of the education. In this context the following are recommended to MEST:

A. Project-related Recommendations (A-1) To continue and extend the EPIP capitalizing on its current results and lessons (A-2) To refine the idea of a Grant Scheme shifting the focus from infrastructure to quality assurance and improving school-based processes (A-3) To ensure better connection between the Project development objectives and the nature of concrete School Projects from the initial phase of their selection (A-4) To capitalize and mainstream the Project Results (best models and practices) (A-5) To strengthen and upgrade the current PCU.

B. Education Policy-related Recommendations: (B-1) Grass-roots Processes should be fostered and integrated into the new Education Strategy (B-2) National vision and strategy for ‘school improvement practices’ as well as a special ‘school-improvement function’ in the Ministry and at the level of the Municipal Education Directorates should be developed and implemented as soon as possible (B-3) Human resources already prepared in the Project should be efficiently used for fostering grass-roots developments all over Kosovo

31

(B-4) Complex model of professional development of the staff in topics related to school development and school improvement should be designed and implemented (B-5) Clear cut monitoring and evaluation mechanisms throughout the system should be designed and implemented in order to foster quality evolutions (B-6) Internal/local student census model should be developed and implemented in order to monitor enrolment as well as implementing a nation-wide consolidated EMIS (B-7) Larger Alliances, Partnership and Networking for educational change should be developed with other central and local institutions.

A number of recommendations go to REO and MED as follow: (A) Strategic planning should be fostered at the level of REO and MED (B) Strategic alliances should be fostered at municipal level (C) Quality monitoring functions and arrangements should be gradually created and fostered at municipal level

In terms of conclusion, one can say that schools have advanced a lot in the last few years. Among many other systemic inputs that influenced their development, the Project had an important effect in fostering a positive trend in changing schools’ ‘internal culture’ (see Chapter 3). A special remark should be made on students. They are the best asset Kosovo education currently has. Most of the meetings and focus groups organized with students gave the image of a brand new generation, open-minded, flexible, and prepared to assume the challenge of an ever changing society. Youngsters met – in big urban areas or in small and remote rural schools – all have strong opinions, clear vision and an assumed ownership on their own future. Their current level reflects without any doubt the fact that an important paradigm shift has happened in the quality of Kosovo education in the last few years. However, a number of comments and recommendations to schools and educational administration will be offered below:

(A-1) The new ‘culture of strategic planning’ should be extended in all schools (A-2) Organizational culture should be fostered and a new participatory management style should be encouraged and gradually implemented in schools (A-3) Better task allocation and delegation of responsibilities should be fostered in Kosovar schools (A-4) Project monitoring and internal evaluation mechanisms should be built into the project design (A-5) Parent and community involvement should be fostered.

As a general conclusion, there is an important political will in the system to capitalize on the project results and bring it to scale and mainstream its concept and principles in the education system. In fact, important decision makers at central level foresee the continuation of the Project with some important Educational Policy Shifts: (a) focusing EPIP towards Quality Assurance and Quality Increase; (b) integrating EPIP mechanisms, procedures and achievements in the large-scale national education policy of decentralization; (c) creating wider alliances of stakeholders and mobilizing the nation for educational change.

32

Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR

The government prepared a Project Closing Report (PCR) on March 2007, which reflects the results of the External Evaluation: School Development Grants Component conducted in December 2006 (as summarized in Annex 6). Below is an extract from the PCR.

Executive Summary

1. School Development Grants The objective of the School Development Grant (SDG) was to improve educational attainment in primary and secondary education and enhance access of vulnerable groups to education in Kosovo. The main and concrete objective of SDG component was to provide funding to schools to help increase enrollment, attendance and retention of primary and secondary education, as well as improving the quality of teaching and learning. From the point of view of EPIP development objective someone can conclude that in the present conditions, it is difficult to offer concrete project-related statistical data in order to quantitatively prove that the Project development objective has been fulfilled. However, in terms of general outcomes the results of EPIP prove to be positive. More than that, based on our observations, EPIP displays an important number of unexpected results that have not been foreseen in the initial project design. These unexpected results are extremely important for the educational system: a public awareness raising concerning the importance of education reform; the development of a system of functional and valuable grass-roots processes that can be considered a prerequisite for any further educational decentralization; a number of relevant evolutions in the functioning of central, municipal and school administration; a general increase of public and community participation in education.

As for the sustainability criteria the project appeared to strengthen or add value to the reform effort in the education system in Kosovo and that: (a) awareness raising, developing school- based processes, fostering ownership, etc.; (b) the project fostered capacity building in education; (c) the project has complemented and created linkages with education projects implemented by other partners (KEPD, UNICEF).

The total number of schools which received grants based on their SDP was 514. While another 90 schools were awarded for the second time based on their successful implementation of their first projects from previous five rounds.

The total number of grants awarded by schools in EPIP was 604 schools among them 534 primaries and 70 secondary levels. The total number of the grant amount was 3.886.642 with a community contribution of 1.214.872. The total amount for this three year project was 5.101.514 euros. The total number of students in all Kosovo that have benefited from EPIP is around 300.000. Not only students have benefit from this project; there are a number of other indirect beneficiaries such as admin workers, teachers and other MED and Procurement Officers who have received some professional training on finance and procurement.

33

2. Education Management Information System (EMIS) The objective of EMIS was strengthening and ensuring the sustainability of the Education Management Information System to monitor and evaluate enrollment, attendance, repetition, dropout, and school completion by students, as well as account for school-age children not participating in the school system. EMIS currently is in its final phase, the main SERVER has been confabulated and it has been tested see (http://smia.epip-ks.org) and in cooperation with the working group MEST – EPIP and REPRON are installing and testing the server within the schools. The preliminary trainings for the users have been provided and the manuals have been delivered and have been approved. The company is obliged by the contract to support EMIS one year after the completing the admission/delivery procedure.

3. Project Coordination Unit (PMU) (PCU) The objective of this component was the effective implementation, monitoring and evaluation of the proposed project, through the provision of goods and consultants’ services. Te PCU has managed to assume the administration and implementation responsibilities for the project in close cooperation with MEST, UNMIK, REO, MED and other stakeholders has managed to complete the project successfully.

Lesson Learned and steps for the future

Generally looking the perception of all participants and beneficiaries of the project is that the project had a great contribution in Kosovo educational system in primary and secondary level. Most of the stakeholders involved in the project think that enrolment, attendance and completion rates increased whereas drop-out rate decreased in the schools that awarded projects. This is the perception of all relevant parties involved in the project. EPIP has addressed two major priorities in Kosovo education: improving school infrastructure and improve the quality of teaching in both primary and secondary schools. Obviously this was a good step towards the improving the quality of the education and the schools and the communities are looking for new projects. Unfortunately because of the current economical situation, the Educational System in Kosovo in the future will need international support in order to implement other necessary school projects. In order to improve future projects in the future the following steps should be undertaken:

- More responsibility, ownership, cooperation within the stakeholders - More cooperation and communication between different levels from schools to MEST; - More ownership and access to financial management of the PCU. - Encourage community participation in school projects, - The community participation in improving the school quality should be permanent - To develop projects which will create better teaching and learning environment - To develop projects that will improve students performance - To foster teachers’ creativity to provide quality learning - Operationalization of the EMIS with all components. A higher level of support is requested, in order to make the system operates more successfully.

34

Conclusion

The EPIP was set up in the right time and in the right place. Kosovo after the 1999 war has inherited a completely destroyed infrastructure including destroyed and burnt schools. It has also inherited and academic style of learning based on teacher-centered methodologies. There was an immediate need to intervene in both improving the ruined infrastructure and in fostering and improving learning and teaching activities and students performance. All this would have been impossible to achieve without Government’s support and especially without international donors.

The WB grant has supported the schools to do their first step towards the fulfillment of their most necessary needs which they considered as their priority for a better functioning. As a general conclusion, one can say that schools have advanced a lot in the last few years and this is also thanks to EPIP support which schools considered that was direct and indispensable. Among many other systematic inputs that influenced their development, the Project had important effect in fostering a positive trend in the change of schools regarding planning, managing and teaching.

Thanks to this project students from both urban and rural areas – all have strong opinions, clear vision and an assumed ownership of their future. Their current level reflects without doubt that a good move forward has happened in Kosovo education in the last few years, and that the education system is in a good path moving forward in educating the new generations so in order to become part of wide World’s education system.

35

Borrower's Comments on Draft ICR

36

Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders

The ICR was shared with the Canadian International Development Agency (CIDA), the cofinancier of this project before it was finalized. CIDA commented that this ICR has "addressed all of the major project achievements and challenges."

37

Annex 9. List of Supporting Documents Project-related Documents - Project Operating Manual Vol. I , Vol. II: School Grants Manual & Procurement Handbook - School Grant Pilot Project Report: Final Report (February 2004) - Progress Reports (various issues), Project Narrative Reports (various issues) - Project Monitoring and Evaluation Reports (June & November 2006) - EMIS and Its Transferal under MEST: Background report for Jan – Jun 2005 (June 2005) - MEST, EPIP Project Closing Report (March 2007) - Dukagjin Pupovci (Kosova Education Center), Report over the Periodical Evaluation of the Project (May 2004) - Dukagjin Pupovci (Kosova Education Center), Monitoring Report (December 2006) - Crisan, Alexandru, Rexhaj, X. and Saqipi Blerim. External Evaluation: School Development Grants (SDG) Component (December 2006) General Education-related Government Documents - Strategic Plan for the Development of Education in Kosova (2002-2007): Better Education for all – Nothing happens in education until it reaches the pupils (January 2003) - Strategy for the Development of Pre-University Education in Kosovo: 2007-2017 (March 2007) - Strategy for Development of Higher Education in Kosova (2005-2015) (final version), 2004 - Kosovo School Funding Handbook: Financial Management Information for Municipal Education Directors and School Directors - Handbook on Municipal Education Governance - Education for Rural People in Kosovo: Conditions, Challenges and Strategies for Development (February 2004) - Salihu, Linda, Kosovo Social Assessment (draft report) (undated, but around 2002- 2003) - Regulation No. 2002/19 on the Promulgation of a Law adopted by the Assembly of Kosovo on Primary and Secondary Education in Kosovo (which introduced mandatory school boards for all schools) World Bank Documents - Project Appraisal Document on Education Participation Improvement Project (April 2003) - Project Appraisal Document on Education and Health Project (May 2000) - Implementation Completion and Results Report on Education and Health Project (November 2004) - Kosovo, Federal Republic of Yugoslavia, Transitional Support Strategy (July 2002) - Interim Strategy Note for Kosovo for the Period FY06-FY07 (March 2006) - Poverty Assessment (2001 and 2005) - Kosovo Medium Term Public Expenditure Priorities (2002) - Kosovo Public Expenditure and Institutional Review (June 2005)

38 IBRD 35406

20°00' E 20°30' E To Kraljevo 21°00' E 21°30' E This map was produced by the Map Design Unit of The World Bank. The boundaries, colors, denominations To Kraljevo and any other information shown on this map do not imply, on the part of The World Bank Group, any judgment

on the legal status of any territory, or

SERBIA ˆ any endorsement or acceptance of KOSOVO To Nis such boundaries.

lica Top

Ib a

r

Leposavic´ ˆ (Leposaviq) To Nis

S 43°00' N 43°00' N Gazivodsko E

Jezero To Leskovac

ˆ R B I A Zvecan Podujevo Zubin Potok (Zvecan) Mitrovica ca

MONTENEGROMONTENEGRO lan

(Mitrovicë) Jab

ˆ Batlavsko Jezero Vucitrn To Ivangrad (Vushtrri) Istok (Istog)

ab

Srbica L

ˆ Beli Dri Djurakovac a n To m li K Obilic´ Ivangrad PRISTINA (Obiliq) (PRISHTINË) Pec´ Gracanicko˘ ˘

(Peje) Glogovac Kosovo Polje Jezero

ˆ Klina (Kline) (Gilogoc) (Fushë Kosovë) Novo Brdo Kosovska

Kamenica

S (Novo Berde) To Leskovac

Decani ˆ i tn ic (Decan) a

Radonjicko Lipljan (Lipjan)

42°30' N Jezero ˆ 42°30' N Malisevo Ereni Gnjilane (Gjilani) k Orahovac ˘ a (Rahovec) Stimlje av

(Shtime) or

ˆ M

ka Djakovica nac Suva Reka Urosevac Bi (Gjakove) (Suhareke) (Ferizaj)

Vitina

ˆ (Viti)

ˆ Gabrica Kacanik Strpce (Kacaniku)

A L B A N I A m ri Prizren (Shterpce) D li L e e B p e n a Liqeni i c Fierzës F Y R To Durrës M A C E D O N I A

Gora (Dragashi) SKOPJE To Sofia 42°00' N MAIN ROADS 21°30' E RAILROADS M 20° O 30° A U S T R I A L D

SELECTED CITIES* O To Durrës H U N G A R Y

V

A SLOVENIA NATIONAL CAPITAL R O M A N I A CROATIA PROVINCE BOUNDARY To Bitola BOSNIA AND HERZEGOVINA SERBIA Adriati INTERNATIONAL BOUNDARIES Black

c Sea B U L G A R I A KOSOVO MONTENEGRO To Bitola ITALY Se FYR a MACEDONIA *When two city names are shown, Serbian spellings ALBANIA are shown first, followed by Albanian spellings in parentheses. 0 10 20 30 Kilometers 40° 40° When only one name is shown, the spelling is the same in

both languages. Tyrrhenian Aegean GREECE TURKEY 0 10 20 Miles Sea Sea

20°00' E 20°30' E 21°00' E 20°

APRIL 2007