Performance Evaluation Sri Lanka: Distance Education Report Modernization Project

Independent Evaluation

Raising development impact through evaluation

Performance Evaluation Report November 2016

Sri Lanka: Distance Education Modernization Project

This document is being disclosed to the public in accordance with the Asian Development Bank’s Public Communications Policy 2011.

Reference Number: PPE:SRI 2016-20 Project Number: 33251 Loan Number: 1999 Independent Evaluation: PE-793

NOTE

In this report, “$” refers to US dollars.

Director General M. Taylor-Dormond, Independent Evaluation Department (IED) Director W. Kolkma, Independent Evaluation Division 1, IED

Team leader F. Ahmed, Lead Evaluation Specialist, IED Team members M.A. Chaneco, Associate Knowledge Management Administrator, IED M.J. Dimayuga, Senior Evaluation Officer, IED

In preparing any evaluation report, or by making any designation of or reference to a particular territory or geographic area in this document, the Independent Evaluation Department (IED) does not intend to make any judgments as to the legal or other status of any territory or area.

The guidelines formally adopted by IED on avoiding conflict of interest in its independent evaluations were observed in the preparation of this report. To the knowledge of the management of IED, there were no conflicts of interest of the persons preparing, reviewing, or approving this report.

Abbreviations

ADB – Asian Development Bank DEMP – Distance Education Modernization Project DEPP – Distance Education Partnership Program GCE – General Certificate of Education ICT – information and communications technology IED – Independent Evaluation Department LMS – learning management system MOHE – Ministry of Higher Education NAC - network access center NDEN – National Distance Education Network NODES – National Online Distance Education Service OUSL – Open University of Sri Lanka PCR – project completion report PMO – project management office PPER – project performance evaluation report PSC – project steering committee PVR – project validation report SDR – special drawing rights

Currency Equivalents

Currency unit – Sri Lanka rupee (SLRe) At Appraisal At Completion At Independent Evaluation (8 May 2003) (24 June 2011) (25 August 2016) SLRe1.00 = $0.0103 $0.0091 $0.0069 $1.00 = SLRs109.5300 SLRs109.5300 SLRs145.5800

Contents

Acknowledgments vii Basic Data ix Executive Summary xi Introduction 1 A. Evaluation Purpose and Process 1 B. Expected Results of the Project 2 C. Evaluation Method 3 Design and Implementation 4 A. Formulation 4 B. Rationale 5 C. Cost, Financing, and Executing Arrangements 6 D. Procurement, Construction, and Scheduling 7 E. Outputs 7 F. Consultants 10 G. Loan Covenants 10 H. Policy Framework 11 Performance Assessment 12 A. Overall Assessment 12 B. Relevance 12 C. Effectiveness 16 D. Efficiency 19 E. Sustainability 19 Other Assessments 24 A. Impact 24 B. ADB and Executing Agency Performance 27 Issues, Lessons, and Follow-up Actions 28

Appendixes 1. Distance Education Modernization Project – Project Framework 33 2. People Met During the Performance Evaluation Mission 38 3. Appraisal and Actual Project Costs 41 4. Design and Monitoring Framework Showing Achievements, 2011 and 2016 42 5. Partner Institutions and Online Degree and Diploma Programs, 2011 47 6. Rating Matrix for Core Evaluation Criteria 49 7. Open University of Sri Lanka Enrollment, Financing, and Cost Recovery 50 8. Open University of Sri Lanka Enrollments and Contribution to National Graduate Output 51 9. Enrollment in Open University of Sri Lanka Study Centers 1999−2014 52

Supplementary Appendix (available on request) Student Survey Questionnaire

Acknowledgments

This report is a product of the Independent Evaluation Department (IED) of the Asian Development Bank (ADB). Farzana Ahmed, Lead Evaluation Specialist, led the evaluation. The team comprised Mary Anne Chaneco (Associate Knowledge Management Administrator), Palita Dammalage (national consultant), Ma. Juana Dimayuga (Senior Evaluation Officer), and Wendy Duncan (international consultant).

The evaluation team acknowledges the support provided by colleagues in the South Asia Department and the Sri Lanka Resident Mission. The team benefited greatly from discussions with officials of the Government of Sri Lanka, and particularly with staff at the Ministry of Education, Ministry of Higher Education, Ministry of Skill Development and Vocational Training, the Open University of Sri Lanka, the University of Vocational Technology and the National Education Commission, whose staff participated in the interviews and assisted in coordinating the field visits in various districts. We also appreciate discussions with other development partners active in the education sector in Sri Lanka. The feedback from participants in our questionnaire and focus group discussions strengthened the study and is much valued.

The report was peer reviewed by Brahm Prakash, Evaluation Consultant, IED and was prepared under the overall guidance of Vinod Thomas and Walter Kolkma of IED. IED retains full responsibility for the report.

Basic Data Sri Lanka: Distance Education Modernization Project (Project Number 33251, Loan 1999)

At Appraisal Actual Key Project Data ($ million) ($ million) Total project cost 60.00 46.99 Foreign exchange cost 24.00 24.81 Local currency cost 36.00 22.18 ADB loan amount/utilization 43.30 35.47

ADB = Asian Development Bank.

Key Dates Expected Actual Fact-finding 28 July–11 August 2002 Loan negotiations 28–29 April 2003 Board approval 11 June 2003 Loan signing 28 August 2003 Loan effectiveness 16 November 2003 4 November 2003 First disbursement of loan 11 November 2003 Loan closing 31 December 2009 28 July 2010

Borrower: Sri Lanka Executing Agency: Ministry of Education

Number of Number of Person- Type of Mission Missions Days Fact-finding 1 No data Appraisal Waived Inception 1 24 Review 6 62 Midterm review 1 22 Special loan administration 2 3 Project completion review 1 36 Independent evaluation 1 28

Development Project Performance Report Ratings Objectives Implementation June 2003–July 2010 Satisfactory Satisfactory

Executive Summary

Following successful provision of near-universal basic and secondary education, most countries in Asia and the Pacific now face the challenge of providing young people with the knowledge and skills needed to boost employment and economic growth, as economies transition from agriculture to manufacturing and services, and the labor market moves from largely public sector to greater private sector activity, but government budgets remain constrained.

In the early 2000s, Sri Lanka exemplified these dilemmas. While successive Sri Lankan governments had been very successful in offering basic and secondary education to almost all children, even as a civil war raged, the postsecondary education system was so small that it covered only 3% of young people of the relevant age. At the same time, the rate of unemployment among the small group of post-secondary graduates was higher than the national rate of unemployment, and employers complained they could not find workers with the skills they needed for a modernizing economy. The government of the time recognized that significant reform of postsecondary education was necessary but it lacked the resources to do so. In line with its strong social policies, the government was also committed to ensuring university education remained free, and that all young people, especially those in conflict-affected and plantation areas, could access postsecondary education and training that would be consistent with private sector employment opportunities. An innovative approach was needed to the expansion of postsecondary education and training, outside the conventional tertiary education system.

To assist the government in achieving these goals, the Asian Development Bank (ADB) supported a project that aimed to expand education and training opportunities for school leavers in rural and conflict-affected areas through the use of online distance education. Although Sri Lanka had a tradition of distance education through printed materials, online education was new. This approach was in line with the government’s strong focus on using information and communications technology (ICT) to modernize the Sri Lankan economy. Distance education learning opportunities were to be expanded through the establishment of a national fiber optic network offering online distance education. Courses were to be provided by public universities, including the Open University of Sri Lanka (OUSL), and private training providers offering more vocationally oriented training. This was another first for Sri Lankan education. While the government recognized the value of greater private sector involvement in education, it had little experience of public–private partnerships. Finally, the project also aimed to upgrade and expand the capacity of the OUSL through the development of online courses and more conventional institutional strengthening and development.

The Distance Education Modernization Project (DEMP) was financed through a loan of SDR33,005,000 (equivalent to $45 million) from ADB Special Funds resources. The 6-year loan was approved on 11 June 2003, and closed on 28 July 2010 after a 6- month extension. The project was selected for performance evaluation because of its strong elements of innovation. The project performance evaluation report (PPER) provides an opportunity to assess the longer-term impact of the project while also

xii Distance Education Modernization Project

identifying lessons generally applicable to the design and implementation of innovative projects.

A. Project Performance

This report presents performance evaluation findings for four evaluation criteria (relevance, effectiveness, efficiency, and sustainability) as well as an additional criterion of impact. Overall, the project is rated less than successful.

Relevance. The project is rated relevant. It responded to government and ADB national and sector objectives, and it addressed the key issue of tightly restricted access to tertiary education in an imaginative and innovative manner. As a new and innovative project, it faced many risks to success, not all of which were mitigated adequately. The project design was too complex, too broad, and most significantly did not adequately anticipate the resistance to online education that emerged among both students and education providers. More groundwork and investigation was needed before launching such an ambitious investment project.

Effectiveness. The project is rated less than effective. It had many noteworthy achievements, but on a smaller scale than originally intended. By the end of the project, it had provided opportunities for over 33,000 tertiary students, through online courses that provided job-relevant qualifications and skills. This was about one-third of the original target. The complexity of the project, resistance and caution among the intended partner institutions, the late uploading of online courses, and hesitation about their value on the labor market were the primary factors behind the failure to attract large numbers of students to online education.

Efficiency. The project is rated less than efficient. Implementation was handled reasonably efficiently given the challenging nature of the project, but the delays in establishing the network and developing courses resulted in too little time for enrolments to grow. The National Online Distance Education Service (NODES) courses were not cost efficient because of the high costs of internet provision in Sri Lanka. The OUSL proved more cost efficient, increasing its level of cost recovery from 25% to 42%, in line with the target.

Sustainability. The project is rated less than likely sustainable. The NODES system established under the DEMP is no longer in operation. The infrastructure and some staff have been handed over to the OUSL for incorporation into its systems. How the OUSL will utilize the network, and whether it will one day be open to other institutions, is not yet known. However, the introduction of ICT-enabled education is likely to be sustainable in various forms. It is also behind emerging changes in how universities provide their external degrees, and moves by the OUSL to slowly increase online content.

Impact. The project’s impact is rated satisfactory. Of the two major project components, the capacity development of the OUSL was most successful. The OUSL doubled its enrolments with project support, and in 2014 provided significant proportions of the country’s graduates in high-demand fields such as law, science, and engineering. Other significant impacts occurred through (i) the accreditation and performance standards developed for distance education and now accepted as the basis of a university accreditation system, (ii) the successful strengthening of the management and staff capacity of the OUSL, and (iii) the building of significant capacity across the tertiary education sector for the development of online courses. Executive Summary xiii

Notwithstanding that the online architecture as designed did not take off, the project raised awareness of the potential of online learning and has influenced the way universities have started to explore this option.

B. Innovation and Risk in Project Design and Implementation

The defining characteristic of the DEMP is that it was a visionary project, albeit one that ultimately failed to achieve its vision as initially designed. However, the PPER finds that ADB should have continued to support the development of this concept with the possibility of exploring other avenues of financing and implementation, given the world trend toward online education. Several key issues arise in relation to this nexus between innovation and failure to succeed.

Innovation and risk. Innovative projects are inherently risky in that they almost always attempt something that has not been tried before. Such projects therefore have a higher degree of risk than others. It is difficult to identify all the possible risks, and to ensure that risk mitigation measures will be effective. All these factors contribute to a higher than usual likelihood that an innovative project will not succeed. At the same time, innovation is being actively encouraged. How much risk is acceptable? If ADB is to consider innovative projects for lending, both parties should explicitly acknowledge the risk of failure.

Time and resource allocation. Projects that are new and innovative require more time and resources during preparation and implementation. In the case of the DEMP, insufficient time was allocated up front to determine whether the project was feasible. This could have been done by assessing (i) whether the demand for online education was sufficient and (ii) whether the expected public and private partners would commit to the project. Although the DEMP was a 6-year project, this was not long enough to bring about the dramatic changes envisaged. The resources devoted to project implementation by ADB were also insufficient. Before innovative projects proceed, both ADB and the borrower should confirm that they are prepared to commit more time and resources than is usual to optimize the chances of success.

Evaluation of innovative projects. Should projects with a high degree of innovation be evaluated and rated according to the same strict requirements as other projects? In the case of the DEMP, for example, while it failed to meet most of its specific outcome targets, it did open people’s minds as to the value of ICT in teaching and learning, and laid the groundwork for later developments.

Sustainability and innovation. When new approaches are being tried, it is often not possible to determine in advance the best way to ensure sustainability. The DEMP had no agreed sustainability strategy initially because it was not known how the distance education network would develop. Neither was a sustainability strategy developed over the course of project implementation, despite intensive effort and lobbying by the project implementers. It is ironic that this flexibility in support of an innovative approach, while positive in itself, allowed the issue to drift unresolved for several years. This led to the ultimate failure of NODES. The competing pressures of flexibility and the need for commitment to a clear path require careful balancing and management.

xiv Distance Education Modernization Project

C. Lessons Arising from the Performance Evaluation

Ensure projects have a clear and achievable focus. The design of the DEMP was very broad, with the project inputs spread too thinly. The project included the construction of a national broadband network; mobilization of nontraditional partners; capacity building and institution strengthening within the OUSL; development of online courses, which is notoriously difficult and slow; and implementation of these courses with the expectation of rapid, high enrolments. The DEMP may have had more chance of success if it had initially focused only on the OUSL, moving to the establishment of NODES as a second phase.

Allocate significant resources for preparation of innovative projects. For innovative projects, significant resources are needed up front. This can include resources for assessing the feasibility of, and demand for, the project objectives and approaches for achieving them, as well as to lay the groundwork for ensuring their acceptability among different interest groups. Resistance to the innovations proposed cannot be overcome at the same time as project implementation is proceeding. The same applies to demand for the project outputs and outcomes. It is critical that a reasonable amount of consensus and commitment toward the project objectives is established before implementation begins.

Require expanded risk matrixes for all innovative projects. Considerable time, effort, and emphasis must be placed on the development and monitoring of the risk matrix for innovative projects. This applies particularly to risk identification, and potential risk mitigation strategies. A range of alternative strategies should be developed in detail for every identified risk, with a common understanding and commitment from ADB and the government. This should be a major focus of ADB- government discussions in project review missions, and should be updated at least annually. A sharper focus on emerging risks, development of appropriate strategies to address these risks as they emerged, and a strong joint commitment to implementation of these strategies would have been very helpful for implementation of the DEMP.

Ensure adequate ADB support for project implementation. By definition, the knowledge, skills, and experience needed to guide and embed innovative change are likely to be scarce in both the borrowing countries and in ADB. This was certainly the case for the DEMP. Additional time and resources are required to support implementation, which should include longer, more intensive, and more frequent review missions, and supplementation of missing technical skill areas on the part of ADB. Additional technical assistance to strengthen the expertise available under the loan and ensure ongoing support to implementation may also be necessary. If the sector department is not committed to providing additional resources and support to a given innovative project, then that project should not proceed.

Ensure sustained investment. Fundamental change to education systems is complex and does not happen quickly. When an entirely new system is being created, as it was under the DEMP, it is critical that a commitment to sustained investment is made. The DEMP was planned, it seems, as a one-off investment. Innovation and reform require long-term commitment to be successful.

Agree on a sound strategy for sustainability. The experience of the DEMP illustrates the importance of developing and operationalizing sound sustainability strategies. The project paved the way for an innovative online learning system, but the lack of a clear long-term plan was one reason why the concept eventually failed. If the Executive Summary xv

best strategy for sustainability is difficult to specify at the outset, specific support and additional time should be allowed in the project design to prepare and operationalize an agreed sustainability plan.

Ensure that the project performance indicators are appropriate for the time frame. In most education projects, the focus of project implementation is on development of systems or programs that are only completed near the end of the project. The final achievement measures, however, are generally framed in terms of the number of student beneficiaries. The DEMP was no exception. Apart from the time required for program development, it generally takes at least one program cycle to convince students that it is worthwhile. The DEMP may have been better as a multiphase project with appropriate targets for each phase, to allow time for incremental development of the national network.

Innovation needs champions. Effective champions are critical for the success of any new initiative. This is even more the case for innovative projects, where strong government and ADB ownership and commitment are critical for project success. The failure to ensure that the DEMP remained physically located within government agencies with a strong commitment to the project, especially following changes of government, was one of the factors that undermined its success. Similarly, the failure of ADB to adequately support project implementation, and to assist the government in identifying and implementing a practical solution for long-term sustainability after the project ended, also contributed to the project’s limited success.

Future potential of online education. The DEMP failed to expand tertiary education opportunities and increase graduate employability to the extent intended. One of the lessons of the DEMP is that online learning is not a magic bullet. Sri Lankan teachers and students were not ready for fully online courses, at least for longer programs. A combination of online learning with a limited amount of face-to-face tutoring appears most appropriate. Some universities have established their own open and distance learning centers, and these could be considered for future ADB assistance, along with support for further capacity expansion at the OUSL.

xvi Distance Education Modernization Project

CHAPTER 1 Introduction

1. This project performance evaluation report (PPER) assesses the Distance Education Modernization Project (DEMP), which aimed to help the Government of Sri Lanka modernize the postsecondary education system, in particular through the introduction of online distance education.1 This was intended to expand postsecondary education opportunities for the large numbers of secondary school graduates unable to gain admission to conventional universities, particularly those in conflict-affected and plantation areas, and was in line with the government’s strategy to strengthen its human resource base. It was intended that this expansion of opportunities would, in turn, promote economic growth and social cohesion across the country.

2. The capacity, quality, and relevance of learning through online distance education was to be boosted through the establishment of a national distance education network for online learning, with participation of both public and private providers, and upgrading and capacity expansion of the Open University of Sri Lanka (OUSL). The promotion of new public-private partnerships to expand education and training opportunities relevant to employment was central to the introduction of online distance education.

A. Evaluation Purpose and Process

3. The project was selected for a performance evaluation because of the innovative approach taken to the expansion of tertiary education opportunities. Both online distance education and public-private partnerships in tertiary education were new to Sri Lanka. The project was evaluated against the four criteria of relevance, effectiveness, efficiency, and sustainability. Although it does not affect the final rating, the impact of the project was also assessed. In particular, the evaluation assesses the project’s success in expanding tertiary education opportunities. The evaluation’s findings will feed into the Sri Lanka Country Partnership Strategy 2016-2018.

4. The project completion report (PCR) was prepared in December 2011 and the project validation report (PVR) in December 2013. 2 The PCR assessed the project outcome and outputs against the revised targets. Both the PCR and the PVR rated the project less than successful. The PCR rated the project highly relevant to the government’s education policies and poverty reduction strategies, as well as to the Asian Development Bank (ADB) country strategy. However, this rating was downgraded to relevant by the PVR. The project received less than satisfactory ratings on all other dimensions. Both the PCR and the PVR rated the project less effective. The project did successfully establish a national platform for online distance education, comprising the

1 ADB. 2003. Report and Recommendation of the President to the Board of Directors on a Proposed Loan to the Democratic Socialist Republic of Sri Lanka for the Distance Education Modernization Project. Manila (Loan 1999-SRI). 2 ADB. 2011. Completion Report: Distance Education Modernization Project in Sri Lanka. Manila (Loan 1999- SRI); ADB. 2013. Validation Report: Distance Education Modernization Project in Sri Lanka. Manila.

2 Distance Education Modernization Project

Distance Education Partnership Program (DEPP) and the National Distance Education Network (NDEN), which embraced the country’s poorer regions. It also expanded student access to a range of postsecondary education opportunities. However, the range of online courses and the number of students accessing them was lower than expected. The project was rated less efficient by both the PCR and PVR because implementation was slow, with the launch of online courses delayed. The resulting lower numbers of students (the project beneficiaries) meant that the cost efficiency was low and the rate of return envisaged at appraisal could not be realized. The project was also rated less likely sustainable because the revenue generated from student fees was insufficient to sustain the organization. In addition, a corporate entity to succeed the DEPP and put it on a business footing had not been established by the time of the PCR.

5. The project was assessed by the PVR as having had a moderate impact, on the basis of what was reported in the PCR. Although the network had not by 2011 managed to solidify its reputation and build a solid customer base, there was potential for this to happen. The project had demonstrated the power of online distance education to reach poorer students, utilizing public-private partnerships, and the potential for achieving this cost-effectively. The downgrading by the PVR of one of the ratings in the PCR, and also its less than satisfactory rating of the PCR itself, was strongly rejected by the division concerned.

6. Because of the potential significance for a follow-on project, the PVR recommended the carrying out of a PPER once the government had implemented a sustainability plan for the DEPP. By that time, the benefits of the project were expected to be more evident.

B. Expected Results of the Project

7. The expected outcome of the project was the development of online distance education to expand postsecondary school enrolments and develop a modern, high- quality human resource base. The outcome was to be achieved through three major outputs: (i) increased capacity and raised quality in online education through the establishment of the DEPP; (ii) better quality programs in distance education through a consortium of public-private partnerships; and (iii) expanded capacity and upgrading of the OUSL. The design and monitoring framework is in Appendix 1. The project was implemented over 6 years, from 4 November 2003 to 28 July 2010, and was national in scope. It should be noted that the DEPP was renamed the National Online Distance Education Service (NODES) by the government in 2011, after the project ended, and is referred to as NODES throughout this report.

8. The outcome and output targets were ambitious and somewhat unrealistic in a country with no previous exposure to online learning and a low level of internet penetration, which continues until today.3 OUSL enrolments were expected to double to 40,000 by 2012, and 150 institutions offering degree and diploma programs were expected to participate in the network. Private institutions were expected to enroll 35,000 students in online diploma programs and public universities 25,000 students in online degree programs. In addition, 20,000 unemployed school leavers were to be given short-term, semiprofessional job training. These targets were revised after the midterm review in 2008, 18 months before the project closed. The target of establishing partnerships with 150 institutions was redefined as the provision of 40 online degree and diploma programs, and the target number of external university

3 World Economic Forum. 2015. The Global Competitiveness Report 2015-2016. Geneva. Introduction 3

undergraduates 4 and private diploma students was reduced to 10,000 from a combined 60,000. The number of unemployed school leavers to receive skills training was similarly reduced to 8,000 from 20,000. The enrollment target for the OUSL was not changed as this was on track to be achieved.

C. Evaluation Method

9. The PPER reviews the project achievements as of 2016 in terms of the original expected outcome and revised outcome indicators. Interviews were conducted with key stakeholders in government, conventional universities, and the OUSL in Colombo. OUSL staff, students, and other informants were also interviewed in four study centers in the conflict-affected Northern, Eastern and North Central provinces, as well as a tea plantation area. The list of people met is in Appendix 2. Information was gathered to assess the impact of NODES in these areas, its potential for future sustainability, and the success and impact of the upgrading of the OUSL. The evaluation also assessed the future potential for online distance education as a contributor to human resource development in Sri Lanka.

10. The project design and implementation placed a strong emphasis on supply factors but less attention was paid to the demand for online education. One of the consequences, noted in the PCR, was that the project failed to build widespread acceptance of distance education, which contributed to the low enrolments achieved by project end. To shed some light on the experience with, and attitudes toward, online distance education among current students, a short questionnaire was administered to 104 students who were present in three of the OUSL study centers visited during the PPER mission. The interviews with key stakeholders and informants form the basis for the evaluation of the project, illuminated where appropriate by the views of the students.

4 External students are registered in courses with public universities but study the syllabus independently. Most universities subcontract a private company to do the tuition. External degrees are 3-year general degrees, while degrees for internal students are 4 years with more specialized content. External and internal degrees are recognized by the government as equivalent, as both types of degree are conferred by the University Grants Commission.

CHAPTER 2 Design and Implementation

A. Formulation

11. In 2000, the government requested technical assistance from ADB to support postsecondary education outside the conventional university system. 5 The initial proposal from ADB to focus support on the OUSL was subsequently broadened in response to this request. At the same time, the government requested the World Bank to help rationalize conventional universities. The ADB project was prepared during 2002. A strongly participatory strategy was followed during project preparation, including extensive workshops and consultations with public and private employers, the Ceylon Chamber of Commerce, nongovernment organizations, funding agencies, students, university faculty, government policy-makers, and educators. The project interventions were well conceived and were in line with programs implemented by the OUSL since its establishment in 1980.

12. The ministries of finance and education were closely involved in the formulation of the project, and were very supportive of the innovative technical solutions proposed. The mood in the country at the time was optimistic as it was thought the civil war was coming to an end, and the government was keen to unite the country and to boost the economy through development of the private sector. This would require a good supply of well-educated, qualified workers. The vice chancellor of the OUSL was an effective champion of the project as well, recognizing that the OUSL needed to modernize quickly in order to strengthen its reputation as a high-quality institution.

13. However, the political landscape was made up of many different interest groups. The government had already finalized new policies outlined in The Future – Regaining Sri Lanka and e-Sri Lanka: An ICT Development Roadmap, which emphasized the mainstreaming of information and communications technology (ICT) for broad-based tertiary education delivery. 6 Work had also begun on a specific ICT policy for education. However, the education community was divided on the issue. Public universities were more interested in upgrading and protecting their established teaching and research programs than in investing resources in distance education. Some, but not all, of the private training institutions were keen to develop online courses but had to be sure of generating a quick profit. Students were opposed to any ‘privatization’ or involvement of the private sector in tertiary education. Even the OUSL itself was divided, with the leadership and regional staff aware of the urgent need for revitalization, while others were reluctant to change their ways. In combination with the ongoing civil war, this made for a difficult and risky environment in which to pursue innovation. While the technical assistance incorporated an understanding of the

5 ADB. 2000. Technical Assistance to the Democratic Socialist Republic of Sri Lanka for the Postsecondary Education Modernization Project. Manila. 6 Government of Sri Lanka. 2002. The Future – Regaining Sri Lanka. Colombo; Government of Sri Lanka. 2002. E-Sri Lanka: An ICT Development Roadmap. Colombo.

Design and Implementation 5

politics and associated project risks, more attention should have been paid to developing effective risk mitigation strategies, particularly in relation to the likely acceptability of the key project strategy, online distance education. For example, much more could have been done to convince the universities and private institutions of the merits of ICT-enhanced learning.

14. ADB and the World Bank worked alongside each other to develop the two projects and ensure complementarity, which resulted in a consistent approach across the higher education sector. The work being carried out by the governments of Sweden and Norway, the two major bilateral donors, on the reform of university management, information, financial, and accounting systems was also incorporated into the project.

15. The project investment modality was chosen because the new government’s policy reform agenda was still being elaborated in several contentious areas, such as the expansion of private investment in education, the planned liberalization of telecommunications, and the modernization of higher education. In addition, high levels of investment were required for upgrading infrastructure and developing software. ADB provided funding of SDR33,005,000 (equivalent to $45 million) from its Special Funds resources. The 6-year loan was approved on 11 June 2003 and closed on 28 July 2010. A 6-month extension was granted at the end of the project to facilitate financial closing.

B. Rationale

16. The project was formulated to address the need to upgrade the skills of the Sri Lankan workforce in line with the changing economy, the low labor-market relevance of much of the existing higher education, and the limited access to postsecondary education, particularly in rural areas. Low levels of government funding combined with a reluctance to charge fees had constrained the growth of public tertiary institutions. The government had spent only 3% of gross domestic product on education for the previous decade, with only 20% of this going to higher education, while private postsecondary institutions were generally viewed with mistrust and were not yet competitive. At the time the project was designed, only 3% of the relevant age group was enrolled in postsecondary education compared with 8% across South Asia and 20% in Thailand. This was in stark contrast to the situation in primary and secondary education, where sustained investments over several decades meant that 99% of the relevant age group was enrolled in primary education and 90% in lower-secondary education. Participation fell after the General Certificate of Education (GCE) O-Level examinations, when about half of the approximately 400,000 grade 11 students left school and entered the labor market, as only limited training opportunities were available. A further half of the students who continued to grade 12 passed the GCE A- Level examinations after grade 13 and qualified for university.

17. The imbalance between school and tertiary education enrolments meant that far more students qualified for university than could gain admission. In 2001, about 25% of the relevant age cohort, or around 100,000 students, qualified for university. Of this group, only 15% or 15,000 entered conventional universities, 5% went to the OUSL or technical colleges, and 10% subsequently entered external degree programs. This meant that around 65% or 65,000 students who had passed their A-Levels failed to enter a university and had to find other alternatives. External degree programs had been introduced by the government as a strategy to expand access to university education at low cost and were essentially distance education. Students studied for university examinations on their own and paid for private tutoring if possible. By 2001, 6 Distance Education Modernization Project

enrolments in these programs had ballooned to a high 166,000 because few students ever succeeded in completing their courses and graduating. The majority of external students were women studying arts and the humanities.

18. Despite this high degree of selectivity, unemployment among educated youth was higher than average. While the national unemployment rate was 8.0%, it was 14.6% for those with A-Levels or higher. Conversely, it was only 1.9% among those with only lower primary schooling. This pattern was due in part to the nature of conventional higher education, much of which did not cater to the emerging needs of the labor market. The economy was in transition, with the share of agriculture declining and manufacturing and services increasing. The public sector was contracting while private sector employment was expanding. The skills required by private sector employers centered on computer science, financial management, marketing, and English, skills which many graduates lacked, especially graduates from external degree programs.

19. The project proposed to address these issues through the use of online distance education. Online education could be used to expand postsecondary education cost- effectively through a new national virtual network, and would be particularly appropriate for expanding access to training opportunities for students in rural, conflict-affected, and plantation areas. The project built on the national outreach system and the print-based distance learning programs of the OUSL, but aimed to broaden the reach of the system by involving conventional public universities and private training institutions as well. It was expected that the private institutions would provide focused, diploma-level skills training better suited to labor market needs, especially in high-demand fields such as business, computer science, English, and engineering. The public institutions were expected to put their external degree programs online, which would be of great benefit to external degree students. The solution was elegant and innovative in the way that it brought together many different elements to solve several problems at once.

C. Cost, Financing, and Executing Arrangements

20. At appraisal, the project cost was estimated at $60 million, toward which ADB provided a loan from its Special Funds resources equivalent to $45 million, the government provided $10 million, and beneficiaries provided $5 million. The actual project cost for ADB, the government, and beneficiaries was $46.99 million, while the amount disbursed from the ADB loan was $36.2 million (see Appendix 3). Three partial loan cancellations were made totaling $14.5 million equivalent. The lower than expected disbursement was due largely to a reduction of output targets for the number of online programs to be developed, but also partly to the appreciation of the special drawing right (SDR) against the US dollar that increased the US dollar loan amount by 13%.

21. Three changes were made in the executing arrangements. Initially, the Ministry of Policy Development and Implementation was the executing agency, with the Ministry of Tertiary Education and Training as the implementing agency. This changed in 2004 when the Ministry of Education was appointed as both the executing and implementing agency. In 2005, these functions passed to the Ministry of Higher Education, where they remained until the project closed.

22. The implementation arrangements were innovative but complex. The project management office (PMO) was established by the Ministry of Tertiary Education and Design and Implementation 7

Training at the start of the project. The PMO was never physically located within a government department, which did not encourage close day-to-day working relationships. The project steering committee (PSC) was the top decision-making body, chaired by the secretary of the executing agency and with broad membership of all key stakeholders. The PSC met every quarter, and was assessed by the PCR as effective. Several teams operating under these structures were established to implement the different components, all coordinated by one project manager. The PCR noted that, while the implementation arrangements were elaborate and comprehensive, they were marred by a lack of coordination among stakeholders and weak links to policy-makers. These weak links resulted in several policies developed under the project and key to its sustainability failing to gain government approval.

D. Procurement, Construction, and Scheduling

23. The project was implemented from 4 November 2003 to 31 November 2009 in accordance with the original time frame. Financial closing was on 28 July 2010. Project implementation was nevertheless marred by delays stemming partly from an initial 9-month delay in recruiting the consultants. The construction of the national ICT network was particularly affected by the delayed consultant recruitment, as the new courses could not be uploaded until the network was operational. The development of a public-private partnership policy also started 2 years behind schedule, which delayed initiation of the partnerships themselves. The social marketing campaign also occurred too late in the project to attract sizeable numbers of students. All of these delays, combined with the smaller than planned ICT network, meant it was not possible to achieve most of the original outcome and output targets.

24. Civil works and equipment comprised 58% of expenditure, with both requiring additional allocations to cover cost increases and local currency depreciation. Some delays in the procurement of equipment occurred due to limited availability and the lack of local suppliers, but the suppliers generally performed satisfactorily. The performance of the contractors was also generally satisfactory, even though 2 of the 14 civil works contractors did not perform satisfactorily and had their contracts terminated.

E. Outputs

25. The project had three major outputs intended to develop distance education: (i) establishment of the distance education partnership program or NODES, (ii) establishment of a consortium of public-private partnerships, and (iii) expanded capacity at the OUSL. The updated design and monitoring framework summarizing the physical accomplishments is in Appendix 4.

26. Establishment of the National Online Distance Education Service. The project was groundbreaking in its attempt to establish a national online learning network that any education or training provider could use to offer a wide array of courses to students all over Sri Lanka. A national distance education network comprising a network operation center, a content development unit, a disaster recovery center, and 26 multimedia network access centers (NACs) were successfully established to deliver distance education across the country, using up-to-date network technology. However, the building of the network took a year longer than planned as a result of delays in the hiring of technical consultants, and was not completed until the third quarter of 2007. The network was also much smaller than planned: the original target of establishing NACs in 150 partner institutions was found to be unachievable due to the lack of 8 Distance Education Modernization Project

willing partners, and the target was reduced at midterm to 40 NACs. Despite this reduction, the network still covered almost every province. All of the NACs had high- speed internet, connected multimedia computers, projectors, videoconferencing, tablets, scanners and printers, an in-house server, and generators. NAC staff members were trained to operate the network. The selection and development of the online platform judged most suitable for course delivery proceeded concurrently with the establishment of the network.

27. To identify the occupational skill areas most in demand, a labor market analysis that included a study of job listings and employer interviews was undertaken early in the project. Subsequently, the specific courses to be developed were identified. The skill areas identified included ICT, engineering technology, agro-technology, biological and physical sciences, business management and administration, marketing, business English, accountancy, teacher education, and hospitality. An online career guidance program with a focus on youth and women did not proceed because an appropriate specialist could not be found. The impact of this was cushioned somewhat by the fact that 75% of all NODES students were already in employment, with most aiming to upgrade through job-relevant qualifications. The career guidance system was to be linked to a new labor market information system (LMIS) that was to be developed under the project. This did not proceed either, as it emerged that the Ministry of Tertiary Education and Training was already developing an LMIS under the World Bank higher education project.

28. A critical part of the network was the content development unit that monitored and supported the development of online courses by partner institutions. This unit undertook an impressive program of capacity building, and succeeded in building a cadre of over 900 partner institution staff skilled in online course design, development, and delivery. Nevertheless, progress in course development was slow. Staff in the partner institutions had first to be convinced of the merits of online education before training could begin, and had only limited time to contribute to the task on top of their regular jobs. The limitations of consultant support also meant that the work of course development had to be staggered among the institutions, which further slowed the pace. As a result, when the network was launched in late 2007, only around 12 courses were available. Better progress was made in the last year of the project, when a further 28 courses were put online. By the time the project ended, there were 42 courses offered through the new network, which just exceeded the revised target of 40 courses (see para 8). The limited number of courses, combined with their late launch, was one of the main factors explaining the low levels of student enrolment achieved.

29. Project marketing activities in the form of general awareness-raising about online education proceeded in parallel with course development, but a more extensive social marketing campaign aimed at attracting students to specific NODES courses did not take place until the last year of the project. The campaign was not successful in delivering its messages to prospective students, which meant that levels of awareness about online education remained low. In any case, the campaign took place too late to have any chance of building up significant student enrolments.

30. Establishment of a consortium of public-private partnerships. A major achievement of the project was the bringing together of public and private training institutions under a common framework for the first time, although not on the scale originally intended. The original target for the consortium was 150 partner institutions; however, by the end of the project, only 24 institutions had joined, comprising 6 public universities (including the OUSL), 3 public institutes, 2 professional organizations and Design and Implementation 9

13 private postsecondary institutes. A list of the partner institutions, online courses, and 2011 student numbers is in Appendix 5. The private institutions were selected on the basis of labor market demand, and included technical colleges (offering skills development training), private non-profit professional organizations, and private for- profit schools (mostly offering language training, ICT, accounting, business, and commerce). The project provided matching grants to partner institutions to help with the start-up costs for online course development. The requirement to commit funds for these matching grants may have discouraged the private institutions from participating in this new venture. Despite having no financial requirement, the public universities were reluctant to commit to the new venture.

31. Interest in online education among partner institutions did increase during the final year, evidenced by the faster pace of course development at that time. It is possible that a project extension or a follow-on project may have allowed these benefits to be captured and further developed. The opportunity to link up with foreign institutions and offer online programs was also available for 150 institutions, but only two took up the offer. Stipends were available for 20,000 needy students undertaking online skills development courses, but the target for this type of training was too high, and the number of stipends was reduced to 8,000 at the midterm review.

32. Various policy initiatives were planned to facilitate and support the participation of private partner institutions, but were not fully achieved. The most important was a national policy for the assessment and accreditation of distance higher education and an ICT policy for online distance learning. The accreditation policy was developed under the project but was taken over for further development by the National Quality Assurance Agency for Higher Education. The policy developed under the project included accreditation standards, evaluation criteria, an accreditation framework and performance indicators that were considered so impressive by the Commonwealth of Learning that they were adopted for international use. Despite this, the government has approved neither the policy nor the establishment of the accreditation board for distance education, so it is still not possible for private institutions or programs to be accredited. A proposed policy on online education with provision for subsidized internet fees was planned, but it was agreed that this would be undertaken later as part of a broader ICT education policy.

33. Capacity expansion of the Open University of Sri Lanka. The lynchpin of the project was the OUSL, the only partner institution with experience in distance education. To expand the capacity of the OUSL, modern facilities and advanced laboratories for science students were established at the OUSL central campus. The library, multimedia center, and the printing press were upgraded to support the development of the curricula for multimedia online courses by 2006. Academic, technical, and administrative staff members were taught new skills and concepts through an intensive program of staff development, including post-graduate overseas training, long-term domestic training, and short-term overseas training. A management information system was put in place to strengthen and streamline administration, human resources management, budgeting, financing, student admissions, and student services. All of these systems were operating at the time of the PCR. In addition, as part of the physical network, 16 OUSL regional centers and 7 new study centers were built, and 3 existing study centers were renovated. This met the original target of 26 OUSL centers. New centers were established in rural and conflict areas and in poor plantation districts. 10 Distance Education Modernization Project

34. Existing OUSL courses were upgraded through the development of new instructional materials. Staff from all faculties attended training on online course development. As a result, 69 online courses were developed and added to the OUSL program as supplementary courses, of which two were fully online courses. All of these achievements reflect the determination of the OUSL leadership to modernize and move beyond traditional print-based distance education into a new era of e-learning.

F. Consultants

35. The consultants were recruited in three packages, one each for the OUSL, DEPP and NDEN. Recruitment of the consultants for the DEPP and the NDEN was delayed by 9 months. This problem was compounded by the need for several subsequent consultant replacements in the DEPP and OUSL packages. The delay in the recruitment of the NDEN consultants was a major cause of the holdup in establishing the network. Consultants were engaged for a total of 751.8 person months, a significant increase over the appraisal estimate of 552 person months. International consultants were increased by 14 person months, and national consultants by 176 person months. The increased need for consultants reflected the many challenges encountered in developing the network, especially the difficult task of modifying course content for online delivery. In addition to the three consulting packages, individual consultants were engaged for 61.5 person months (of which 59.5 person months were national), primarily to support the modification of course content for online learning systems.

36. Several consultants did not have the required technical skills and had to be replaced. The replacements generally took a long time, which delayed implementation. Finding an appropriate consultant to develop the online course for career guidance proved especially difficult. The consultants generally performed satisfactorily. They had cordial and professional relations with the staff of the executing agencies and their counterparts, technology transfer was effective, and a positive exchange of ideas took place. The international consultant inputs were staggered because the consultants were in high demand, which sometimes caused delays. Identifying local consultants with expertise in instructional design, management information systems, and network technology proved challenging, but those finally engaged proved competent and committed.

G. Loan Covenants

37. The PCR reports that, of the 31 covenants, 23 were assessed as fully or partially complied with, while 18 were assessed as complied with late. One covenant regarding the executing agency was modified following a change in executing agency, and another, concerning a tracer study to examine the employment of graduates from the online programs, was dropped during the midterm review because it was judged to be premature.

38. Covenants relating to the engagement of consultants, procurement of equipment, and civil works were complied with. Project accounts and audit reports were submitted to ADB in a timely manner. Completion of a baseline study of the internal and external efficiency of the OUSL, to determine future growth and funding requirements, was delayed by almost 3 years, mainly because the executing agency delayed the appointment of OUSL staff to the study. Only one covenant was assessed as partially compliant, covering implementation of the government’s national ICT policy in relation to distance education. While a policy paper on ICT in distance education was prepared, the government decided to subsume it into a policy covering ICT in Design and Implementation 11

education more broadly. A covenant covering the conduct of the public awareness campaign for promotion of the online courses, and establishment of a career guidance program with a focus on women was assessed as compliant even though the public awareness campaign was ineffective and took place late in the project, and the labor market information system that was to be the basis for the career guidance system was dropped. A more rigorous assessment of compliance may have resulted in more covenants being assessed as only partially compliant.

H. Policy Framework

39. The government in place at the time of project design and approval was committed to modernizing the economy and the labor market through a flexible education system with special emphasis on minimizing differences across regions, especially through ICT and new learning technologies. The government’s policy framework set out in The Future – Regaining Sri Lanka signaled its intention to pursue financial investment, public sector reform, and infrastructure development and productivity in key sectors (footnote 6). A central part of government policy was the modernization of the labor force though education and training for lifelong employability, support for self-employment, and encouragement of job creation for small and medium enterprises. At the same time, the government was also investigating education finance reform and public-private participation in tertiary education. Another key vision of the government was of using ICT to provide knowledge connectivity throughout the country. E-Sri Lanka: An ICT Development Roadmap set out a program to (i) develop national hardware and software infrastructure through a specialized agency for ICT strategy and policy, (ii) develop ICT human resource capacity, (iii) use ICT for broad-based tertiary education across the country, and (iv) move toward e-government service delivery (footnote 6). Despite the failure of the 2002 ceasefire agreement, the economy was continuing to expand and the pressure to develop a workforce more suited to the needs of the growing private sector remained strong.

40. The project was closely aligned with all of these policies and enjoyed strong support from key ministries, such as the ministries of planning and finance. However, a change of government in April 2004, just a few months after project approval, reduced this strong support. The new government did not have the same interest in ICT or in advancing online learning. A further blow came with the retirement of the project’s champion, the vice-chancellor of the OUSL, about halfway through project implementation. While OUSL leadership remained committed to the project, the drive to achieve reform across the distance education landscape was diminished. Both of these factors adversely affected the success of the project, as well as the fate of the strong policy agenda designed to support ICT and promote public-private participation in education. Finally, ADB did not demonstrate a consistent and strong commitment to the project, and failed to ensure that adequate technical support was provided.

CHAPTER 3 Performance Assessment

41. The overall assessment is based on the assessment criteria for relevance, effectiveness, efficiency, and sustainability, in accordance with ADB guidelines.7

A. Overall Assessment

42. Overall, the project is rated less than successful. The ratings for the individual criteria are shown in Table 1. The likely impact of the project, as well as the performance of ADB and the borrower, is assessed but not included in the final rating. The rating of less than successful is consistent with the assessment of the PCR and PVR (see Appendix 6 for a comparison of the PCR, PVR, and PPER assessments on the individual criteria).

Table 1: Assessment of Project Performance Rating Weighted Criterion Weight Assessment Value Rating Relevance 25 Relevant 2 0.50 Effectiveness 25 Less than effective 1 0.25 Efficiency 25 Less than efficient 1 0.25 Sustainability 25 Less likely sustainable 1 0.25 Impact Less than satisfactory ADB performance Less than satisfactory Executing agency Less than satisfactory performance Overall assessment Less than successful 1.25 ADB = Asian Development Bank. Note: Highly successful (>2.5), successful (<2.5 >1.75), less than successful (<1.75 >0.75), unsuccessful (<0.75). Source: Asian Development Bank.

B. Relevance

43. The project is rated relevant. The project was aligned with the government’s policies for education and human resource development at the time of design and at project end, and continues to be relevant today. The project was also consistent with the ADB Country Strategy and Program Update: Sri Lanka 2003-2005, which prioritized poverty reduction and encouraged pro-poor investments to strengthen public-private partnerships. 8 It responded to the government’s desire to expand tertiary education access cost-effectively, particularly in conflict-affected and plantation estate areas where access was low. This was further intended to help unify the country after the cessation of the conflict. The design cleverly harnessed the capacity of private institutions to help reduce inequality and improve the quality and job-relevance of

7 Independent Evaluation Department. 2016. Guidelines for Preparing Performance Evaluation Reports for Public Sector Operations. Manila. 8 ADB. 2002. Country Strategy and Program Update: Sri Lanka 2003-2005. Manila.

Performance Assessment 13

tertiary education. The project interventions were well conceived and generally appropriate, and were responsive to the needs of the sector. However, the project did not assess the demand for online education in sufficient depth, and did not include adequate measures to convince universities and external students in particular of the value of online education. The project was complementary to the activities of other development partners, particularly the World Bank. It also complemented other ongoing ADB projects aiming to improve the employability of school-leavers and university graduates through greater exposure to ICT, science, and English in secondary schooling, as well as the successful expansion and upgrading of the postsecondary skills development system.

44. Project justification. At the time of project design, only 3% of the relevant age group was enrolled in the conventional university system, and enrolments had stagnated. The participation rate has increased since then but remains very low, with less than 7% of the age group enrolled in 2014. 9 The government was both unwilling and unable to dramatically expand capacity. Other avenues for tertiary education were also limited. The RRP argued that the high level of enrolments in external degree courses indicated a demand for better quality distance education. In the search for alternatives, the project offered a practical and cost-efficient option for rural students in particular to pursue certificate, diploma, and degree programs. The OUSL already offered tertiary education opportunities to a large number of students, but the programs were all print-based. The project took advantage of this infrastructure and the recent innovations in ICT to revitalize the OUSL and establish an expanded national delivery system for online distance education. This involved both public and private partners, offering more employment-relevant training. A set of policies to support the inclusion of private institutions was also proposed. The project was an appropriate and relevant response to the limited access to tertiary education at both the beginning and end of the project. The involvement of skills training institutions offered a further range of job-relevant alternatives for school leavers at the certificate level.

45. Project design. The project outputs corresponded to the three major thrusts of the project and were linked clearly and logically to the project outcome. The link between the project outcome and the project goal or impact was less direct, as many factors intervene between the expansion of postsecondary education and increased socioeconomic growth. One important aspect of the design can be questioned: whether Sri Lankan students were ready for a purely online modality. Sri Lanka’s levels of internet penetration were low, at 3.7% of the population in 2008. In a survey of 70 Asian countries, Sri Lanka ranked 60th on e-learning readiness in 2003 and 59th on e-readiness in 2008.10 In the short survey administered to OUSL students as part of the PPER, half of the responses identified face-to-face teaching as the preferred instructional method. A further 20% preferred online learning supported by an online tutor, while only 10% preferred solely online instruction. These responses are not surprising. Many of the officials and academics interviewed for the PPER also commented that Sri Lankan students have a strong preference for face-to-face teaching and learning. This is likely to be true of the teachers as well, who generally do not have the skills needed to design and deliver online courses. In the context of distance education, where full face-to-face teaching is not an option, an approach blending online learning with face-to-face teaching seems most appropriate. These critical issues should have been explored in much greater depth before the project was designed.

9 University Grants Commission. 2015. Sri Lanka University Statistics 2014. Colombo. 10 ADB. 2012. Access without equity? Finding a better balance in higher education in Asia. Manila. 14 Distance Education Modernization Project

46. Finally, the design of the project was too broad, and many of the targets and time frames were too ambitious for a project introducing complex new infrastructure, new technologies, and new ways of designing and delivering courses—particularly in a context where there was some resistance. The project period should have been longer to allow time to achieve the targets. Alternatively, it may have been better to start with just the OUSL as a first step, moving to the establishment of the national network if online education through the OUSL proved a success. As it was, at the midterm review, the project outcome and output targets for the public and private partner institutions had to be reduced significantly. The targets for the OUSL were not changed. The changes made to the project targets were appropriate given these delays, and the new targets were met. However, the large reductions inevitably limited the impact of the project. No major changes were made to the project scope.

47. Risk analysis and risk mitigation. Any project pursuing significant innovation or reform is inherently risky. The DEMP was bold in its solution to the problem of restricted tertiary opportunities, and incorporated many new elements including new technologies and new partnerships. It also faced resistance from various vested interest groups. Little in the way of past lessons were available to guide the design, as ICT-enabled education had not been attempted in Sri Lanka before. Nevertheless, the project design and monitoring framework managed to identify most of the major risks (described as assumptions) to the project’s success. These assumptions included: (i) that partner institutions would have the motivation and capacity to effectively employ distance education, (ii) that universities would be willing to participate in a joint venture rather than proceeding alone, (iii) that external degree students would recognize the quality and convenience of NODES, (iv) that NODES would provide high-quality programs that would attract students, (v) that the government would implement a public-private partnership policy in postsecondary education, (vi) that public and private institutions would cooperate to improve capacity, and (vii) that the OUSL would be willing to modernize. All of these risks were adequately identified; however, the key risk of the acceptability of online distance education was not recognized.

48. The strategies for risk mitigation were less adequate. A central assumption was that the demand for online distance education would grow rapidly as an automatic response to the huge unmet demand for tertiary education. However, in the 5 years prior to the project, the OUSL had suffered from declining enrolments in its distance education courses, a fall in its reputation, and marginalization from the other universities. While the project aimed to rejuvenate the OUSL, it was likely that the years of decline had given distance education in general a bad name that would take time to counter. Students and their parents were likely to be especially cautious given that fees are paid for all distance education courses. Additionally, no interventions were made to convince universities to put their courses online, nor to persuade external degree students to take up online studies under NODES. The design did include a limited information and awareness-raising campaign to help stimulate student enrolments, but it was ineffective and did not succeed in generating the level of demand expected. Building demand for the new courses was always going to be difficult, given that no prior programs existed by which students and parents could judge the employability of graduates. This should have been a major focus of early project, or even pre-project, activities.

49. Government ownership. Ownership of, and commitment to, the project by the government and the OUSL was high at the time of project design and initial implementation. However, this was weakened over time by political changes and Performance Assessment 15

ministerial restructuring that led to frequent changes of executing agency and leadership of the project. Over the 6 years of implementation, the project was moved among three different ministries and ministers, and was led by six ministerial secretaries. This made it difficult for the project to establish a clear institutional home and diminished the level of ownership and commitment to the project by key government officials.

50. Continuing relevance of the project. The project focus on identifying alternative avenues and modes of study for higher education, with emphasis on ICT-enabled distance education, remains relevant. The restrictions in access to higher education in Sri Lanka have barely eased, as shown in Table 2. Of the almost 150,000 students who passed the final school examinations (GCE A-Level) in 2014, only 25,200 or 16.8% were admitted to conventional university studies (as on-campus or ‘internal’ students), and another 6,400 or 4% enrolled at the OUSL. This leaves around 80% of the students who passed their A-Levels without any satisfactory options for higher education. Many registered as external university students (43,000 or 30% of students who passed their A-Levels), but the quality and employment outcomes of these programs are extremely poor. This left more than 75,000 students who could not access university studies, in addition to a further 97,000 who did not pass the A-Level examinations and required other forms of tertiary training. There remains good potential for effective distance education to expand educational opportunities to these groups, in fields with good employment prospects, whether fully online or through blended learning with some face-to-face teaching. Online education has exploded in popularity around the world, and Sri Lankan students are now also studying online with private institutions. Different technologies for content delivery, such as mobile phones and tablets, are promising options for countries with low internet usage and coverage.

Table 2. Intake and Output for Public Universities, 2014

Intake Graduate Output Graduation Total Total % Total % Item rate (% of students Females Female Students Female intake) A-level students 2014 247,376 143,604 68 149,489 65 60 Students not passing 97,887 46,653 48 Admitted to Degree Programs 2014 On-campus (conventional 25,200 15,694 62 15,518 61 55 university studies) Off-campus (external students) 42,755 32,843 77 11,673 77 41 OUSL 6,367 3573 56 1,041 62 4 Total 74,322 52,110 70 28,231 68 100 A-Level graduates not 75,167 27,729 57 admitted to degree programs % admitted as on-campus 16.8% 16.2% students % admitted all students (on- 50% 54% campus and off-campus) OUSL = Open University of Sri Lanka. Notes: Internal students study on-campus and attend lectures, tutorials, seminars, etc. as conventional university students. External students register with a university, receive a syllabus, and then study off-campus independently or through a private tutoring company. Sources: University Grants Commission. 2015. Sri Lanka University Statistics 2014. Colombo; Open University of Sri Lanka. 2014. Statistical Handbook 2014. Colombo. 16 Distance Education Modernization Project

51. In sum, the project is rated relevant because it responded to government and ADB national and sector objectives, and it addressed the key issue of tightly restricted access to tertiary education in an imaginative and innovative manner. The lack of access to tertiary education continues to be a key sector issue today. As a new and innovative project, it faced many risks to success, not all of which were mitigated adequately. The project design was too complex, too broad and, most significantly, did not anticipate the resistance to online education among both students and providers. This only became apparent once project implementation began. While the demand for more traditional distance education was steady, as evidenced by the growth in enrolments at the OUSL, demand for the new online education was weak. More groundwork and investigation was needed before launching such an ambitious investment project.

C. Effectiveness

52. This evaluation rates the project less than effective, as it was by the PCR and PVR. The project outcome was to develop distance education to expand postsecondary school enrolment, in order to develop a modern, high-quality human resource base. Although the project successfully created a national platform and online delivery mechanism for distance learning, utilization of the network by providers and students remained low. The project did not succeed in developing the online distance education system to the level where it could significantly expand postsecondary school enrolments or ease the bottleneck in postsecondary education.

53. The outstanding achievement of the project was the establishment of the NDEN, linking rural and urban regions across the country to Colombo. Online courses could be offered through NODES by both public and private institutions, operating within innovative postsecondary education partnerships established by the project, through a sharing of common resources and activities. The introduction of these partnerships was a noteworthy achievement, in a sociopolitical environment where cooperation with private institutions was not common. Together, these accomplishments meant that, by the end of the project, Sri Lanka had a robust national platform through which all postsecondary institutions could potentially offer online courses.

54. As it was clear that the original outcome targets relating to expansion and utilization of the network could not be met, the following targets were revised at the midterm review: (i) the number of students undertaking external degrees through the NODES system was reduced from 25,000 by 2012 to 10,000 by 2009; (ii) the target of 150 public-private partnerships operating among postsecondary institutions was replaced with a target of 40 programs to be offered by partner institutions by 2009; (iii) the expected increase in private school enrolments was reduced from 35,000 by 2012, to 10,000 by 2009; (iv) the number of unemployed school leavers receiving semiprofessional job training was reduced from 20,000 to 8,000, and the age limit increased from 21 to 30. The targeted increase in OUSL enrolments from 20,000 to 40,000 by 2012 remained unchanged.

55. The project succeeded in providing online degree, diploma, and certificate courses for 33,347 students by project end in 2009, which was one-third of the original target of 100,000 students by 2012, and 85% of the revised target of 40,000 by 2009 (see Table 3). To put this in context, the final enrollments were around 5% of all higher education enrollments. This was far from the original vision of NODES as a major Performance Assessment 17

facilitator of tertiary education that would provide opportunities to 1.4 million students by 2020.11 Why was the original vision for NODES not achieved?

56. First, the unrealistically high targets would always have been impossible to achieve within the time frame, given the complexity and breadth of the design. This challenge was compounded by the technical difficulties involved in constructing a national high-speed virtual network and developing large numbers of online courses. Second, while some demand existed for online education, which did pick up toward the end of the project, the central issue was that student enrolments were much lower than anticipated. Although mass distance education was beginning to show success in other Asian countries (footnote 12), in Sri Lanka it faced resistance among students and wariness among education and training providers. Many factors explain the failure to attract high numbers of students, of which the most important are discussed below.

Table 2. Students Studying through the National Online Distance Education Service, 2009 2011 Enrolments Original Revised Degree Diploma Certificate Total Target Target OUSL 20,000 20,000 20,000 Public universities, 499 499 25,000 10,000 external degrees Private institutions 970 970 35,000 Private institutions, 11,878 11,878 20,000 10,000 skills training Total 33,347 100,000 40,000 Note: OUSL was targeted to increase from 20,000 students pre-project to 40,000 by 2012, an increase of 20,000. Source: Government of Sri Lanka. 2010. Distance Education Modernization Project. Project Completion Report. Colombo.

57. Difficulty of establishing partnerships. A major factor was the lack of success in forming partnerships between NODES and the institutions. This meant that only a small number of online courses were available by the end of the project. The government completion report notes that it was extremely difficult to draw the leadership and staff of universities and public and private institutions into online learning. Even the generous grants available for course development and purchase of materials failed to attract them. The RRP targeted external students as a large source of potential course participants, but did not foresee that the universities would resist putting their courses online to the extent that they did. Officials knew little about online education and were wary of the costs of providing ongoing learner support, which they felt would be too high for students. 12 University investment priorities generally lay elsewhere. The University of Moratuwa, which developed a well-respected and popular fully online degree program in informatics, was one outstanding exception. Some private institutions were initially enthusiastic but found the development process protracted and the financial returns too slow. Given the costs of developing online courses, the matching grant funds required, and the small size of many private institutions, it is not surprising that these institutions were reluctant. Online courses must attract large numbers of students to be profitable.

58. While the decision to include public, private, and professional institutions and organizations in NODES was one of the highlights of this innovative design, it also

11 Footnote 1, p 16. 12 Interviews during the PPER mission; and C. Pereira. Online Learning and Examination System for State University Entrants. Pre-Orientation Programme. In ADB. 2013. ICT for Development Forum 2013. Manila. 18 Distance Education Modernization Project

proved the most difficult to implement. Different approaches to implementing the partnerships could have been trialed before or during the project. More effort was also needed to help overcome their initial resistance.13

59. Slow development of the online courses. A second factor was that the slow development of online courses meant that most became available only in the last year of the project. The first step was to identify courses relevant to labor market needs, which was done through a lengthy labor market survey. The second step, the development of the online courses, is always time-consuming and requires special expertise, a fact that was initially not appreciated by partner institutions. This meant that the project staff first had to do a lot of awareness-raising. Through this process, the project built significant expertise in content development across institutions through the training of 900 staff; nevertheless, the slow development of courses had a significant negative impact on project success.

60. Accessibility and affordability. While it is unlikely that physical accessibility was a major deterrent given the comprehensive network of centers across the island, having all the courses in English would have limited accessibility for rural students. The fees charged for distance education programs are also likely to have been a deterrent for some students, especially non-working students (60% of the students surveyed paid the fees from their salaries).

61. Preference for degrees from conventional universities. Finally, students in Sri Lanka have a strong preference for a conventional university education, which partly reflects the strong preference for public sector jobs. In line with its aim to increase the market relevance of tertiary education, the project favored vocationally oriented certificates and diplomas in addition to the degree programs. It should have been expected that demand for these courses would be low, at least initially. While a degree has a recognized value, this was not the case for the new online diplomas and certificates. In the absence of an accreditation system for distance education, students would wait until the value of such courses was demonstrated through employment outcomes. The lack of external validation would have harmed the potential for private institutions especially to attract students to their courses.

62. The PCR observes that a comprehensive social marketing campaign would have helped generate demand. Some outreach was conducted to officials but the social marketing to students was limited and less effective. It is possible a campaign would have stimulated greater demand to some extent, but caution on the part of students and parents would probably have persisted anyway, especially as the courses were fee- paying and their pay-off in the labor market was not yet evident. Strong demand for courses is generated once they are seen to be successful.

63. In summary, the project had many noteworthy achievements, albeit on a smaller scale than originally intended. By the end of the project, it had provided opportunities for over 33,000 tertiary students in online courses that had been identified as providing skills relevant in the labor market. This was around one-third of the original target and just below the revised target of 40,000 students. The complexity of the project, resistance and caution among the intended partner institutions, the late uploading of online courses, and hesitation about their value on the labor market

13 ADB. 2013. Is ICT the Missing Link in Development Operations? Session on ICT for Education. ICT for Development Forum 2013. Manila. 28 February–1 March. Performance Assessment 19

(which was as yet unproven) were the primary factors behind the failure to attract large numbers of students to online education.

D. Efficiency

64. The project is rated less than efficient, which concurs with the assessments of the PCR and PVR. The PCR did not recalculate the economic internal rate of return (EIRR) because the low enrolments rendered it highly unlikely that the EIRR envisaged at appraisal would have been achieved. For the same reason, the decision was taken not to recalculate the EIRR for the PPER.

65. Implementation was handled reasonably efficiently given the challenging nature of the project, in particular, the development of the high-speed optical fiber network. But the delays in establishing the network and developing courses resulted in too little time for courses and enrolments to grow.

66. The National Online Distance Education Service was not cost efficient. One reason distance education is attractive to policy-makers is its perceived cost efficiency. Large numbers of fee-paying students can offset the significant costs of course development and operational costs. In 2011, when NODES was still functioning, the PCR noted that the courses were not cost efficient because of the high costs of internet provision in Sri Lanka. While the DEMP was in operation, all of these costs were covered, but they had to be paid by the partner institutions after project closure. This was unaffordable at the time, given the low enrollments, but this did not mean that NODES would not be cost efficient in the future. The business plan prepared by the DEMP estimated that it could become self-financing in about 6 years, with 16,000 course registrations.

67. The Open University of Sri Lanka proved more cost efficient than the National Online Distance Education Service. From 2003-2014, the OUSL increased its level of cost recovery from 25% to 42%, which exceeded the project target of 40%. Between 2003 and 2014, fee revenue increased more than nine times, while unit costs increased around 2.5 times (see Appendix 7). This was possible partly because of the increase in the number of students that occurred at the same time, which indicates that the OUSL could become even more cost efficient if enrollments were expanded further. However, a balance must be found between cost efficiency and affordability. The current level of OUSL cost recovery appears to be on the cusp of affordability for rural students. Half of those surveyed for the PPER considered the fees charged for their OUSL courses to be manageable, while the other half considered them to be too high. A greater proportion of students in plantation areas would probably find them too high. Therefore, in light of the high costs of internet access through the NACs and the capacity of rural students to pay, it is likely that the fees required to make NODES 100% self-financing would have been unaffordable for students.

E. Sustainability

68. The project is rated less likely sustainable. The PCR and PVR also rated the project as less likely sustainable because the steps taken by that time to ensure the sustainability of NODES were judged inadequate. The PCR argued strongly that the establishment of NODES as a corporate entity was essential for success and sustainability. Well before project closure, the guidelines for the formation and operation of a successor corporate entity had been drawn up. The legal framework, management structure, and business plans were for a fit-for-purpose, professional 20 Distance Education Modernization Project

organization. The move to a new organization was regarded as urgent so as to protect the NDEN and the NODES platform from deteriorating through a lack of maintenance and investment.

69. Nevertheless, this did not happen. The proposed corporate entity was not established because student enrolments by the end of the project were not high enough to sustain it. In 2011, the management of NODES was transferred to a unit operating under the Ministry of Higher Education (MOHE). The government provided a further 3 years of funding support (2011-2013) to ensure its continued operation and to give NODES the time to build up enrollments and revenue. From the beginning, though, the outlook was not positive. A new ministerial secretary with no previous experience of NODES took up office. Many administrative problems were encountered during the handover, particularly in regard to the 56 NODES staff. No provision had been made for them within the ministry structure, and there was no way to pay the salaries, which reflects a lack of preparedness and perhaps commitment on the part of the government. The anxiety and time spent without salary—as well as the subsequent contract arrangements, lower salaries, and the insecurity of the employment—resulted in deteriorating staff morale. The commitment and energy required to make NODES bigger and better was no longer present. Gradually, key staff began leaving for alternative employment.

70. During this time, no substantial new courses were added to NODES, existing courses and students were not given the same level of attention, and the infrastructure was not maintained well. The only new courses run were some short courses put on by private entities such as banks, primarily for their own staff. As a result, interest in NODES waned and student numbers started to decline. The MOHE leadership formed the view that NODES would take too long to become financially sustainable. At the end of the 3-year period, therefore, it was agreed that the physical network and the 35 remaining staff would be handed over to the OUSL.14 The handover took place in 2014. From then on, the vision of a single distance education network and platform offering a wide range of high-quality courses to students across the island was at an end.

71. On the other hand, the considerable strengthening of the OUSL and upgrading of the courses and staff that took place under the DEMP is likely sustainable. OUSL enrolments have been growing steadily since 2003 from the base of 23,500, finally surpassing the DEMP target of 40,000 students in 2014 (see Table 4). As Appendix 7 shows, the OUSL has received a steady increase in government funding since 2003, and receives almost the same amount from student fees. It fulfills a valuable function in Sri Lankan higher education by providing opportunities for working students and those not admitted to conventional universities. It has more flexibility than other universities in responding to student needs. Students lacking A-Levels are accepted for some courses, with foundation courses available to bridge the gaps. OUSL courses range from certificates to doctorates, with over 50% of enrolments at degree level, followed by skills training certificates, accounting for about 20% of enrolments (Appendix 8). In some important aspects, then, the OUSL embodies many of the advantages envisaged for NODES, albeit on a much smaller scale. With its steady sources of funding, there is no reason why the OUSL could not be expanded in future.

14 According to the MOHE, this decision was in accordance with the advice given by ADB during the PCR mission. Performance Assessment 21

Table 3. Open University of Sri Lanka Enrolments, 2003-2014

2003 2011 2013 2014 Faculty Total M F Total M F Total M F Total Education 3,660 1,654 6,259 7,913 1,887 6,781 8,668 1,908 6,991 8,899 Engineering 5,910 4,321 1,711 6,032 4,752 1,922 6,674 3,763 1,830 5,593 Humanities and 9,746 6,403 6,576 12,979 6,807 8,100 14,907 6,856 8,601 15,457 Social Sciences Natural Sciences 4,192 2,091 5,080 7,171 2,075 5,970 8,045 1,781 5,965 7,746 Common Courses 2,015 1,634 3,649 Total 23,508 14,469 19,626 34,095 15,522 22,772 38,294 16,323 25,021 41,344 F = female, M = male. Source: Open University of Sri Lanka. 2015. Statistical Handbook 2014. Colombo.

72. The OUSL core courses continue to be print-based, supplemented by non- compulsory face-to-face tutoring and occasional lectures broadcast from the main campus. Under the DEMP, 72 online courses were developed, of which 69 were offered as optional, supplementary courses. Today, the OUSL has about 200 courses online (25% of total courses), although none are part of the regular course offerings. The OUSL is slowly moving toward greater use of the new technologies. Under the DEMP, an online learning management system (LMS) was introduced, which is now housed on the NODES server. All courses are offered through this system. Some of the regular courses already have online content in addition to print, while continuous assessment and assignments are on the LMS. Tutoring is available through the LMS as well, operated from the main campus.

73. According to the current vice-chancellor of the OUSL, rural areas experience many connectivity problems, which limit the pace at which new technologies can be adopted. Students interviewed for the PPER in the Jaffna and Batticaloa OUSL study centers confirmed that it is difficult to access the LMS, and that it is often down. Despite these obstacles, the university policy is to have all courses online by 2018, blending online learning with the current practice of face-to-face tutoring, in response to student preferences. Two degree programs revamped under the World Bank higher education project have already moved in this direction, with 20% of these two courses now online. It is expected that the NODES server and the NACs will play a more central role as the OUSL embraces more online learning.

74. The OUSL is still in the process of working out how best to utilize the 35 staff and the infrastructure transferred from NODES. After minimal maintenance during the previous 3 years, many problems have been encountered in getting the network fully operational. It was noted during the PPER field visits that usage of the NACs in the study centers is infrequent for regular courses. At the moment, they are being used primarily to run computing courses for A-Level students. The skills of the NODES staff will be valuable to the OUSL in further developing online courses and maintaining the network. The DEMP put in place a core of trained staff in each faculty to design, develop, and deliver online courses, as well as bringing in new technologies which have already been institutionalized.

75. Even though NODES will not continue in the form envisaged, some off-shoots from the project will themselves be sustainable. First, two universities, apart from the OUSL, have continued to offer the online courses first developed under the DEMP. The universities of Kelaniya and Sri Jayewardenepura run external degree programs in arts, management, and commerce that are partly online. Administrative procedures such as registration are online, along with seminar notes, past examination papers, and 22 Distance Education Modernization Project

examination results. Some seminars are provided throughout the year but, for most learning, students from both universities rely on weekend tuition from private providers. The Institute for Agro-technology and Rural Sciences runs an online program for farmers that offers certificate, diploma, and recently degree-level qualifications. This program is fully online and encompasses chat groups, peer contact, and online tutoring as in the DEMP, and has a 75% completion rate. The University of Colombo also runs external programs in environmental science and informatics that have an online component and include the use of mobile phone technology. Rajarata University continues to run the blended or dual mode science degrees developed under the DEMP, with materials online and tutors available both online and face-to-face. The degree in business administration was initially online, but the university was forced to take it offline because many of the students did not have computers. Rajarata University also has its own open and distance learning center located off campus, as do other universities.

76. Each of these institutions has successfully continued with, and adapted, the innovative work of the DEMP to develop new models that are arguably better suited to the Sri Lankan context. Through their success in offering online and blended external degree programs, these universities are not only leading the way in reforming their own external degree programs, but are injecting new ideas into the system and demonstrating to other universities how they can respond to the government directive to improve the quality of all external degree programs. Some of the larger private institutions are also putting courses online now.

77. The DEMP also had an unanticipated impact on teaching in the internal degree programs. It was reported during interviews for the PPER that the extensive training given to university lecturers has opened their eyes to the value of integrating new technologies into their teaching. Many lecturers had previously been suspicious of anything other than the traditional lecture mode, but the DEMP helped them see the value of blending different techniques.

78. Other important activities set in motion under the DEMP are not only likely to be sustainable but will also have a fundamental impact on higher education. The national accreditation policy, plan, and performance standards for distance education were drafted under the DEMP, primarily to facilitate the involvement of private partners in distance education. All of the partner institutions were exposed to these standards and a ‘culture of quality’ through their regular participation in workshops for drawing up the accreditation and quality standards. All partner institutions were presented with a quality assurance toolkit for distance higher education. Recently, these tool kits have been extended into quality assurance manuals for public universities and higher education institutions, with the support of the World Bank. 15 Public universities previously had some quality assurance processes but no standards. The performance standards for distance education will be transformed into accreditation standards for external degrees and private institutions, and later for public institutions. This is a highly significant legacy of the DEMP, and it will shape the higher education sector for years to come. Quality assurance will not only foster improved quality in higher education, but will make it easier for private sector institutions to flourish.

15 University Grants Commission. 2015. Manual for Review of Undergraduate Study Programmes in Sri Lankan Universities and Higher Education Institutions. Colombo; University Grants Commission. 2015. Manual for Institutional Review of Sri Lankan Universities and Higher Education Institutions. Colombo; University Grants Commission. 2015. Manual for Review of External Degree Programmes. Colombo. Performance Assessment 23

79. To summarize, the NODES system established under the DEMP is no longer in operation. The infrastructure and some staff have been handed over to the OUSL for incorporation into its systems. How the OUSL will utilize the network, and whether it will one day make the network available to other providers, is not yet known. However, the introduction of ICT-enabled education is likely to be sustainable in various forms, and is behind significant changes now emerging in the introduction of online content for the large numbers of external students studying at conventional universities, and the moves by the OUSL to increase online content.

CHAPTER 4 Other Assessments

A. Impact

80. The project’s expected impact and long-term development objective was increased socioeconomic growth to raise human development levels. The growth indicators were: increased private sector employment (from 25% to 30% of total employment by 2009); decreased educated unemployment (from 15% to 10% by 2012); and increased economic growth above 6% annually. By project completion, private sector employment had reached 41% of total employment, and the 2010 unemployment rate of A-Level graduates was 12% (8% for males and 16% for females). In 2015, private sector employment remained at 41% of total employment, and the unemployment rate for A-Level graduates and above was 8% (3.8% for males and 12.3% for females). While these measures exceed the project impact targets, they are primarily the result of Sri Lanka’s general economic development.

81. A more direct indicator of the potential of the OUSL and distance education to contribute to human resource development is the share of distance education graduates in total graduate output. Data on graduate output from the OUSL (Appendix 8) shows that it made a significant contribution to the supply of graduates in 2014, providing 74% of law graduates, 25% of paramedical graduates, 14% of science graduates, and 10% of engineering graduates. Although it produced just 1,041 graduates (3.7% of total university graduates), the OUSL contribution is significant for socioeconomic development because it focuses on several fields that are in high demand in the labor market.

82. If, as now seems likely, universities will move their external students onto blended programs that combine online learning with a limited degree of face-to-face tutoring, then the contribution of distance education to total graduate output will be sizeable. The PCR argued that the impact of the DEMP would be felt mainly in its “strong demonstration value”; current developments in the way universities are now introducing online education into their external degree programs, and the introduction of online programs by private educational institutions, bear this out.

83. The capacity of the OUSL could be expanded further to increase its impact on human resource development, particularly in the regions. The PPER mission observed that the study centers visited were underutilized. The centers opened only on weekends for regular courses, with some short courses running on weekdays for part of the year. Some of the center directors expressed a desire to increase their enrollments, noting that their resources could be used better. One suggestion was to hold evening classes. Currently, however, the number of students and other key matters are determined in Colombo. Giving more autonomy to selected center directors would help to inject more dynamism, innovation and flexibility into the system.

Other Assessments 25

1. Institutional Impact

84. Clearly, the project did not succeed in transitioning NODES into a strong and self-standing institution. However, as shown in the previous chapter several other products of the DEMP have had a significant institutional impact on the tertiary education system as a whole, including the development of the accreditation policy and performance standards for distance education. These have become the basis for accreditation of external degrees and will ultimately form the basis for accreditation of conventional universities. The World Bank has adopted these standards to review university programs under its higher education project. As it is widely accepted that these standards will eventually become the accreditation standards for all higher education, the universities have already begun to use these standards to improve their quality.

85. The project had a more direct impact on the OUSL. Apart from the construction and renovation of facilities on the main campus and in the regions, which upgraded the quality and reach of the OUSL system, an extensive program of staff development gave many staff higher academic qualifications (doctoral and master’s degrees) from overseas and in-country. Many staff also undertook short-term overseas training in open and distance learning, and technical officers received technical training in various disciplines. Improvements in course offerings were achieved through the upgrading by OUSL staff of more than 170 courses and 14 entire study programs, as well as the development of 2 fully online courses and 69 short optional courses online. All of this yielded not only higher quality programs but also a significant build-up of staff skills in instructional design and development. The marketability of the OUSL programs was improved through the institutionalization of social marketing, a specific budget allocation, and the production of attractive brochures and pamphlets promoting the OUSL. Finally, the OUSL management information system (OUMIS) was developed and implemented, requiring significant upgrades of the ICT infrastructure. OUMIS covers human resource management, document management, registration and financial management, and student support. OUMIS continues to function successfully, as evidenced at least in part by the detailed and timely statistics the OUSL produces each year on its operations. With the support of the project, the OUSL has become a stronger institution, which is reflected in the steady enrolments and firm budgetary commitments from the government. It is a pity, however, that the OUSL has not taken greater advantage of its strengths and opportunities to continue growing its enrollments, and done more to overcome the bottlenecks in access to postsecondary education in rural areas in particular.

86. Finally, if the universities move further toward putting their external degrees at least partly online, this will fundamentally change the way universities run their external programs, leading to greater integration between the internal and external programs. The cost recovery model progressed by the OUSL under the DEMP may also be introduced. More widespread use of online distance education will also help promote a more learner-based rather than teacher-based approach to teaching in tertiary institutions, as the student has greater control over the learning process.

2. Socioeconomic Impact

87. In addition to expanding overall capacity, the establishment of the national network had a strong equity focus, aiming to expand opportunities for disadvantaged sections of the population. The only impact indicator linked directly to the project outcome is that poor regions would be linked to Colombo through public-private initiatives in education. After establishment of three new study centers and renovation 26 Distance Education Modernization Project

of 16 existing centers with strengthened staffing, 19 districts were connected to Colombo through 25 access centers. Five of the centers included estate or plantation areas, the poorest areas in Sri Lanka.

88. However, during 2003-2009, enrolment shares among the central and regional campuses remained largely the same. Enrolment in the regional campuses was intended to reach 60% of total enrolment by the end of the project, but enrolments in the central campus increased more rapidly than in the regions. This meant that the regional share of enrolments dropped from 46% in 2003 to 41% by the end of the project. In 2014, the regional share had moved back up to 47%, which is about where it started in 2003. Of course much larger numbers of rural students are being served now, some of whom are in very poor areas where they had no access previously, but much demand in regional areas is still unmet. Private colleges (often affiliated with Indian universities) are moving into some regional centers to meet this demand. Many of these colleges are of low quality but their degrees are recognized.

89. The underutilization of study centers was noted earlier. One reason for the reluctance to expand OUSL enrolments outside Colombo is the reported difficulty of finding lecturers locally. Central OUSL staff may also be unwilling to relinquish these teaching opportunities. At present, many of the tutoring sessions are led by central OUSL staff visiting from Colombo. Also, degrees such as science and engineering requiring specialized teachers and facilities are taught in the regional centers for the first year only, after which students continue their studies in Colombo.

90. A further barrier to increased regional enrolments is the generally lower level of English among students in rural areas. Fees are also a deterrent in very poor areas such as tea plantations. In one of the new centers built by the DEMP in a plantation area, the staff reported that enrollments could double if there were no fees. The data showing the number of applications and subsequent registrations after payment of fees supported this. In estate or other high-poverty areas, consideration could be given to a ‘no fees’ policy or a scholarship program for plantation workers and their families, to help these people lift themselves out of poverty.

91. Reducing gender inequity was not an explicit focus of the project. At the time of project formulation (and now) girls and women were well-represented at all levels of the education system. However, female university graduates were twice as likely as men to be unemployed after graduation thanks to their high enrolments in the arts and humanities. Women once comprised a high proportion of new school teaching staff, but the steady contraction of the public sector, and in particular the primary education teaching force, has directly affected employment opportunities for women. The project sought to address this inequity by providing women (and men) with education and training opportunities in areas more likely to lead to employment such as ICT, computer science, engineering technology, business management, accountancy, and agricultural technology. In addition, the project sought to provide women in particular with stronger career guidance and job search skills.

92. Unfortunately, the project was not successful in achieving gender equity in its online programs. In contrast with the OUSL and conventional university programs, women were in the minority among online students, making up just 34% of students enrolled. During the project period, female enrolments at the OUSL made up 48-54% of total enrolments, and in conventional universities were 53-57% of total enrolments. The most likely explanation for the lower participation of women in the online programs was the nature of the programs themselves, which were largely ICT and science focused. Perhaps this trend could have been reversed if the social marketing Other Assessments 27

and career guidance programs had been implemented successfully. If the online programs had continued, it is likely that this situation would have changed slowly in line with the increasing enrolments of girls seen now in school science. Distance education nevertheless continues to provide women in rural areas in particular with valuable opportunities to upgrade their qualifications without leaving their homes or their jobs.

93. In light of the potential of the OUSL and online distance education generally to contribute to the development of skilled human resources and socioeconomic development, the PPER rates the impact of the project as satisfactory.

B. ADB and Executing Agency Performance

94. The PPER rates the performance of ADB less than satisfactory. The PCR rated the performance of both ADB and the executing agency as satisfactory, but the PVR downgraded both ratings to less than satisfactory. Although the project suffered serious delays, the input from loan review missions was limited. The first loan review mission did not take place until 9 months after inception, with another 9-month gap before the midterm review. Only two special project administration missions were launched over the 6 years of project implementation, one for just 1 day and another for 2 days. The midterm review took place very late as ADB wanted the network completed first, but it should have been done earlier and included a more fundamental investigation of all the issues facing the project. More specialized technical support could also have been sought externally or from within ADB, such as from the Office of Information and Systems Technology, to address the many technical issues involved in constructing the network. Informants interviewed during the PPER mission commented that ADB officers did not fully understand the project and were unable to give useful guidance on many of the complex technical issues. The project generally suffered from a lack of leadership in technical areas. The project director was also a non-technical person. Finally, delays occurred in providing timely approval for procurement of goods and in the complex process of engaging consultants under the quality- and cost-based selection method, which required multiple approvals.

95. The PPER also rates the performance of the executing agency as less than satisfactory. The project began with a high level of government support, reflected in the prompt meeting of loan effectiveness requirements. However, later failures to gain approval of key policy documents indicate that the level of government commitment waned. While the project steering committee managed to function effectively despite the many changes of leadership and management, these changes did have a destabilizing effect, especially in light of the many challenges faced in implementing the project and coordinating across stakeholders. The government was able to establish and institutionalize the online distance-learning platform, but was not able to agree on, or establish, a management structure to ensure the sustainability of NODES. It was also unable to convince more students that distance education was a worthwhile investment. The government’s decision to fund the running of the project for 3 years after project closing was commendable, but it then did little to expand and broaden the online course offerings to increase revenues and put NODES on a more sustainable footing. The government could also have intervened to actively encourage more public universities in particular to put their external degrees online.

CHAPTER 5 Issues, Lessons, and Follow-up Actions

A. Issues

96. The defining characteristic of the DEMP is that it was in many senses a visionary project; albeit one that ultimately failed to achieve its vision. Several key issues arise in relation to this nexus between innovation and failure to succeed.

97. The first general issue concerns innovation and risk. Innovative projects are inherently risky in that they almost always attempt something that has not been tried before. Such projects therefore have a higher degree of risk than other less innovative projects, and it can be harder to identify all of the risks for the same reason. It is rarely possible to pursue innovation without any element of risk. Furthermore, it is never possible to ensure that risk mitigation measures will be effective. All of these factors contribute to a higher than usual likelihood that an innovative project will not succeed. At the same time, innovation is actively encouraged. How much risk is acceptable? If innovative projects are to be considered for lending, both parties should explicitly acknowledge the risk of failure.

98. The second issue concerns time and resource allocation. Projects that are new and innovative require more time and resources, both during the preparatory stages and during implementation. In the case of the DEMP, not enough time was allowed up front to determine whether the project was feasible by assessing (i) whether the demand for online education was sufficient and (ii) whether the expected public and private partners would commit to the project. Although the project had a 6-year implementation period instead of the usual 5 years, this was still not long enough to bring about the dramatic changes envisaged by the project designers. The resources devoted to project implementation by ADB were also clearly insufficient. It is to be expected that ADB staff will not have sufficient expertise in all areas, as in this case. Nevertheless, no other specialist assistance for the project officers was identified. When innovative projects are being prepared for approval, clear commitments should be agreed by both ADB and the borrower. Finally, it should be acknowledged and agreed by both parties that a highly innovative project will require investment over a span of at least 10 years to be successful.

99. A third issue concerns sustainability. When new approaches are being tried, it is often not possible to determine in advance the best way to ensure sustainability. In the case of the DEMP, initially no sustainability strategy was agreed because it was not known how the national distance education network would develop. This was understandable. But neither was a sustainability strategy developed over the course of project implementation, even though the need for this was clear. It is ironic that the desire to be flexible in support of an innovative approach allowed the issue to drift

Issues, Lessons and Follow-up Actions 29

unresolved for several years. This ultimately led to the failure of NODES. These competing pressures of flexibility and the need for commitment to a clear path require careful balancing and management.

B. Lessons

100. Ensure projects have a clear and achievable focus. Apart from its innovative nature, the DEMP was also very broad, with the project inputs spread very thinly. The project included the construction of a national broadband network; mobilization of non-traditional partners including private institutions; capacity building and institution strengthening within the OUSL; development of online courses, which is notoriously difficult and slow; and implementation of these courses with the expectation of rapid, high enrolments. The DEMP may have had more chance of success if it had focused only on the OUSL, or only on NODES, with the strengthening of the OUSL as a separate project.

101. Allocate significant resources for preparation of innovative projects. For innovative projects, significant resources are needed up front. This can include resources for assessing the feasibility of, and demand for, the project objectives and approaches for achieving them, as well as to lay the groundwork for ensuring their acceptability among different interest groups. Resistance to the innovations proposed cannot be overcome at the same time as project implementation is proceeding. The same applies to demand for the project outputs and outcomes. It is critical that a reasonable amount of consensus and commitment toward the project objectives is established before implementation begins.

102. Require expanded risk matrixes for all innovative projects. Considerable time, effort, and emphasis must be placed on the development and monitoring of the risk matrix for innovative projects. This applies particularly to risk identification and potential risk mitigation strategies. A range of alternative strategies should be developed in detail for every identified risk. The risk matrix and mitigation strategies for innovative projects should be discussed extensively by ADB and the government to ensure a common understanding and commitment. The risk matrix should be a major focus of ADB-government discussions in project review missions during implementation, and should be updated at least annually through a formal process such as a memorandum of understanding. DEMP implementation would have benefited significantly from a sharper focus on emerging risks, development of appropriate strategies to address these risks as they emerged, and a strong joint commitment to implementation of these strategies.

103. Ensure adequate ADB support for project implementation. By definition, the knowledge, skills, and experience needed to guide and embed innovative change are likely to be scarce in both the borrowing countries and in ADB. It is evident that additional time and resources are required to support implementation, which did not happen in the case of the DEMP. This should include longer, more intensive, and more frequent review missions, and supplementation of missing technical skill areas by bringing in additional staff from across ADB or engaging external support for review missions in particular. Technical assistance to strengthen the expertise available under the loan and ensure ongoing support to implementation may also be necessary. If the sector department is not committed to providing additional resources and support to innovative projects, then these projects should not proceed. 30 Distance Education Modernization Project

104. Ensure that the investment will be sustained. Fundamental change to education systems is complex and does not happen quickly. When an entirely new system is being created, as it was under the DEMP, it is critical that a commitment for sustained investment is made. Neither the government nor ADB had any interest in a follow-on project. Even a 1-year extension to allow enrolments to continue their upward trend at the end of the project was viewed unfavorably. Innovation and reform require more long-term commitment to be successful.

105. A sound strategy for sustainability must be agreed. The experience of the DEMP provides a good lesson on the importance of developing and operationalizing sound sustainability strategies. The project paved the way for the establishment of an innovative national online learning system, but the lack of a clear long-term plan was one reason why the concept eventually failed. If the best long-term strategy for sustainability is difficult to specify at the outset, specific support and additional time should be allowed in the project design to prepare and operationalize an agreed sustainability plan.

106. Ensure that the project performance indicators are appropriate for the time frame. In most education projects, and the DEMP was no exception, the focus of project implementation is on the development of facilities, teacher training, and programs that are only completed near the end of the project. The final achievement measures, however, are generally framed in terms of the number of students intended to benefit from the program. Again, the DEMP was no exception. It takes time, however, for institutions to train staff and develop online courses—especially technical courses, to launch them, develop good online or face-to-face tutorial support, and then to attract reasonable enrolments. It generally takes at least one cycle of a program to convince students that new programs are worthwhile. In the case of the DEMP, additional time should have been built into the design, or it should have been a multi- phase project. This would have allowed not only development of the network and associated courseware, but also time to allow the network to establish itself, experiment with different approaches, and develop into a mature system with good potential for sustainability.

107. Innovation needs champions. Effective champions are critical for the success of any new initiative. This is even more true for innovative projects, where strong commitment to the project on the part of both the government and ADB are critical to success. The failure to locate the DEMP within a government agency with strong ownership of the project was one of the factors that led ultimately to its failure. The sustainability of initiatives beyond the lifetime of the project rests heavily on the commitment and support provided by the relevant ministry, in the form of formal institutionalization of systems and processes, further training, and regular follow up by relevant staff. Similarly, the failure of ADB to adequately invest time and resources in project implementation, and the failure to remain involved in ensuring network sustainability in the years after the project ended, when its fate remained in limbo, also contributed to the project’s limited success.

108. Future potential of distance education. The DEMP failed to expand tertiary education opportunities and increase the employability of graduates to the extent intended. The question of how to achieve both of these objectives remains. A central issue in this regard is how to improve external degrees. One of the lessons of the DEMP is that Sri Lankan students may not yet be ready for fully online courses, at least for the longer degree courses. To raise the quality and range of offerings to external students, a blended learning approach would be most appropriate, combining online learning Issues, Lessons and Follow-up Actions 31

with a limited amount of face-to-face tutoring. This would allow the introduction of science and engineering as external degrees, currently not offered. Some universities have established their own open and distance learning centers, and these could be considered for future ADB assistance, along with support for further capacity expansion at the OUSL.

C. Follow-up Actions

109. There are no actions for follow up.

Appendixes

APPENDIX 1: DISTANCE EDUCATION MODERNIZATION PROJECT – PROJECT FRAMEWORK

Performance Indicators Monitoring Assumptions and Design Summary Mechanisms Risks I. Goal Increase socioeconomic Increased private sector Household surveys growth to raise human employment (small and Government's or development levels medium-sized enterprises) from funding agencies' 25% to 30% of total poverty assessments employment by 2009 Public–private business Decreased unemployment statistics among the educated from 15% to 10% by 2012 Increased economic growth above 6% annually Poor regions linked to better off Colombo through public– private initiatives in distance education II. Purpose OUSL enrollment doubling Project reports The government is from 20,000 to 40,000 by 2012 committed to Develop distance PPMS reforms linking education to expand Core policies on academic Surveys education to relevant postsecondary school accreditation, ICT for online employment. enrollment to develop a distance learning, career OUSL, DEPP, project modern, high-quality guidance, and private data records Educational human resource base participation in postsecondary institutes have the education ADB tracer studies motivation and

capacity to employ DEPP system established to MPDI and MTET distance education accommodate 25,000 external monitoring reports methodologies to degree students (2012) increase program Public–private partnership task quality and access. force operating among 150 MTET sets up a task postsecondary institutes by force to facilitate 2005 policies for participation of the MTET helps private schools increase enrollment by 35,000 private sector in through equipment purchase, degree programs. stipends for the poor, and access to distance learning by 2012 20,000 unemployed secondary school-leavers to be provided semiprofessional job training III. Project Deliverables/ DEPP national distance learning ADB project reports The OUSL is Outputs network operational in all determined to be a MPDI, MTET, and PMO OUSL campuses by 2006 key player in distance A. DEPP established to reports learning and will increase educational DEPP learning network OUSL reports share its distance opportunities available to 150 public and education ICT private educational institutions

34 Project Framework

Performance Indicators Monitoring Assumptions and Design Summary Mechanisms Risks on a cost basis by 2006 network with other postsecondary DEPP management; business schools. plans; marketing strategy; and curriculum web writing, External degree training, and multimedia students will operational by 2006 recognize the quality and convenience of DEPP engineering and technical DEPP distance services operate 150 learning. multimedia center hookups by 2005 DEPP online admission system, career guidance and labor market information system, and accreditation of transcript evaluation exist by 2006 B. Consortium of public– MPDI and MTET policy in place MPDI, MTET, and PMO The DEPP will private partnerships to support public–private reports provide high-quality established to improve partnerships in postsecondary programs that ADB studies standards education institutes, by 2006 attract clients to purchase services at Career guidance program a reasonable cost. linked to academic prerequisites, and labor market demand for males and females by 2006 Foreign–domestic link programs in 150 schools to increase technology transfer by 2006 Policy for accreditation tied to transcript transfer across postsecondary institutions and academic programs by 2005 Stipends for 4,500 (annually) needy OUSL students, needy DEPP students, and unemployed A-Level graduates seeking short-term skill development Public–private fund to 150 schools for equipment and stipends operational by 2004 National ICT policy for distance learning completed by 2006 C. OUSL (central and Effective MIS; cost-accounting Project reports The government can regional centers) upgraded system using modern develop and PPMS management practices for implement policy on outreach to all campuses; Government policy on public–private training and programs in public–private partnerships in budgeting; admission online; partnerships postsecondary and performance-based education. evaluation, with results-based OUSL progress reports Appendix 1 35

Performance Indicators Monitoring Assumptions and Design Summary Mechanisms Risks budgeting in place by 2006 MPDI, MTET, and PMO The OUSL reports administration is Student-centered admissions willing to modernize system for males and females administrative by 2006 procedures to Establishment of modern improve efficiency. facilities and equipment in The OUSL regional educational learning and study center technologies and in natural staff can be and engineering sciences to increased using train master trainers and contract staff to provide advanced labs for supplement existing students in core science professional and programs, functioning by 2006 technical staff. Upgrade of OUSL central university press, library, and multimedia facilities for use in preparing and upgrading curriculum to multimedia web- based learning, by 2006 Development of administrative, academic, and technical staff in key areas by 2007 Proposal-based upgrade of facilities of one new and four regional centers, and staff development by 2006 Proposal-based upgrade of 21 of 17 existing and 5 new OUSL study centers by 2009 Regional outreach admissions and accreditation by 2008 IV. Project Activities

A. Quality and efficiency of content and service delivery UGC program goals The MPDI and MTET are committed to Establish DEPP to develop MPDI, MTET, and PMO to Project reports and working with foreign online quality programs on outsource development of monitoring postsecondary a cost-recovery basis business plan and operations PMO monitoring of institutes to transfer (mid-2003 to mid-2007); turn contractors technology. over to OUSL or private firm to be decided in 2008 PMO monitoring of Public and private contractor business postsecondary Establish a business and MPDI, MTET, and PMO to plan and evaluation of institutions marketing plan, supervise contractor to recurrent costs and cooperate to curriculum, and training organize and operate DEPP receipts improve the capacity unit to produce production and distribution of postsecondary multimedia materials for services (mid-2003 to mid- PMO monitoring of education. online access by students 2008) contract for hardware 36 Project Framework

Performance Indicators Monitoring Assumptions and Design Summary Mechanisms Risks Establish an ICT system MPDI, MTET, and PMO to installation, operation Outsourcing of the network linking all OUSL contract hardware and maintenance, and DEPP is cost-effective centers with 150 configuration, installation, and servicing costs before project postsecondary institutes operation (start end-2003; end completion. UGC decision on 2007) performance-based systems ILO development and installation of labor market information system and job listing for online access

B. Public–private partnership models to improve efficiency Student stipend program Socially targeted program PMO data The MPDI and MTET for poor unemployed managed by PMO (mid-2003 to task force on public– Special reports, advisory secondary school leavers mid-2008) private partnership committee reports and those using distance will develop learning system MPDI and MTET guidelines on evaluations resource sharing.

ADB evaluation The UGC will School development fund Private sector matching grants implement the program to support to support expansion and ILO evaluation accreditation board. postsecondary institutes stipends for the needy, managed by PMO (mid-2003 to The ILO will mid-2008) implement the job website.

The MPDI and MTET Public awareness for career Public awareness program to will reform tertiary guidance, job information promote career guidance and education. system, and academic job information online counseling PMO through consultants to design and execute DEPP (2003–2005)

Foreign link programs to Proposal-based awards though transfer technology to PMO (2003–2008) local institutions

National accreditation National accreditation policy program to set standards and plan developed by PMO and allow flexible transfer through UGC (2003–2006) of credits

Results-based performance Surveys administered by PMO program linked to to establish guidelines and budgeting and recovering program for OUSL and for recurrent costs public and private postsecondary institutes under the project (2003–2006) Appendix 1 37

Performance Indicators Monitoring Assumptions and Design Summary Mechanisms Risks C. OUSL programs to improve access to public goods Upgrading of facilities, OUSL and PMO (2003-2006) Project OUSL faculties and library, multimedia, documents/reports regional centers are university press, committed to quality

courseware, and faculty improvements and have budgetary discretion to hire contract staff. Staff development for OUSL and PMO (2003-2006) Evaluation of private The OUSL will foreign and local sector participation finance expansion fellowships, curriculum through course fees.

materials, research, and training Performance-based PMO and OUSL (2003-2006) Number of foreign administration, financing, universities linked to budgeting, auditing, and local universities student services V. Project Inputs Total $60 million Project monitoring Counterpart funds $45 million loan from ADB data, progress reports, are available.

$10 from the government project accounts Procurement and $5 from beneficiaries contractor Cost Category arrangements are Research studies: $0.35 million well managed. Civil works: $6.50 million Equipment/furniture: $16.70 million Instructional materials: $1.50 million Staff development: $2.40 million Consulting services: $5.20 million Stipends: $9.00 million Matching grants: $8.00 million Recurrent costs: $3.40 million Project implementation: $1.00 million Others: $6.00 million ADB = Asian Development Bank, DEPP = Distance Education Partnership Program, ICT = information and communications technology, ILO = International Labor Organization, MIS = management information system, MPDI = Ministry of Policy Development and Implementation, MTET = Ministry of Tertiary Education and Training, OUSL = Open University of Sri Lanka, PMO = project management office, PPMS = project performance and monitoring system, UGC = University Grants Commission. Source: Asian Development Bank.

APPENDIX 2: PEOPLE MET DURING THE PERFORMANCE EVALUATION MISSION

District Name Gender Designation Institute Jaffna S Uthayakumar Male Provincial Director Department of Education, Northern Province Jaffna M Rathnakrishnam Male Additional Provincial Department of Education, Northern Director Province Jaffna P Selvin Ireneuss Female Additional Provincial Department of Education, Northern Director Province Jaffna K Selvan Male Assistant Provincial Department of Education, Northern Director Province Jaffna L Manimarphan Male Deputy Director - Department of Education, Northern Education Province Jaffna KA Sivanarulrujai Male Assistant Provincial Department of Education, Northern Director Province Jaffna K Bratley Male Deputy Director - Department of Education, Northern Education Province Jaffna C Kamalarajan Male Assistant Director - Department of Education, Northern Education Province Jaffna AC Valantine Female Assistant Director – Department of Education, Northern Mathematics Province Jaffna S Hapukotuwa Female Assistant Director – Department of Education, Northern Education Province Jaffna K Kanthavel Male Assistant Director OUSL - Jaffna Regional Center Jaffna N Balamurali Male Lecturer OUSL - Jaffna Regional Center Jaffna N Udayakumar Male Lecturer OUSL - Jaffna Regional Center Jaffna 15 students 10 male, 5 Student OUSL - Jaffna Regional Center female Jaffna M Selvasthan Male Principal School - Skandavarodaya College, Jaffna Jaffna A Sanjeevan Male Teacher School - Skandavarodaya College, Jaffna Jaffna V Umasuthan Male Teacher School - Skandavarodaya College, Jaffna Jaffna S Sivakumar Female Principal School - Victoria College, Jaffna Jaffna I Thavanayagam Female Deputy Principal School - Victoria College, Jaffna Jaffna P Nirmalanarthan Female Deputy Principal School - Victoria College, Jaffna Jaffna J Bastian Male Teacher - ICT/English School - Victoria College, Jaffna Jaffna S Easwaran Male Teacher School - Victoria College, Jaffna Jaffna L Velnamby Male Dean, Faculty of University of Jaffna Business Management Batticaloa AD Kamalanathan Male Senior Assistant OUSL - Batticaloa Regional Center Director Batticaloa K Alloysius Female Coordinator - BSc OUSL - Batticaloa Regional Center Nursing Batticaloa E Senthirakumar Male Coordinator - Social OUSL - Batticaloa Regional Center Science Batticaloa K Male Visiting lecturer OUSL - Batticaloa Regional Center Shanmuganathan Batticaloa T Jayasingam Male Vice Chancellor Eastern University Batticaloa U Kumaraswami Female Adviser to Vice Eastern University Chancellor Batticaloa P Kovintharajah Male Deputy Director – Provincial Education, Eastern Province Education Batticaloa S Kangeswaran Female Deputy Director – Provincial Education, Eastern Province Education

Appendix 2 39

District Name Gender Designation Institute Batticaloa C Nadarajah Male Deputy Director – Provincial Education, Eastern Province Education Batticaloa R Kirushnathasan Male Engineer Provincial Education, Eastern Province Batticaloa K Kalanithy Male Provincial Project Provincial Education, Eastern Province Coordinator Batticaloa K Baskaran Male Zonal Education Zonal Education Office, Batticaloa Director Batticaloa K Arudpragasan Male Principal School - Hindu College, Batticaloa Batticaloa SD Muralidaran Male Deputy Principal School - Hindu College, Batticaloa Batticaloa S Punniamoorthi Male Sectional Head School - Hindu College, Batticaloa Batticaloa K Udayakumar Male In charge - Technology School - Hindu College, Batticaloa Lab Batticaloa 20 students Male Students - Technology School - Hindu College, Batticaloa Faculty Batticaloa SAM Salim Male District Manager - National Apprentice and Industrial Maulana Batticaloa Training Authority Polonnaruwa V Senevirathna Male Assistant Director OUSL Study Center - Polonnaruwa Polonnaruwa 6 students Female Students - OUSL OUSL Study Center - Polonnaruwa Polonnaruwa DA Jayarathna Male Assistant Director VTA - Polonnaruwa Polonnaruwa S Indika Male ICT Instructor VTA Training Center - Minneriya Polonnaruwa NDS Sandaruwan Male Welding Instructor VTA Training Center - Minneriya Polonnaruwa 20 students Students - vocational VTA Training Center - Minneriya training Anuradhapura S Madugalla Female Assistant Director OUSL - Anuradhapura Regional Center Anuradhapura PDP Rohana Male Deputy Director of North Central Provincial Education Education Anuradhapura MM Hettiarachchi Female Assistant Director North Central Provincial Education (Development) Anuradhapura A Vithanapathirana Female Assistant Director North Central Provincial Education (Development) Anuradhapura YM Male Director - Distance University of Rajarata Wickramasinghe Learning Anuradhapura WP Wijewardana Male Coordinator University of Rajarata Anuradhapura DPHNJ Male Full time tutor - online University of Rajarata Chandrasena education Anuradhapura WG Wijerathna Male Deputy Director VTA - Anuradhapura Anuradhapura SRACM Female ICT Instructor VTA - Anuradhapura Samarathunga Anuradhapura HMSPH Bandara Male Cookery Instructor VTA - Anuradhapura Anuradhapura PD Pushparani Female MIS Officer VTA - Anuradhapura Colombo L Jayathilake Male Chairperson National Education Commission Colombo J Gunasekara Female Vice Chairperson National Education Commission Colombo DC Disanayake Male Secretary Ministry of Higher Education Colombo GMRD Aponsu Male Director – Planning Ministry of Higher Education Colombo M Hapuarachchi Male Additional Secretary Ministry of Higher Education Colombo SU Wijerathne Male Additional Secretary – Ministry of Education Planning Colombo K Karunasiri Male Project Director ESDP/EKSP, Ministry of Education Colombo U Indika Male M&E Specialist ESDP/EKSP, Ministry of Education Colombo RM Abeyrathna Male Additional Secretary Ministry of Skill Development and Vocational Training Colombo P Sivanathan Male Programme Reforms SSDP - MSDVT Specialist 40 List of Persons Met during PPER Mission

District Name Gender Designation Institute Colombo S Wijesinghe Female Assistant Project SSDP - MSDVT Manager Colombo KMVJ Priyanjith Male Assistant Project SSDP - MSDVT Manager Colombo BKS Suraweera Male Programme Reforms SSDP - MSDVT Specialist Colombo P Saputhanthri Male Project Manager SSDP - MSDVT Colombo HGM Kaldera Female HRD Specialist SSDP - MSDVT Colombo TA Piyasiri Male Vice Chancellor University of Vocational Technology Colombo P Ariadurai Male Vice Chancellor OUSL Colombo GM Kamal Male Deputy Vice Chancellor OUSL Gunaherath Colombo V Disanayake Female Education Consultant World Bank, Sri Lanka Country Office Nuwara Eliya S Wadivoo Female Officer in Charge OUSL Study Center, Hatton BSc = Bachelor of Science, EKSP = Education for Knowledge Society Project, ESDP = Education Sector Development Program, HRD = human resource development, ICT = information and communications technology, M&E = monitoring and evaluation, MIS = management information system, MSDVT = Ministry of Skills Development and Vocational Training, OUSL = Open University of Sri Lanka, SSDP = skills sector development program, VTA = Vocational Training Authority. Source: Independent Evaluation, ADB.

APPENDIX 3: APPRAISAL AND ACTUAL PROJECT COSTS

Per Appraisal ($ million) Actual Cost ($) Total Actual ADB and Local Currency ADB Counterpart Funds Counterpart Funds ($) Foreign Govern- Bene- Total Foreign Local Govern- Bene- Exchange ADB Total Total Category ment ficiaries Exchange Currency ment ficiaries % 1. Base Cost A. Civil works 1.00 1.00 4.30 6.30 1,403,690 1,403,690 2,807,379 5,902,621 8,710,000 19%

B. Furniture, equipment, 12.20 3.00 0.15 15.35 15,824,919 2,605,092 18,430,011 18,430,011 39% and vehicles 1 Furniture 0.20 0.30 0.15 0.65 232,568 356,182 588,749 2 Equipment 11.86 2.70 14.56 15,452,361 2,248,911 17,701,272 3 Vehicles 0.14 0.00 0.14 139,990 0 139,990

C. Instruction materials 0.80 0.50 1.30 121,210 675,247 796,457 796,457 2%

D. Staff development 1.30 0.80 2.10 1,491,986 135,756 1,627,741 1,627,741 3% 1 Foreign 1.30 0.00 1.30 2 Domestic 0.00 0.80 0.80

E. Consulting services 3.90 1.20 5.10 4,867,782 547,598 5,415,380 5,415,380 12% 1 International 3.90 0.00 3.90 4,867,782 2 Domestic 0.00 1.20 1.20 547,598

F. Research studies 0.10 0.50 0.60 648 345,471 346,119 346,119 1% G. Stipends, scholarships 0.00 9.00 9.00 0 3,370,215 3,370,215 3,370,215 7% H. Matching grants 1.00 0.50 2.00 2.50 6.00 326,200 105,192 431,392 403,608 75,000 910,000 2% I. Recurrent costs 0.00 0.00 0.40 2.50 2.90 0 0 0% J. Project management 0.00 0.85 0.15 1.00 2,242,225 2,242,225 267,775 2,510,000 5% K. Taxes and duties 0.00 0.00 3.00 3.00 4,100,000 4,100,000 9%

Total base cost 20.30 17.35 10.00 5.00 52.65 24,036,434 11,430,486 35,466,920 10,674,004 75,000 46,215,923 98%

2. Contingencies A. Physical contingency 1.00 1.40 2.40 B. Price escalation 1.00 2.25 3.25

3. Interest Charge 1.70 0.00 1.70 772,169 772,169 772,169 2%

Percentage 40% 35% 25% 100% 53% 24% 23% 100% ADB = Asian Development Bank. Source: Independent Evaluation, ADB.

APPENDIX 4: DESIGN AND MONITORING FRAMEWORK SHOWING ACHIEVEMENTS, 2011 AND 2016

Design Achievements by 2011 Achievements by 2016 Summary Performance Targets/Indicators (from PCR) I. Impact Increased private sector The share of private sector The share of private sector employment (small and medium- employment decreased from 46.4% employment continued to drop, Increase sized enterprises) from 25% to in 2004 to 41.1% in 2008 and 42.1% declining to 40% in 2014. socioeconomic 30% of total employment by 2009 in 2009. Employment has risen in the growth to raise public sector and for the self- human employed, although marginally development (UNDP 2014). levels Decreased unemployment among The unemployment rate was 9.9% for The unemployment rate for A- the educated from 15% to 10% by A-Level achievers and 7.4% for O- Level achievers and above was 2012 Level achievers in 2008. The 8.1% in 2015 (3.8% for males unemployment rate for A-Level and 12.3% for females).

achievers and above was 11.6% in 2010 (7.9% for male and 15.8% for female). Increased economic growth above Economic growth for 2008 was 6%, Economic growth was 4.5% in 6% annually which declined to 3.5% in 2009. 2014 (revised downwards from 7.4% due to a national accounts

rebasing). Average growth for 2010-2014 was 6.7%. Poor regions linked to better off Colombo was linked to outstations Unchanged. Colombo was Colombo through public–private through NODES, covering 19 districts linked to rural areas through initiatives in distance education (26 access centers, including 4 in both the OUSL network and the Colombo). network established through NODES, although both are now housed at the OUSL. II. Outcome OUSL enrolment doubles from OUSL enrolment was up by 12.3% OUSL enrolment reached 20,000 to 40,000 by 2012 (from 23,508 in 2003 to 26,395 in 41,344 in 2014, a 106% Develop distance 2009 and 30,355 in 2010). increase since 2003. education to expand Core policies on academic (a) The National Assessment and The National Assessment and postsecondary accreditation, ICT for online Accreditation Policy was approved by Accreditation Policy has still not school enrolment distance learning, career guidance, the PSC in 2007, and the been approved by government to develop a and private participation in establishment of the Interim Board but guidelines and performance modern, high- postsecondary education for Quality Assurance and Assessment standards for quality assurance quality human for Distance Higher Education was have been released. resource base approved by the PSC in 2009.

(b) ICT policy for online distance learning not yet approved. Career guidance is now in place (c) Career guidance policy outcomes were pursued through a World Bank- in secondary schools and TVET funded project. training institutions supported by the government, ADB, and the World Bank. DEPP system established to NODES was established. Only 1,469 NODES was closed in 2014. The accommodate 25,000 external external degree students were server, NODES access centers, degree students (2012) (reduced registered through NODES (499 from and 35 staff members were to 10,000 by 2009 at MTR) 4 national universities in 5 programs, transferred to the OUSL. and 970 from other partner institutions). Enrolments were low since online programs were offered only after 2007.

Appendix 4 43

Design Achievements by 2011 Achievements by 2016 Summary Performance Targets/Indicators (from PCR) Public–private partnership task 42 courses or programs were The courses are no longer force operating among 150 launched by 24 partner institutions offered through NODES, postsecondary institutes by 2005 by January 2010. although some of the partner (target reduced to 40 programs institutions, including the OUSL, offered by partner institutions at are still offering the online MTR) courses developed under NODES. MTET helps private schools 24 partner institutions (13 private NODES is no longer in increase enrolment by 35,000 schools, 2 professional associations, 3 operation. through equipment purchase, public institutions, 5 national

stipends for the poor, and access universities, and the OUSL) were to distance learning by 2012 supported and 1,469 students were (reduced to 10,000 for 2009 at registered for online diplomas and MTR) postgraduate courses. 20,000 unemployed secondary 11,878 unemployed secondary The programs were completed school-leavers to be provided school-leavers were trained (10,488 under the DEMP. semiprofessional job training demand-driven, 1,065 on-the-job, (reduced to 8,000 at MTR and age and 325 online training). Stipends limit increased from 21 to 30) were not offered to needy OUSL students. III. Outputs DEPP national distance learning 20 NACs in OUSL premises were The 20 NACs in OUSL premises A. DEPP network operational in all OUSL operational from 2007. continue to operate as part of established to campuses by 2006 the OUSL. increase educational DEPP learning network available to The learning network was made The network is no longer opportunities 150 public and private educational available to 24 partner institutions on available. institutions on a cost basis by 2006 a cost basis, which developed 42 (reduced to 40 at MTR) courses and/or programs. DEPP management; business plans; NODES was established as an MOHE NODES staff and infrastructure marketing strategy; and curriculum unit in 2010 (the corporate unit was were transferred to the OUSL in web writing, training, and yet to be established). The business 2014. multimedia operational by 2006 plan was prepared, the social marketing program launched, a fully DEPP engineering and technical staffed CDU at NODES was services operate operational in 2007, and 938 individuals were trained in course design, development, and delivery. 150 multimedia center hook-ups The NODES network, comprising the The NODES network still by 2005 NOC, 26 NACs (including 20 at OUSL connects to all the NACs, from premises), and the CDU, was the OUSL. operational. DEPP online admission system, The NODES MIS was completed but Unchanged. career guidance and labor market not utilized. The labor market analysis information system, and was completed. The provision of accreditation of transcript career guidance was pursued through evaluation exist by 2006 a World Bank-supported project. The accreditation service was yet to be available. B. Consortium of MPDI and MTET policy in place to The policy supporting public-private The government is supportive public–private support public–private partnerships was reflected in the and encouraging of public– partnerships partnerships in postsecondary provision of matching grants. NODES private partnerships in established to education institutes, by 2006 provided training and support to postsecondary education. improve partner institutions. standards

44 Updated Project Framework Showing Accomplishments

Design Achievements by 2011 Achievements by 2016 Summary Performance Targets/Indicators (from PCR) Career guidance program linked to Career guidance activities were Career guidance in schools and academic prerequisites, and labor pursued under a World Bank- TVET institutions is currently market demand for male and supported project. supported by ADB and the females by 2006 World Bank. Foreign–domestic link programs in 2 programs are being developed by The 2 OUSL programs and the 150 schools to increase technology the OUSL and the Informatics Informatics Institute of transfer by 2006 (reduced to 40 Institute of Technology. Technology online course are partner institutions and 10 still being offered successfully. foreign–domestic links) Policy for accreditation tied to The accreditation policy was Unchanged. However, manuals transcript transfer across approved in 2007 but the Interim for quality assurance in postsecondary institutions and Accreditation Board was not yet universities were released in academic programs by 2005 functional. 2015. They will initially be used to accredit external degree programs. Stipends for 4,500 (annually) 11,878 students were granted This was completed under the needy OUSL students, needy DEPP stipends and trained. No stipends DEMP. students, and unemployed A-Level were provided to needy OUSL and graduates seeking short-term skill DEPP students. development (reduced to 8,000 at MTR) Public–private fund to 150 schools Only 42 programs were offered by 24 No longer applicable. for equipment and stipends partner institutions. The planned operational by 2004 (reduced to program of stipends for students in 40 at MTR) partner institutions was not implemented. . National ICT policy for distance The national ICT policy for distance There are still no plans to learning completed by 2006 learning was submitted to the PSC in approve a separate ICT policy 2008, but had not yet been for distance education. approved. C. OUSL (central Effective MIS; cost-accounting Software for the OUSL MIS was The OUSL MIS is in operation. and regional system using modern developed, partly integrated, and Admissions occurs online. centers) upgraded management practices for utilized. outreach to all campuses; training and programs in budgeting; admission online; and performance-based evaluation, with results-based budgeting in place by 2006 Student-centered admissions The online student admission module The online student admission system for males and females by was tested and implemented in 2009. system is still in operation 2006 (https://payment.ou.ac.lk/). Establishment of modern facilities Modern facilities and labs were Unchanged. and equipment in educational established at the OUSL central learning technologies and in campus and was equipment natural and engineering sciences provided. to train master trainers and provide advanced labs for students in core science programs, functioning by 2006 Upgrade of OUSL central university The OUSL central library, multimedia Unchanged. press, library, and multimedia center, and press facilities were facilities for use in preparing and equipped to support online delivery. upgrading curriculum to multimedia web-based learning, Appendix 4 45

Design Achievements by 2011 Achievements by 2016 Summary Performance Targets/Indicators (from PCR) by 2006

Staff development of Long-term training was provided to Unchanged. administrative, academic, and 34 staff (14 completed, 2 technical staff in key areas by 2007 discontinued, 1 left the OUSL). Short- term foreign training was provided to 135 staff. Proposal-based upgrade of Long-term domestic training was Unchanged. facilities of one new and four provided to 31 staff (4 completed). regional centers, and staff development by 2006 Proposal-based upgrade of 21 of 16 regional centers were upgraded, 7 Unchanged. 17 existing and 5 new OUSL study new study centers completed, and 3 centers by 2009 study centers renovated. Regional outreach admissions and Regional registrations took place in Regional registration continues accreditation by 2008 OUSL centers through the OUSL to operate. virtual private network. IV. Project Establish DEPP to develop online NODES was established but the basis Unchanged. Activities quality programs on a cost- of cost recovery was not established. recovery basis A. Quality and efficiency of Establish a business and marketing The 5-year business plan was Unchanged. content and plan, curriculum, and training unit approved by the PSC in 2007, and a service delivery to produce multimedia materials successor organization was for online access by students established. The NODES CDU was

fully equipped with multimedia equipment, staff were hired and trained, CDU services were made available, 8 programs were outsourced to them by partner institutions, and extensive staff training was completed. Establish an ICT system network An ICT network was formed Unchanged. linking all OUSL centers with 150 connecting all NACs in a virtual postsecondary institutes private network. B. Public–private Student stipend program for poor The stipend program was completed Unchanged. partnership unemployed secondary school in 3 phases. models to leavers and those using distance improve efficiency learning system School development fund program Matching grants were disbursed to Unchanged. to support postsecondary 43 partner institutions. institutes Public awareness for career Career guidance was to be pursued Unchanged. guidance, job information system, under a World Bank-supported and academic counselling project. Foreign link programs to transfer Applications called for a foreign– Unchanged. technology to local institutions domestic link and two institutions developed link programs. National accreditation program to A draft accreditation policy had not Unchanged. set standards and allow flexible been implemented. transfer of credits Results-based performance Partner institutions offered programs Unchanged. program linked to budgeting and 46 Updated Project Framework Showing Accomplishments

Design Achievements by 2011 Achievements by 2016 Summary Performance Targets/Indicators (from PCR) recovering recurrent costs demanded by the market.

C. OUSL programs Upgrading of facilities, library, Upgrading of facilities was Unchanged. to improve access multimedia, university press, completed. to public goods courseware, and faculty Staff development for foreign and The staff development program was Unchanged. local fellowships, curriculum implemented. materials, research, and training Performance-based administration, The OUSL MIS, incorporating general Unchanged. financing, budgeting, auditing, administration, human resources, and student services financing, and student support services, was installed, implemented, and partially integrated. ADB = Asian Development Bank, CDU = content development unit, DEMP = Distance Education Modernization Project, DEPP = Distance Education Partnership Program, ICT = information and communications technology, MIS = management information system, MPDI = Ministry of Policy Development and Implementation, MTET = Ministry of Tertiary Education and Training, MTR = midterm review, NAC = network access center, NOC = Network Operation Centre, NODES = National Online Distance Education Service, OUSL = Open University of Sri Lanka, PCR = project completion report, PSC = project steering committee, TVET = technical education and vocational training, UNDP = United Nations Development Programme. Source: Independent Evaluation and South Asia Department, ADB.

APPENDIX 5: PARTNER INSTITUTIONS AND ONLINE DEGREE AND DIPLOMA PROGRAMS, 2011

Partner Institution Program Type of Institution Enrolment 1. Open University of Sri Lanka Post Graduate Certificate In Medicine Public University 36 Teacher Educator as an Education 44 Technologist 2. University of Rajarata BSc in Biological Science Public University 22 BSc in Physical Science 12

3. University of Jaffna Bachelor of Business Management (Tamil Public University 85 medium) 4. University of Moratuwa Bachelor of Information Technology Public University 270 Institute of Technology Diploma in Civil Engineering Technology 35 5. University of Peradeniya Bachelor in Business Administration Public University 110 6. University of Colombo MSc in Organic Chemistry Public University 18 MSc in Environmental Science 14 Certificate in Bioinformatics 9 e-Diploma in Business English 49 Institute of Agro-Technology Diploma in Agro-Technology 63 and Rural Sciences Centre for the Study of Human Diploma in Human Rights 13 Rights Postgraduate Institute of MSc in Medical Toxicology 35 Medicine Diploma in Family Medicine 17 Diploma in Disaster Management 29 7. Institute of Engineering Diploma in Civil Engineering Public Institute 24 Technology 8. National Institute of Health Diploma in Pharmacy Training Public Institute 11 Sciences Diploma in Exercise Therapy 11 9. Sri Lanka Institute of Diploma in Management for Public Institute 84 Development Administration Professionals 10. International Institute of Diploma in Hospital Management Private Training 12 Health Sciences Institution 11. Sri Lanka Institute of Postgraduate Diploma in Marketing Private Training 57 Marketing Certificate in Marketing Institution 0 12. Ladies College, Diploma in Teaching Children with Private Training 27 Department of Vocational Special Learning Difficulties Institution Studies 13. Open Arc School of Diploma in Software Development Private Training 156 Business and Technology Diploma in Computerized Drafting Institution 17 Diploma in Project Management 0 14. IDM Computer Studies Higher National Diploma in Teacher Private Training 4 Education Institution Higher National Diploma in E-business 3 15. Win-Stone School of Diploma in Food and Beverage Private Training 36 Culinary Art Diploma in Professional Cookery Institution 43 16. Damrivi Foundation Diploma in English Private Training 10 Diploma in Buddhist Studies Institution 7 17. Informatics Institute of Diploma in Computing and Business Private Training 43 Technology Studies Institution 18. Institute of Supplies and Diploma in Purchasing and Materials Private Training 6 Materials Management Management Institution

48 List of NODES Partners Institutions and Online Programs

Partner Institution Program Type of Institution Enrolment 19. Techland Institute of Diploma in Information and Private Training 6 Computer Science Communications Technology Institution

20. Indination Computer Diploma in Computer Studies Private Training 8 Institute Institution 21. Society of Certified Technician and Intermediate Private professional 2 Management Accountants of Professional I and II association 5 Sri Lanka 22. Sri Lanka Library Diploma in Library and Information Private professional 29 Association Science association 23. Institute of Quantity Certificate in Quantity Surveying Private professional 7 Surveyors (dropped from association project in 2009) Total 1,469 BSc = Bachelor of Science, ICT = information and communications technology, MSc = Master of Science. Source: Government of Sri Lanka.2010. Distance Education Modernization Project Completion Report. Colombo.

APPENDIX 6: RATING MATRIX FOR CORE EVALUATION CRITERIA

Ratings PCR PVR PPER Comments Relevance Highly Relevant Relevant The project responded to the government’s relevant desire to expand non-conventional tertiary education through an innovative strategy and design.

The project established a national delivery system for online distance education that catered to disadvantaged areas.

The national distance education system incorporated partnerships with public and private institutions to broaden course offerings and improve market relevance.

Government ownership was strong at the time of design but no strategy for sustainability was agreed.

The project faced many risks, not all were addressed properly.

The project remains very relevant to key issues in higher education. Less Less than Less than Effectiveness effective effective effective Less Less than Less than Efficiency efficient efficient efficient Less likely Less likely Less likely Sustainability sustainable sustainable sustainable Impact Not rated Moderate Moderate ADB Less than Less than ADB invested fewer resources than was performance Satisfactory satisfactory satisfactory necessary for a project of this complexity and innovativeness. The executing agency could have done Executing Less than Less than more to stimulate student demand for agency Satisfactory satisfactory satisfactory courses and encourage the public performance universities to put their courses online. Overall Less than Less than Less than Rating successful successful successful ADB = Asian Development Bank, PCR = project completion report, PPER = project performance evaluation report, PVR = project validation report. Source: Independent Evaluation, ADB.

APPENDIX 7: OPEN UNIVERSITY OF SRI LANKA ENROLLMENT, FINANCING, AND COST RECOVERY

Item 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 Total Enrolment 23,508 24,692 24,373 24,846 25,251 26,296 26,395 30,355 34,095 35,647 38,294 41,344 Colombo and Kandy 16,539 16,746 17,549 17,583 19,162 19,676 19,994 22,120 22,283 23,754 26,018 Other region/centers 8,153 7,627 7,297 7,668 7,134 6,719 10,361 11,975 13,364 14,537 15,326 Total Enrolment 23,508 24,692 24,373 24,846 25,251 26,296 26,395 30,355 34,095 35,647 38,294 41,344 Male 12,505 13,050 12,587 12,527 11,660 11,745 11,525 12,587 14,469 13,958 15,522 16,323 Female 11,003 11,642 11,786 12,319 13,591 14,551 14,870 17,768 19,626 21,689 22,772 25,021 Increase by year 2,398 1,184 (319) 473 405 1,045 99 3,960 3,740 1,552 2,647 3,050 Total OUSL expenditure 377,489 400,215 463,392 590,368 669,661 740,329 819,177 878,078 1,062,552 1,085,281 1,434,814 1,663,149 Recurrent 358,628 354,583 424,814 544,185 630,129 676,208 770,539 852,581 1,004,281 975,873 1,277,615 1,550,906 Capital 18,861 45,632 38,578 46,183 39,532 64,121 48,638 25,497 58,271 109,408 157,199 112,243 Sources of Financing – 354,514 339,549 402,109 544,200 655,477 692,761 741,621 921,136 1,024,521 1,039,679 1,544,929 2,014,610 Recurrent Government 211,562 225,440 286,000 394,200 449,321 479,800 505,000 525,000 628,386 663,100 844,715 965,000 Fees 101,153 101,183 105,169 134,202 188,617 182,970 205,934 362,446 355,807 320,243 627,222 958,130 Other 41,799 12,926 10,940 15,798 17,539 29,991 30,687 33,690 40,328 56,336 72,992 91,480 Amount paid to NODES 6,000 0 0 0 0 Unit recurrent cost per 15 14 17 22 25 26 29 28 30 27 33 37 student (SLRs) Government share in 56 56 62 67 67 65 62 60 59 61 58 58 total expenditure % () = negative number, NODES = National Online Distance Education Service, OUSL = Open University of Sri Lanka. Source: Government of Sri Lanka.2010. Distance Education Modernization Project Completion Report. Colombo; Open University of Sri Lanka. 2015. Statistical Handbook 2013- 2014. Colombo; Open University of Sri Lanka. 2012. OUSL Annual Report 2011. Colombo.

APPENDIX 8: OPEN UNIVERSITY OF SRI LANKA ENROLLMENTS AND CONTRIBUTION TO NATIONAL GRADUATE OUTPUT

Table A8.1: Open University of Sri Lanka Enrollments across Types of Programs, 2013 and 2014 2013 2014 Program of Study Total % Total % Master’s degree programs 940 2.45 1,008 2.44 Postgraduate diploma programs 5,467 14.28 5,529 13.37 Undergraduate programs 16,739 43.71 20,916 50.59 Diploma programs 3,718 9.71 1,662 4.02 Certificate programs 11,430 29.85 8,201 19.84 Other (short courses/workshops) 0.00 4,028 9.74 Total 38,294 100.00 41,344 100.00 OUSL = Open University of Sri Lanka. Source: Computed from OUSL. 2015. OUSL Statistical Handbook 2014. Colombo

Table A8.2: Contribution of Open University of Sri Lanka to National Graduate Output, 2014

Undergraduate Level Total Arts Law & QS Mgt & Mgt Dental Studies Science Science Science Science Medicine Medicine Education Computer Veterinary Commerce Agriculture Indigenous Indigenous Engineering Paramedical Paramedical Architecture

Number of undergraduate 2,082 173 2,607 4,236 4,706 122 2,925 4,065 20,916 students at OUSL Total undergraduates in conventional universities in Sri 25,049 336 18,264 1,477 6,131 416 309 3,884 6,373 1,436 3,816 1,830 11,884 1,515 82,720 Lanka Total undergraduates in Sri Lanka's 27,131 509 20,871 5,713 6,131 416 309 3,884 11,079 1,436 3,938 4,755 15,949 1,515 103,636 universities OUSL as % of total 7.7 34.0 12.5 74.2 0 0 0 0 42.5 0 3.1 61.5 25.5 0 20.2 undergraduates Graduate Level Number of graduates from OUSL 171 26 91 176 138 16 98 325 1,041 Total graduates in Sri Lanka 14,636 26 5,726 239 1,144 68 57 867 1,438 214 947 399 2,295 175 28,231 OUSL as % of total graduates 1.2 100.0 1.6 73.6 0.0 0.0 0.0 0.0 9.6 0.0 1.7 24.6 14.2 0.0 3.7 OUSL = Open University of Sri Lanka. Source: OUSL. 2015. OUSL Statistical Handbook 2014. Colombo; University Grants Commission. 2015. Sri Lanka University Statistics 2014. Colombo. APPENDIX 9: ENROLLMENTS AT OPEN UNIVERSITY OF SRI LANKA STUDY CENTERS, 1999−2014

Number of Students Distribution across Centers (%) Centre Province 1999 2011 2014 1999 2011 2014 Colombo Western 9,608 16,718 19,624 55.75 49.03 47.47 Kalutara Western 373 510 529 2.16 1.50 1.28 Gampaha Western 755 864 2.21 2.09 Negombo Western 60 14 6 0.35 0.04 0.01 Kandya Central 2,395 5,388 6,394 13.90 15.80 15.47 Nuwara Eliyaa Central 33 18 1 0.19 0.05 0.00 Trincomalee Eastern 145 18 119 0.84 0.05 0.29 Batticaloa Eastern 438 1,808 2,022 2.54 5.30 4.89 Ampara Eastern 208 92 326 1.21 0.27 0.79 Sammanthurai Eastern 48 73 93 0.28 0.21 0.22 Kegallea Sabaragamuwa 273 206 244 1.58 0.60 0.59 Ratnapuraa Sabaragamuwa 274 410 573 1.59 1.20 1.39 Badulla/Bandarawelaa Uva 228 299 555 1.32 0.88 1.34 Monaragala Uva 107 146 117 0.62 0.43 0.28 Hattona Central 123 316 0.36 0.76 Galle Southern 344 720 541 2.00 2.11 1.31 Ambalangoda Southern 183 419 399 1.06 1.23 0.97 Matara Southern 980 1,999 2,796 5.69 5.86 6.76 Ambalantota Southern 94 281 0.28 0.68 Jaffna Northern 151 1,661 1,920 0.88 4.87 4.64 Vavuniya Northern 105 405 420 0.61 1.19 1.02 Anuradhapura North Central 248 809 1,340 1.44 2.37 3.24 Polonnaruwa North Central 102 126 226 0.59 0.37 0.55 Kurunegala North Western 724 990 1,072 4.20 2.90 2.59 Kuliyapitiya/Chilaw North Western 159 42 25 0.92 0.12 0.06 Puttalam North Western 238 328 0.70 0.79 Matalea Central 33 14 4 0.19 0.04 0.01 Kilinochchi Northern 153 0.37 Mullaitivu Northern 56 0.14 Akkaraipattu Eastern 15 0.09 Total 17,234 34,095 41,344 100.00 100.00 100.00 OUSL = Open University of Sri Lanka. a Centers in estate areas. Source: OUSL. 2015. OUSL Statistical Handbook 2014. Colombo.