Lessons from Indian Flagship Programmes: The Disconnect for Evaluation Framework 1

Abstract This paper critically examines the management information system (MIS) in thirteen Indian flagship programmes that are under operation for quite sometimes now. Yet the MIS for these programmes has not reached maturity and as many as four programmes do not have any MIS. Clearly, there is a disconnect among the information gathering, monitoring and measuring impacts in the development programmes.

Key Words: Management information system, Flagship programmes, Log-frame hierarchy, Result-based monitoring and evaluation, Implementation framework

Introduction The MIS plays a critical role in the implementation of programme in terms of monitoring periodic progress. A well designed MIS facilitates flow of information among various levels and enables setting up of the necessary feedback mechanism for planning and management of a programme, project or policy. A comprehensive MIS is a necessary condition for taking informed and timely decisions including those related to operational, strategic and tactical ones. A well designed management information system (MIS) must be simple and easy to comprehend by the different stakeholders of the programme at national, sub-national and community levels; and it should provide reliable information. The information should be specific, accurate and verifiable; and facilitate timely management decision in terms of frequency and flow of information (i.e. a two-way feedback system in a decentralized framework), and management of database. Information generated by the system should be easy to access, process and use thereby facilitating wider dissemination and it should be amenable to computer software. However, literature on information system in the national level planning and decision making process is scarce and most of the literature is contextualized to organizations that facilitates managers to take various decisions (Gupta, 1996). With the advancement of information and communication technology (ICT), importance of MIS has been widely acknowledged in resolving many decision making problems at macro level.

1 Dr. I.C.Awasthi, Faculty, Institute of Applied Manpower Research, , . e-mail: [email protected]

1 Conventionally, governance structures have been characterised by rule-based approaches that primarily focused on process regulation, compliance with centrally prescribed standards and rules. Performance has been thus judged not by the results or outcomes but by compliance with processes and inputs. This has severely undermined the performance of the development projects or programmes. However, in a changed economic milieu, the performance is evaluated by results or outcomes not by the outputs; the fallacy of mere outputs has been seriously questioned in evaluation literature (Kusek and Rist, 2004; Linda and Rist, 2009).

A large number of centrally sponsored and central sector schemes are implemented through different Ministries across the country. With enormous diversity in the implementation hierarchy across space, it has become all the more important to have information about the physical and financial details of a project or a programme in order to monitor the progress. The Eleventh Five Year Plan underscored the deficiencies in the existing accounting system for the Plan schemes and its inability to support informed planning, budgeting and effective monitoring of these schemes (Planning Commission, 2008).

This paper examines the efficacy of the MIS in flagship programmes and checks how credible MIS can become an effective tool for management. The MIS is critical for generating information which is indispensable for conducting result-based monitoring and evaluation (M&E). This paper argues that MIS is still in its infancy stage in most of the flagship programmes, and this has to be put in place in order to provide evidence- based management. The second section explores the rationale of MIS as an evidence-based tool for management. The third section reviews the MIS in India’s flagship programmes and the last section recapitulates some of the issues from the policy perspectives.

2 Rationale of MIS Some of the basic principles of MIS are indicated as under:

MIS has to be derived from to the Log Frame The MIS must describe performance and impact indicators with a clear inter se relationship. However, indicators for outputs are mainly used for monitoring and no separate indicators are used for activities or inputs. Indicators must be relevant, realistic, specific and readily available. In the implementation framework, these involve inputs, activities and outputs, while outcomes and impacts are parts of the result based monitoring and evaluation; though all these must function as an integrated whole. The log frame must clearly outline the defined and realistic objectives and assumptions that describe how the interventions are designed to work. Log-frame hierarchy must identify the inputs, activities, outputs, programme objectives and outcomes with explicit clear links. It has been noted that in most of the flagship programmes no log frame matrix is structured for the programme design – what to speak about revising the original programme design. Without detailing the critical components of the objective hierarchy, it is most likely that programme objective indicators are not specified and hence the impact of programme is unclear. Authentication and Validation The data need to be collected at disaggregated levels and it must be ensured that the data are collected with agreed indicators. The database must reflect the log frame structure, and comprise all the activities with budgets, and all the outputs with their respective indicators. The system should be able to quickly collect and edit data, summarize results and be able to adjust and correct errors promptly. Traveling of data from source area to web portal should be quick and supported by ICT infrastructure. Data Reporting Mechanism

The MIS must provide timely information about the project or the programme indicating how well and to what extent it has performed or not performed as envisaged in the implementation framework. The MIS must clearly indicate the demonstrated performance at the outputs and objectives levels. Activities are generally reported on a

3 recurrent basis, yet these can be reported as per programme strategy depending on the nature of the project or the programme. The information should be such designed as to expedite the reporting of the information. Efforts are needed to bring out analytical reports quickly in the form of fact-sheet that ought to give signals for policy purposes. However, there appears to be a gap in data generation and reporting system in some of the flagship programmes that often hinder in realization of the project or the programme objectives (for example, the rural road programme (PMGSY), Irrigation, Mid-Day Meal Scheme (MDMS) and Jawaharlal Nehru National Urban Renewal Mission (JNNURM)). Accessibility of Data for Public

The data and information need to be fully accessible to and usable for all the stakeholders including researchers and the community at large. The data collected at local or community level must bear the ownership among the people and eventually benefit the people from the information generated. The information should not only flow to the national monitors but also be made available to the community. It has been noted that in some of the flagship programmes, either no feedback system exists or the existing mechanism is weak (for example, Irrigation, Rural Telephony, Rural Electrification, Universalization of Elementary Education (SSA), Mid-Day Meal Scheme, and Integrated Child Development Service (ICDS)). Utilization of Data

Accessibility of data helps in using the data for analysis and further research, which eventually helps improve the implementation framework. The data can be utilized by the variety of stakeholders. The intent should be made transparent enabling the participation of the major stakeholders in the utilization of the data with a proper feedback mechanism. The MIS data are primarily used for monitoring purposes and there is virtually no linkage between the MIS information with that of evaluation studies. It has been noted that M&E functions in a compartmentalized manner rather than articulating relationship between the two. Enormous data are generated in every programme but there is very little utilization of these data in terms of generating analytical reports. Even evaluation reports do not use the data generated by these programmes.

4 Connecting Output, Outcome and Survey Data It is often observed that there is a weak connection or linkage between the implementation framework and the outcome framework within the perspective of the theory of change. This arises due to lack of clarity on the indicators and assumptions of the log frame. It is also noticed that there is a divergence between the project or the programme data and that of large survey data that often makes difficult to aid policy (ICDS and National Family Health Survey (NFHS), for example). User-friendly Website While developing a website, the users’ needs should always be in the forefront of the decision-making process. The design and development process must create a site that satisfies the visitors with customer-focused content. The website should have an in-built feedback device. Feedback is an essential component of the evaluation process that provides the link between the past and future activities. In order to ensure that the results of evaluations are utilized in future policy and programme, it is necessary to establish feedback mechanisms involving all the stakeholders (OECD, 1991). But most of the flagship programmes in India though have dedicated websites but in some cases these do not have any feedback devise and lack user-focused design.

MIS in Flagship Programmes There is a growing need for effective and efficient utilization of public money through continuous monitoring process in order to measure the progress. Precisely, with this in view, every project and programme has some monitoring process in place in order to track the progress. For major schemes, MIS have been developed by the different Ministries for tracking the progress of the schemes, both in terms of physical and financial parameters (PEO, 2009).

At present as many as thirteen flagship programmes are being implemented, primarily with a view to constructing of rural infrastructure; education for all; improving health and sanitation; providing opportunities for improved living conditions as well as

5 livelihoods; and augmenting the economic and social infrastructure of cities etc. with huge investments.2 This section examines the MIS in the flagship programmes which is in no way exhaustive appraisal but indicative in some sense that might raise research questions for further work in this area. This section draws on preliminary analysis based on the information available in the website. There are no MIS (as many as in four flagship programmes) worth the name and also in many cases one is not permitted to visit MIS online without a valid authorization. The programmes which have no MIS are: Irrigation, Rural Telephony, Mid-Day Meal Scheme and Jawaharlal Nehru National Urban Renewal Mission. The flagship programmes are as under: 1. Building Rural Infrastructure: Bharat Nirman (www.bharatnirman.gov.in) (i) The Rural Road Programme: Pradhan Mantri Gram Sadak Yojana (PMGSY) (ii) Low Cost Housing to Rural Poor: Indira Awaas Yojana (IAY) (iii) National Drinking Water Programme (iv) Irrigation (v) Rural Telephony (vi) Rural Electrification Programme: Rajiv Gandhi Gramin Vidyut Yojana (RGGVY) 2. The Mahatma Gandhi National Rural Employment Guarantee Act (MGNREGA) 3. Universalization of Elementary Education: (SSA) 4. Mid-Day Meal Scheme (MDMS) 5. Integrated Child Development Services (ICDS) 6. National Rural Health Mission (NRHM) 7. Jawaharlal Nehru National Urban Renewal Mission (JNNURM); and 8. Total Sanitation Campaign (TSC)

First, we discuss national flagship programmes that have MIS and subsequently those programmes which do not have.

The Rural Road Programme: Pradhan Mantri Gram Sadak Yojana (PMGSY)

The programme was launched in December, 2000 by the Ministry of Rural Development, . The primary focus of the programme is to provide

2 Total allocation under Flagship Programmes during the XIth Five Year Plan is of the order of Rs. 5,310 billion and for 2011-12 it is as under: Bharat Nirman: Rs. 5,80,000 million; MGNREGA: Rs. 4,00,000 million; SSA: Rs. 2,10,000 million; MDMS: Rs. 1,03,800 million; NRHM: Rs. 1,81,720 million; JNNURM: Rs. 1,37,000 million; TSC: Rs. 16,500 million; ICDS: Rs. 1,03,300 million.

6 rural road connectivity that involves construction of new roads and up-gradation of the existing roads so as to ensure connectivity through good all-weather roads. The key component of this programme is to improve the access to economic and social services and thereby generating increased agricultural incomes and productive employment resulting in a sustainable poverty reduction. A modern management and monitoring system has been set up for the PMGSY (www.pmgsy.nic.in). This web-based online monitoring system is developed to facilitate monitoring and create transparency and embedded decision support systems for on-time delivery, cost management and quality control. The entire database is placed in the citizens’ domain. The analysis of the MIS we undertook, brought out some of the following observations:  The guidelines deal primarily with the inspection procedure of the works rather than monitoring and evaluation. It focuses on a three-tier quality control and quality monitoring mechanism. The procedure discusses the inputs, activities and outputs.  Some of the information on the on-going and completed works is of qualitative nature (comments and observations) which is often hard to measure directly. There is therefore a need for evolving proxy to measure the progress.  Quality monitoring indicators are not expressed in terms of quantifiable entities. For instance, it is very difficult to assess the progress on the basis of judgments such as ‘satisfactory’ or ‘unsatisfactory’.  National, state and district reports are also prepared in terms of physical and financial targets and achievements, but it is not clear how this information is being used to track the progress.

Low Cost Housing to Rural Poor: Indira Awaas Yojana (IAY)

The IAY, low-cost housing for the rural poor, was launched during 1985-86. The objective of the IAY is primarily to help construction/up-gradation of dwelling units of the members of socially disadvantaged groups and other minorities in the below poverty line (BPL) category living in rural areas (www.bharatnirman.gov.in).

There is an on-line data entry provision in the portal by field staff. Guidelines and the prescribed procedure have been neatly structured and also envisioned that officers dealing with IAY shall ascertain through field visits whether the programme is being

7 implemented satisfactorily and whether construction of houses is in accordance with the prescribed procedure. The state governments shall conduct periodic evaluation studies through reputed institutions and organizations on issues thrown up by the concurrent evaluation. The software was designed and developed for collecting information from the location where the scheme is implementing. Main purpose of MIS is to collect data to know the progress of physical and financial aspect of the scheme

(http://iay.nic.in).

The portal contains a variety of information at Gram Panchayat, block, district, state and national level on BPL, above poverty line (APL) families requirements of houses, women headed houses; disabled persons’ housing requirement details etc. However, updating data is perennial problem owing to lack of ICT infrastructure.

An examination of the MIS reveals some of the issues as follows:

 State-wise and district-wise monthly progress report (MPR) is presented in terms of physical and financial parameters, but it is not clear how the information is being utilized to measure the outcomes.  The scheme was implemented from 1st January, 1996 but the information is available for the last three years i.e. 2009-10, 2010-11 and 2011-12 in the MIS.

 MPR in terms of physical, financial details by state/district-wise is given, but it is not mentioned for which month the data relate to.

National Rural Drinking Water Programme (NRDWP)

NRDWP is one of the six components of Building Rural Infrastructure: Bharat Nirman. During its Phase – I period (2005-06 to 2008-09), 55,067 uncovered and about 0.33 million slipped back habitations were to be covered with provisions of drinking water facilities and 0.217 million quality-affected habitations were to be addressed for water quality problem. The phase II is being implemented from 2009-10 to 2011-12. The Department of Drinking Water Supply, Ministry of Rural Development, is responsible for meeting this goal in partnership with the State Governments (www.bharatnirman.gov.in). Ministry has been assisting the States and Union Territories (UTs) to accelerate the pace of coverage of drinking water supply. The entire programme was given a

8 mission approach in order to improve the performance, cost effectiveness and to ensure adequate supply of safe drinking water. The Technology Mission on drinking water and related water management was launched in 1986. It was also called the National Drinking Water Mission (NDWM), which was renamed as Rajiv Gandhi National Drinking Water mission (RGNDWM) in 1991(www.ddws.nic.in). From 2005 onwards it is called NRDWP and it is the only flagship programme to provide safe drinking to every habitations/villages. The scheme is 50:50 cost sharing basis between the Central and State governments. State government is fully implementing the scheme because water resource is state subject. During the last one and half year the Integrated Management Information System has been functioning (including for TSC) (www.indiawater.gov.in).

It is also planned to introduce the automatic short message service from next financial year for getting MPR and other related information. Also, there is a plan to introduce social auditing by including the village communities and to utilize Block Resource Centre-block level to be made nodal centre of the scheme (every block will have a technical person in charge for the scheme). Then the data entry level would come down one level below from district to block. Some of the observations on the scheme are described as under:  The data is cumulative from 2005-06 to 2011-12 making it difficult to carry out the analysis annually. It is also not clear whether data relates to monthly or quarterly.

 The delivery monitoring unit (DMU) report is quarterly but the information is given on annual basis. It would have been better had the information is collected and presented quarterly basis in order to monitor the effectiveness of implementation.

 It is not clear whether the information provided is used for the preparation of analytical reports in order to measure progress.

 As regards accuracy of data, Ministry depends completely upon the state governments. The data is placed in the portal by the district officials dealing with the programme. No validation mechanism exists in this process.

9  Monitoring is done by the concerned state government department and not by the Ministry. However, if wrong or inconsistent data is noticed then concerned person in the Ministry points it out then the same is got rectified by the concerned officials from the state government.

 Some sort of feedback mechanism flows to the Ministry from the local elected representatives/audit performance report/general public about the progress and the problems in the programme. It is not clear how and to what extent this feedback is incorporated in the programme.

 There is huge data generated in the scheme but it is hardly used for analysis purposes and there is vast scope for statistical analysis of the data, in particular interlinking of health indicator with safe drinking water which is extremely important. It is also unclear how huge data collected would help in generating analysis for policy purposes since no analytical framework appears to exist.

Rural Electrification Programme: Rajiv Gandhi Grameen Vidhyutikaran Yojana (RGGVY) The Ministry of Power has, a rural electrification programme, which is one of the flagship programmes launched in March 2005 with an objective to electrify over 0.1 million un-electrified villages and to provide free electricity connections to 23.4 million rural BPL households. The MIS of the RGGVY is detailed in terms of projects approved for implementation, achievements and compliance (www.rggvy.gov.in).

Under the programme 90 percent grant is provided by Govt. of India and 10 percent as loan by Rural Electrification Corporation (REC) to the State Governments. REC is the nodal agency for the implementation of the programme. The scheme aims at: Electrifying all villages and habitations as per new definition, providing access to electricity to all rural households, and providing electricity connection to BPL families free of charge. The basic infrastructure under RGGVY scheme is; rural electricity distribution backbone with 33/11 KV (or 66/11 KV) sub-station of adequate capacity in blocks where these do not exist; village electrification infrastructure with provision of distribution transformer of appropriate capacity in villages/habitations; and decentralized distributed generation systems based on conventional & non- conventional energy sources where grid supply is not feasible or cost-effective.

10 Some preliminary observations of the above MIS are presented as under:  The MIS is very brief (one page) that has little or no usability for analytical purposes for evaluating the scheme.

 While the RGGVY claims that a village or a BPL rural house hold has been electrified, there is no information whether any electricity ever flows through the lines that are running into the village or households. Whenever urban areas are suffering serious load-shedding, it is unlikely that rural BPL households are receiving power.

The Mahatma Gandhi National Rural Employment Guarantee Act (MGNREGA)

The MGNREGA, 2005 aims at enhancing the livelihood security of the people in rural areas by guaranteeing hundred days of wage employment in a financial year to a rural household whose adult members volunteer for doing unskilled manual work. This is one of the major programmes launched by the Ministry of Rural Development (MHRD). The scheme was started in February 2006, initially in 200 districts, and later extended to all districts from April 1, 2008 (www.nrega.nic.in). Massive information is collected – right from the registration to implementation stage from about 0.25 million Gram Panchayats, 6,465 Blocks, 619 Districts and 34 States & UTs through computerized information system. The MIS portal provides a single-point access to all stakeholders of MGNREGA. The aim is to provide data that are transparent and user friendly. The portal places complete transaction level data in public domain for job cards, demand for work and muster rolls (attendance-cum-payment sheet) for a worker. Proper checks are used to validate the data entered and to stop the wrong entries in MGNREGASoft. Recently, biometric database of the MGNREGA linking Unique Identification Authority of India has been introduced and it is expected to bring about more transparency, to curb irregularities and reduce corruption. The DMU and MIS reports provide information on employment, works and funds across states. Information for the majority of states and UTs is provided for current years and months. Some of the preliminary observations on the MIS are as follows:

11  The DMU treats employment generation, assets and finances as ‘outcomes’ which in fact are outputs and not outcomes. Outputs and outcomes are conceptually misconstrued.  In the case of MGNREGA, there is divergence or lack of update in Prime Minister’s Office (PMOs) DMU section of the scheme. Data are available for the years 2008-09 and 2009-10 only in PMO’s DMU but in the scheme DMU Report section, the information is available till November, 2011.  The time taken between data sending and uploading ranges between 1 to 7 days.  There appears to be a time-lag in the ICT network interconnecting all the village committees, blocks, districts, States and the union ministry, particularly, in certain small states/UTs. Lack of a dedicated broadband connectivity at the block level hampers the flow of information upwards.  Despite the fact that MGNREGA has the most comprehensive MIS system, it has not been able to raise an alert on delays in wage payments. The principal reason for this is attributed to an undue delay in updating the website data, and hence undermining the very basis of monitoring (Ambasta, 2009).  The assets have been reported in the form of number of assets taken up and completed, but there is no information regarding their durability and sustainability – an issue that has never been raised by the MHRD.  The overall status of the completed works as per cent of total projects turns out to be less than half (49 per cent for the financial year 2009-10) with huge differences across states.  It would be useful if the time taken for transfer of funds (flow chart) from different sources (Centre, State, district planning committee, programme officer at the village committee level) are also presented which would have been helpful in tracking the financial progress.  Enormous data at disaggregated level are collected but no analytical reports are being generated, which could have been an effective instrument for monitoring the progress and taking corrective measures.

Universalization of Elementary Education: Sarva Shiksha Abhiyan (SSA) The SSA is Government of India's flagship programme for achievement of Universalization of Elementary Education in a time-bound manner, as mandated by the 86th amendment of the Constitution of India – making free and compulsory education to the children of 6-14 years age group a fundamental right. The scheme of SSA was launched by the Ministry of Human Resource Development in 2001 and is being

12 implemented in partnership with the state governments to cover the entire country addressing the needs of 192 million children in 1.1 million habitations (www.ssa.nic.in). The monitoring and MIS unit are in place. The monitoring institutions have been identified state–wise, but much of the work of monitoring is to bring out data on access, enrolment & retention and quality (including infrastructure).The MIS has been divided into three modules viz. district level, state level and ministry level. Each level has four types of users, namely, data entry, data approver, view reports, and user manager. Some of the observations made are recorded in what follows:

 Surprisingly, there is no access to the MIS for the public that prevents one to make any assessment of the information system of such a massive programme.  The data reported under the SSA web portal is only the quarterly progress of the physical and financial activities of the scheme.  There is very limited updated information in the portal of the scheme. The on-line data entry provisions are in place in the portal but updated information on all the aspects and for all the states is not placed on time. The latest data paced are for the first quarter (April - June) of 2011. The DMU information is also available up to June, 2011.  No feedback mechanism is developed in order to improve the MIS.  Travelling time of data from source to destination is between 15-30 days.  The outcome of the programme is not being checked with the programme objective; presumably no log-frame has been used in the design.

Integrated Child Development Services (ICDS) The ICDS is one of the most unique programmes for early childhood development launched on 2nd October, 1975 by the Ministry of Women and Child Development, Government of India. Its objectives are to improve the nutritional and health status of the children in the age-group 0-6 years; to reduce the incidence of mortality, morbidity, malnutrition and school dropout; and to enhance the capability of the mother to look after the normal health and nutritional needs of the child through proper nutrition and health education (www.wcd.nic.in). The MIS is in place since its inception but it is inadequate as an effective tool for improving programme implementation (Adhikari and Bredenkamp, 2009). As yet, there is no online computerized MIS developed for the ICDS and Centre is the focal point of delivery of services under the scheme. Therefore, all the data pertaining to

13 the services (six types of records and registers prescribed) and the beneficiaries are captured at this level. Some of the observations on the MIS structure are presented below:  The information about the scheme especially on the physical targets and achievements; and the release of fund are given in the portal. Data formats do not help in checking the consistency as the data were presented quarterly and annually, separately. The one-point data are given in a particular year that makes difficult to report the progress over time.

 No uniformity is followed in the presentation of data under the scheme.  The given information is not available by social groups.  The information provided does not help much to track the progress of the scheme.  No institutional mechanism is in place in order to know the links between the progress made and the financial allocation for the scheme.  There is neither any feedback device in the portal nor any evaluation report placed in order to examine the effectiveness of the scheme or the results.  No data are available for DMU section.  The MIS programme mainly tracks inputs, some of the intermediary processes and limited outputs rather than nutrition outcomes, such as, indicators of appropriate infant care and feeding behaviours (Adhikari and Bredenkamp, 2009).  The divergence in the ICDS and NFHS findings poses some kind of puzzle that needs to look into and a mechanism need to be evolved for data authentication and validation.

National Rural Health Mission (NRHM) The NRHM was launched in 2005 by the Ministry of Health & Family Welfare with an objective to improve the availability of and access to the quality health care by the people, especially the poor, women and children and those who reside in rural areas. The mission (2005-12) seeks to provide an effective healthcare to rural population throughout the country with special focus on 18 states, which have weak public health indicators and/or weak infrastructure. The health MIS was launched in October, 2008 and for capturing data it has generated formats on a web-based system at the district level so that the primary data can easily be aggregated and the information/reports flow quickly to the State headquarters and the Ministry. It also compiles periodic reports from Health Management Information System (HMIS) portals (www.mohfw.nic.in). Some preliminary observations of the MIS are presented below:

14  The scheme has collaboration with the international agencies like World Health Organization, World Bank, The United Nations Children's Fund to build an accurate reporting system under the HMIS.  More focus is needed on generating analytical reports, which should help measuring the real outcomes and impacts.  The HMIS system has a strong mechanism to control inconsistency and wrong entries and assures reliable and consistent information and reporting.  Community-based monitoring of health services is a key strategy of the scheme and there is a provision for monitoring and planning committees at primary health centre (PHC), Block, District and State levels.  The MIS does not report any information on maternal deaths and estimates of maternal mortality ratio on a regular basis.  For monitoring and evaluation of the programme, health MIS should be developed up to the community health centre level. Sub-centers, primary health centres and community health centers would be reporting on their performance regularly to the people’s bodies like village committees and District Health Mission (Bajpai, et al., 2009).  HMIS reporting system should entertain feedback/suggestions from its stakeholders at all levels.

Total Sanitation Campaign (TSC)

The TSC was launched by the Department of Drinking Water and Sanitation, Ministry of Rural Development. TSC is a comprehensive programme to ensure sanitation facilities in rural areas with broader goal to eradicate the practice of open defecation. TSC as a part of reform principles was initiated in 1999 when Central Rural Sanitation

Programme was restructured making it demand driven and people centered. The main goal of the TSC is to eradicate the practice of open defecation by 2010 (www.ddws.nic.in). According to National Sample Survey 2009, 51 percent of households in India do not even have a toilet, and many of those that do have toilets do not use them. However, the objective seems to be distant dream to be realized. For TSC, on-line progress monitoring system is in place and data entry system (district wise) has been devised. The state has designed district, block and panchayat level data collection form and base-line survey for reporting monthly progress. The information as on date is available but the portal is not updated with its site map. TSC is,

15 probably the first scheme which introduced the on line MIS among the government of India’s programmes in 2002 and other programmes followed the MIS pattern of TSC. Under the online monitoring system, the household sanitation details for villages/habitations are entered at district level on daily basis. The scheme is being implemented in 609 districts in the country of which almost 80-90 percent of the district send their monthly reports on time (for instance, monthly report for December 2010 is forwarded by 403 districts as on today). There are certain unexpected delays owing to technical and communication (on line connectivity) that hinders the delay in submission of monthly reports on time.

District level officials who are in charge for the scheme are responsible for editing and accuracy of the data given in the portal. For validation and cross checking the data, other national and international organizations are also independently doing the field studies. Panchayat Raj Department is planning to give the ICT facilities to every panchayats; once it is being implemented the data would be available straightaway from panchayats which will ensure more transparency, faster and timely availability of data.

In spite of relatively better performance of the scheme, there are certain issues that need attention:

 The goal of the TSC is to eradicate the practice of open defecation by 2017 seems distant objective, as the scheme is nowhere near the goal. Evidently, lack of monitoring of actual use of toilets built and the poor quality of toilets appears to be the reason for poor progress.

 There are no reports based on the MIS data

There are following four flagship programmes that do not have MIS. Irrigation The scheme is being implemented by the Ministry of Water Resources in collaboration with State Governments for creation of additional irrigation capacity. There is a huge gap between irrigation potential created and the potential utilized. There are considerable areas in the country with unutilized ground water resources. The target of creation of additional irrigation potential is 10 million hectare. The 4.2 million hectare is

16 planned to be created by expeditiously completing such on-going major and medium projects, one million hectare through implementation of extension, renovation and modernization of schemes along with command area development and water management practices, 2.8 million hectare through ground water development, one million hectare is planned by way of minor irrigation schemes using surface flow and one million hectare is also planned by way of repair, renovation and restoration of water bodies and extension, renovation and modernization of minor irrigation schemes (www.bharatnirman.gov.in). No online system is in place for data collection. Only the Ministry is the predominant user of the data which is for own review and sanctioning and release of grants. Headquarters knows the range for the data which is normal for a state. On that basis, a check is done at head quarter. Through the monitoring of the data the inconsistency is checked and if any visible inconsistency found then concerned states are asked to rectify the data. During the review meeting organized at head quarter each states information were used to do double check for the approval/sanctioning.

Some of the initial observations on the above programme are as follows:  No MIS has been developed for the irrigation component of Bharat Nirman. However, information on achievement is given till 2009-10. Even that information is not complete for all the states and in some cases either the information is not reported or partially reported.

 There is divergence in the data presentation and updating of the data. The web portal gives DMU data for the year 2010-11 (data up to 10th March, 2011). The PMOs’ DMU report gives the data for 2009-10 only.

 The DMU format suggests that a quarterly report must be submitted, but surprisingly, achievements by the Ministry are given for 2009-10 and 2010-11, that is the entire year

 The information regarding achievement is given as reported by the state governments but, it is not clear whether or not the information is validated at the level of Ministry.

17  No formats for data collection are provided either on-line or otherwise without which it becomes extremely difficult to make any observation and track the monitoring.

 No feedback mechanism exists from the major stake holder’s point of view.

Rural Telephony

Telecommunication connectivity constitutes an important part of the effort to upgrade the rural infrastructure and it is important to improve the tele-density in rural areas. It aims at achieving 40 per cent rural tele-densities by 2014 and ensures broadband to all panchayats by setting up India Building Service Centre. The Department of Telecommunication in the Ministry of Communications and Information Technology has the responsibility of providing telephone connectivity to the villages that remain to be covered (www.bharatnirman.gov.in). The Central Government provides village public telephones in uncovered villages through subsidy support through universal service obligation fund. The target of the scheme is that, under the Bharat Nirman Programme, rural teledensity of at least 40 per cent by 2014, and broadband coverage of all 2,50,000 village panchayats and Setting up of Bharat Nirman Common Service Centers at Panchayat level by 2012 will be achieved

Some preliminary observations are reported as under:

 There is no MIS for the programme even though the programme was initiated in 2005, and without MIS it becomes extremely difficult to monitor the programme effectively.

 There is very little data available on the website that too is highly aggregated and not beyond state/ circles levels.

 There is no provision of on line data submission and data is provided by service provider. State-owned Bharat Sanchar Nigam Limited (Indian state-

18 owned telecommunications company) is the largest service provider for telephones and broad band connectivity, which can only provide date.

 Data forwarding authority (at state level) have to self certify for the validity of the data. Controller of Communication Accounts also cross checks the service on sample basis and conducts monthly or quarterly review of the progress of the work performance etc.

 Latest information on rural tele-density is provided till 2009. But the information is insufficient to make any kind of analysis. Disaggregated information on a number of revenue villages are provided with village public telephone (VPT), remotely located villages provided with VPTs through digital satellite phone terminals. Broadband coverage would have been useful to make any definitive analysis across districts and states.

 Similarly, targets and achievements should have been presented at disaggregated levels.

 The DMU report gives only broadband coverage (up to March, 2011) and not VPTs year-wise.

 A comprehensive MIS detailing activity and outputs is warranted.

Mid-Day Meal Scheme (MDMS)

MDMS, the world’s largest school feeding programme, was launched by the Ministry of Human Resource Development in 1995, which is aimed at boosting universalization of primary education and is expected to attract enrolment, attendance, retention and reduction of the drop-out rates, simultaneously improving the nutritional levels among children. The scheme is spread over 1.2 million schools and covering over 120 million children’s in the country and collecting information on time. There are reports that show that as a result of MDMS there has been a remarkable improvement in attendance and retention of the children in the schools. No MIS has been developed for the scheme separately. However, the mechanism for monitoring and supervision of the MDMS has been in place which includes local level

19 monitoring, display of information under Right to Information Act and inspection by the state government officials, periodic returns etc. (www.education.nic.in). There are national, state review committees and also over three dozen social science research institutions that have been identified and entrusted for monitoring the scheme. These institutions bring out monitoring reports based on 5 per cent sample basis. But there is no on-line monitoring mechanism. The scheme largely relies on inspection as a principal monitoring instrument that primarily focuses on inputs and activities rather than deliverables. The National Programme of Nutritional Support to Primary Education was launched as a centrally sponsored scheme on 15th August, 1995. However, data are presented only for last two years (2008-9 and 2009-10). Information for first quarter 2009-10 (April-January) is available only for 14 states. Even quarterly information for the last quarter (January-March) for the year 208-09 is not available for all the states.

At present the scheme has wide coverage - 120 million children, 1.2.6 million schools, 2.4 million staff (cooks cum helpers) and 0.58 million kitchen cum store rooms. At present a system is in place to gather all the important data in a systematic manner. It is planned to have the Mid Day Meals Index for the states/schools on the lines of human development index

Some of the pointers for assessment bring about the following observations:

 No MIS has been developed for the scheme even after existence of this scheme for quite long time. Without comprehensive MIS, monitoring of the scheme becomes somewhat difficult.

 The information is available for fewer states for the year 2011-12 and majority of the states have not yet submitted their last quarter reports for the year 2010-11. No information is available beyond the state levels and even no consolidated all India report is available.

 Normally, monthly statement should be submitted by the district level officials to the states and states in turn are required to compile it and forward it to the Central Ministry.

20  The information also does not provide disaggregation by gender and social groups.

 Some linkages between provision of mid-day meals and enrolment and dropouts may also be included in the report formats in order to determine whether the programme is performing according to plan and budgets.

 There is no feedback devise under the scheme that is essential to improve the design for effective implementation of any scheme/programme.

 Enrolments data are presented for one point of time and it would be worthwhile to add for more years in order to make the comparison over time.

Jawaharlal Nehru National Urban Renewal Mission (JNNURM)

The JNNURM was launched by the Ministry of Urban Development in 2005 with the objectives to create economically productive, efficient, equitable and responsive cities; to improve and augment the economic and social infrastructure of cities and to ensure basic services to the urban poor including security of tenure at affordable prices, etc. The JNNURM has two sub-missions – one on urban infrastructure and governance and another on basic services to urban poor. Urban infrastructure and governance has elaborate programme monitoring and evaluation system that has user manual data entry formats for city development plan. Monitoring formats have been devised for quarterly reporting. City development plan and detailed project report user manuals have been prepared as tools for programme monitoring and evaluation system (www.jnnurm.nic.in). Elaborate guidelines have been structured and expected outcomes outlined but how these outcomes will be achieved is not defined. Some of the observations on the scheme are as follows;

 Amazingly, no clear time-lines with mile-stones of project are given. Similarly, flow of daily progress report (DPR) is presented and progress is tracked through on-line but it is not clear how frequently the progress is tracked and how the information generated is analyzed for the operational and strategic decisions.  More quantitative and qualitative details are required for better reporting of the scheme, particularly in regard to projects proposed, projects completed on housing, road, drainage, etc. along with cost benefit analysis, impact assessment etc.

21  The quarterly progress report is available up to the first quarter (April-June, 2010). Apparently, time-lag is substantive in reporting the information.  There is no information on which analytical reports are prepared based on the information generated, nor if there outcome indicators developed in the case of DPR of beneficiaries.  No feedback device is given in the portal which is essential to measure the project implementation framework.

The Way Ahead It is of paramount importance to institutionalize a credible MIS in every major project or programme with a detailed conceptual framework. It has been noted that in many of the programmes or projects there have been some sort of MIS for tracking the progress, but still there is a scope for further improvement so that it becomes evidence- based management tool. Massive public investments are being made on development programmes and projects and obviously governments and other stakeholders want to know how well and to what extent the delivery mechanism is achieving the desired goals or intents of policies. For instance, in thirteen flagship and iconic programmes Rs. 5,310 billion has been proposed to be invested during the XIth Five Year Plan, and without any credible MIS and monitoring system in place the efficacy and effectiveness of these programmes will remain largely unknown. There is disconnect between the information gathering, monitoring and measuring impacts in most of the projects or programmes. Even having elaborate information collection system and monitoring mechanism in place does not necessarily lead to impact evaluation of most of the social programmes. Report of the Evaluation Gap Working Group (2006) aptly described “most of these resources are directed toward monitoring the use of funds, deploying and managing personnel, and producing outputs and services. By contrast, relatively little is spent to rigorously assess whether programs are having the desired impact beyond what would have occurred without them”. There is greater demand for effective and efficient utilization of public money with clear impacts and governments are under enormous pressure to demonstrate results both on account of their survival and greater value to public money.

22 There is, therefore, a need for an M&E framework in place in every project or programme that eventually aids to improve the design and delivery of projects, programmes and policies and it must move beyond an emphasis on inputs and outputs to a greater focus on outcomes and impacts or results. There ought to be visibly clear linkages between evaluation findings and resource allocation in order to narrow down the hiatus between the intents and outcomes of the programmes. We must clearly learn the lesson that is driving development evaluation in many developing countries with clear results and also to unlearn from our own experiences. Obviously, there is a need for building institutional capacity, developing capabilities and competencies and strong demand for ownership of an evaluation system in order to spreading evaluation culture as part of economic reforms. Acknowledgements Author is grateful to Dr. Santosh Mehrotra for guidance and helpful comments and Mr. Indra Kumar for providing valuable inputs in preparation of this paper.

Bibliography

1. Adhikari, Saroj K. and Bredenkamp, Caryn, Moving Towards an Outcomes- Oriented Approach to Nutrition Program Monitoring: The India ICDS Program. Discussion Paper, Health, Nutrition and Population, The World Bank, 2009. 2. Ambasta, Pramathesh, Programming NREGS to Succeed. The Hindu, October, 30, 2009. 3. Bajpai, Nirupam, Jeffrey D. Sachs and Dholakia, Ravindra H., Mid-term Evaluation of the National Rural Health Mission. Working Paper No. 37, Center on Globalization and Sustainable Development, The Earth Institute at Columbia University, 2009. 4. Center for Global Development, Report of the Evaluation Gap Working Group, When Will We Ever Learn? Improving Lives through Impact Evaluation. Washington, D.C., 2006. 5. Gupta, M.P., Management Information System at National Level in India: Analysis of Problems and Issues. Vikalpa, Vol. 21, No. 3, July-September 1996. 6. Kusek, Jody Zall and Ray C. Rist, Ten Steps to a Result-based Monitoring and Evaluation System. The World Bank, DC, 2004. 7. Linda, G. Morra Imas and Ray C. Rist, The Road to Results: Designing and Conducting Effective Development Evaluation. The World Bank, Washington DC 20433, 2009. 8. Organization for Economic Co-operation and Development, DAC Principles for Evaluation of Development Assistance. Paris, 1991.

23 9. Planning Commission, Government of India, Eleventh Five Year Plan (2007-12), Vol. I, Chapter 3, 2008. 10. Programme Evaluation Organization (PEO), Planning Commission, Monitoring Information System (MIS) of Major Schemes. September, 2009. 11. http://www.bharatnirman.gov.in; www.pmgsy.nic.in; www.nrega.nic.in; http://www.ssa.nic.in; www.wcd.nic.in; www.mohfw.nic.in; www.ddws.nic.in; www.education.nic.in; www.jnnurm.nic.in

Glossary

Anganwadi: Courtyard shelter is government sponsored child-care and mother-care centres.

Bharat Nirman: Unlocking rural India’s development potential through infrastructure.

Block: Community development block is a rural area earmarked for development.

Gram Panchayat: Gram panchayat or village panchayat is units of self-government. They are like village courts.

Gram Sabha: All men and women in the village who are above 18 years of age form the Gram Sabha (village councils).

Panchayat Raj Department: Government by the village council.

Indira Awaas Yojana (IAY): The scheme is named after the former Prime Minister Mrs. Indira Gandhi.

Jawaharlal Nehru National Urban Renewal Mission (JNNURM): The programme is named after the first , Jawaharlal Nehru.

Mahatma Gandhi National Rural Employment Guarantee Act (MGNREGA): The programme is named after the Father of the Nation, Mahatma Gandhi

Pradhan Mantri Gram Sadak Yojana (PMGSY): Rural road connectivity programme named after the Prime Minister.

Rajiv Gandhi Grameen Vidyut Yojana (RGGVY): The programme is named after the former Prime Minister Mr. Rajiv Gandhi.

Sarva Shiksha Abhiyan (SSA): is a programme for achievement of universalization of elementary education (UEE) in a time-bound manner making free and compulsory education to the children of 6-14 years age group, a fundamental right.

24