IMPACT ASSESSMENT OF UPAZILA PARISHAD GOVERNANCE PROJECT [UZGP] & UNION PARISHAD GOVERNANCE PROJECT [UPGP]
FINAL REPORT JUNE, 2017
Team Leader Dr. Salahuddin M. Aminuzzaman
Team Members Dr. Mobasser Monem Dr. Ferdous Arfina Osman Dr. Mohammad Shuaib Dr. Bazlul Huq Khondker Dr. Mohammad Abu Eusuf Abu Hossain Muhammad Ahsan Mr. Riaz Uddin Khan
2 Table of Contents
Composition of the team 2 Abbreviations and Acronyms 6 Maps of the Study Area 7 Executive Summary 8-15 Foreword 16
Part I Overview of Project Context 17-29 1.1. Introduction 1.2. Relevance of Project interventions and Government of Bangladesh Policy framework 1.3. UPGP and UZGP At a Glance- Objectives and Result framework of UPGP and UZGP 1.4. Background of Assessment 1.5. Reflections from other studies having bearing upon UPGP and UZGP 1.6. Scope and Objectives of Assessment
Part II Evaluation Methodologies 30-41 2.1 Survey Methodology of the study 2.1.1 Overview of survey design 2.2 Study area and target population of the study 2.2.1 Study area 2.2.2 Target population 2.2.3 Matching methodologies 2.3 Matching results 2.3.1 Methods of data collection 2.4 Quantitative survey 2.4.1 Sample size 2.4.2 Sample design 2.4.2.1 Sample design for Upazila Parishad and Union Parishad Surveys 2.4.2.2 Sampling design for household survey and Household selection 2.4.2.3 Sampling design for exit client survey 2.5 Qualitative survey 2.6 Review of Secondary Data 2.7 Data collection methodologies 2.7.1 Development of study instruments 2.7.2 Recruitment of and training to field staff 2.8 Data collection operation 2.8.1 Field visits and quality checking 2.9 Data management and quality assurance
3 2.10 Data Analysis and preparation of the report 2.11 Special consideration and limitation of the methodology
PART III UPGP: Overview of Impacts/Results 42-137
3.1. Impact Assessments of Results: UPGP Output 1 3.1.1. Key Findings 3.1.2. Capacity building of UP Chairs and members in the light of roles and responsibilities defined by UP Act 2009 3.1.3. Activating Ward Shavas for inclusive decision-making 3.1.4. Strengthening Standing Committees for Effective Governance 3.1.5. Up-Scaling of Women’s Development Forum 3.1.6. Building Citizenship and Promoting Downward Accountability and overall efficiency and effectiveness 3.1.7. Overview of performance grants for improving core governance areas of UPs 3.1.8. Identifying the areas of relative performance on composite governance indicator
3.2 Impact Assessments of Results: UPGP Output 2 3.2.1. Key Findings 3.2.2. Quality of development plans and needs assessments 3.2.3. Quality of financial and revenue management 3.2.4. Quality of budgeting, auditing and reporting practices 3.2.5. Number and nature of gender focused projects 3.2.6. Extent of use and impact of Right to Information 3.2.7. Performance and Challenges of WDF 3.2.8. Use and Impact of Citizen Charter 3.2.9. Assessment of quality of service delivery system 3.2.10. Right to Information at UP level
PART IV UZGP: Overview of Impacts/Results 138-202 4.1. Impact Assessment of Results – UZGP Output 1 4.1.1. Key Findings 4.1.2. Capacity and role of UZP members 4.1.3. Women Development Forum 4.1.4. Citizen Charter 4.1.5. Performance grants and improvement in core governance areas
4.2 Impact Assessment of Results – UZGP Output 2 4.2.1. Key Findings 4.2.2. Extent of links of the plans of the Line Departments
4 4.2.3. Quality of plan and budget making and execution 4.2.4. Co-financed Upazila level projects 4.2.5. Impact of performance based UFF to Upazila and how good practices have been replicated in other interventions 4.2.6. Assessment of quality of service delivery system
4.3 Impact Assessment of results-Output 3 4.3.1. Key Findings 4.3.2. Policy Advisory Group (PAG) 4.3.3. Policy Advocacy 4.3.4. System design 4.3.5. Peer Learning Forums 4.3.6. Policy Dialogue Forums
PART V Economic Analysis of the Project (UPGP and UZGP) 203-242 5.1. Economic Analysis of the Project (UPGP) 5.2. Key Findings 5.3. Trend Analysis of Grants, Revenues and Investments in Program and Control Areas 5.4. Assessment on Changes in Resource Base of UP and UZP 5.5. Assessment on Changes in the Pattern of Utilization of Funds 5.6. Impact on how investments benefit the communities, including poor households and gender 5.7. Estimate the impact on local fiscal space, fixed capital formation and economic return 5.8. Assessing the Changes in Financial Management Practices 5.9. Conclusion PART VI - Assessment of Overall Effectiveness, Efficiency, Sustainability, Relevance, and Impact 243-254 Part VII - Gender Analysis 255-265 PART VIII - Self-Assessment of LGD Policy makers and Project Management of the UPGP and UZGP 266-271 PART IX - Lessons Learned and Good Practices 272-275 PART X - Recommendations 276-278 Annex: Questionnaire 279-335 Interview Related Note (Write Specifically) 336-523
5 Abbreviations and Acronyms
ADP Annual Development Programme AP Annual Plan CSO Civil Society Organization DANIDA Danish International Development Agency DC Deputy Commissioner DDLG Deputy Director, Local Government DF District Facilitator EPBG Extended Performance Based Grant EU European Union FY Financial Year FYP Five-Year Plan GoB Government of Bangladesh LD Line Department (Deconcentrated department at UZP level) LG Local Government LGD Local Government Division LGI Local Government Institution LG Local Government
LM Line Ministry MDG Millennium Development Goal NILG National Institute of Local Government NGO Non-government Organization OSR Own-source revenue
RTI Right to Information SC Standing Committee SDC Swiss Agency for Development and Cooperation UFF Upazila Fiscal Facility UNCDF United Nations Capital Development Fund UNDP United Nations Development Programme UNDAF United Nations Development Assistance Framework UNO Upazila Nirbahi Officer UP Union Parishad UPGP Union Parishad Governance Project UZP Upazila Parishad UZGP Upazila Parishad Governance Project
6 Map of the UZGP and UPGP Areas
7 Executive Summary
Bangladesh is going through a transition process of democratisation. Various policy documents including Five-Year Plans (6th and 7th) envision a stronger, effective, participative and accountable local government system as the basis for democratic transformation and consolidation.
Policy regimes and plan documents over and over again emphasized the need for making Local Government Institutions (LGIs) participative, accountable, inclusive, gender-sensitive and responsive to the needs of citizens in general and the disadvantaged groups, in particular.
In view of the context, the Upazila Governance Project (UZGP) intends to build Upazilas as active, bringing all service providers at Upazila level under the accountability framework of Upazila Parishad (UZP).
The project also aims at transforming Upazilas into a corporate local government body by developing and strengthening a mechanism of participatory, democratic and accountable governance process, as envisioned in the Local Government (UZP) Act.
The Union Parishad Governance Project (UPGP) on the other hand has been piloting innovations to improve the functional and institutional capacity of Union Parishads. The project strives for enhancing democratic accountability of Union Parisha ds and increasing citizens’ involvement in order to achieve effective, efficient, accountable and responsive delivery of local services. The initiatives also included a significant gender mainstreaming effort, looking at local women leadership empowerment, participation and voice.
Both UZGP and UPGP aimed to strengthen the national capacity for effective policy review, monitoring, and capacity development of local government institutions (LGIs) to enhance the quality and process of Local Governance.
The broad goals and objectives of the both projects are very much in line with the existing policy regimes, perspective plans and broader long- term visions of the government.
This study attempts to make an impact evaluation of the projects using both qualitative and quantitative methods, including econometrics techniques. Broadly the study attempts to assess the institutional processes, intervention measures and the impact, value for money, and sustainable improvement in livelihood of citizens at Union and Upazila levels.
A comprehensive sampling has been done to gather data and information from the UP and Upazila levels. A total of 3103 households (1541 from project and 1562 from control areas) have been taken as sample respondents of the study.
A total of 220 UPs and 80 UZPs have been covered under the survey representing both the project and the control areas. Institutional surveys based on a questionnaire were conducted on these Upazila and Union Parishads.
8 Besides, an exit- point survey on 720 UP service seekers and 720 Upazila Parishad service seekers was also conducted to assess their opinions about the quality of services and their satisfaction with the service delivery process.
The study also used the qualitative techniques of data collection from the sampled UPs and Upazilas. The study relied on elected representatives and appointed officials of Upazila and UP as Key Informant Interviews (KIIs).
Focus Group Discussions (FGDs) were organized at two different levels- at the institutional level with elected UP and Upazila representatives and appointed officials and also at the community level with community members Besides, a number of case studies were conducted both at the Upazila and UP levels. Finally, the study team members reviewed available secondary literature relevant to focus of the study vis-à-vis the two projects.
It reconfirms that grassroots based governments have significant potential to institutionalise democracy, participatory processes and pro-poor and gender sensitive development.
Capacity building and technical assistance support have noticeably contributed to accountability, responsiveness and social engagement of LGIs. Most importantly LGIs are gradually being recognised for their “functional credibility and institutional trust”.
The strong engagement of communities through the Ward Shava has deepened the pro-poor focus in local investment planning and local revenue mobilisation also emerged as a social accountability platform.
There are evidences that the creation of Women Development Forum (WDF) has promoted gender responsive planning, budgeting, strengthened women’s voice and leadership in decision making and addressed such issues as child marriage, sexual harassment, violence against women and children.
Both the projects with effective use of performance grants have improved transparency, accountability and participation of LGIs. There are evidences of improved pro-poor, gender- friendly planning and MDG- focused service delivery in pilot areas.
Furthermore, the grant system has improved women’s participation and capacity building support to female leaders and LGI’s own resource mobilisation processes.
The projects are found to be complementary in many ways. The collaboration between UZGP and UPGP facilitated strengthening of service delivery in the UPs.
A greater number of households (78%) are visiting the project UPs than it was in the baseline (72%). Household awareness about services at both UP (85%) and UZP (72%) has increased. Quality of services provided by line departments has seen little improvement from ‘poor’ to ‘average’ in the end-line compared to the baseline.
9 A number of projects sponsored by UZGP have ultimately been implemented at the UP level. Integration and synergies were achieved in planning and supporting various service delivery investments across the UPs. Furthermore, leveraging of partnerships among LGSP II and UPGP and UZGP has widened and deepened the impact of development interventions.
The projects are found to be complementary in many ways. Collaboration between UZGP and UPGP facilitated strengthening of service delivery in the UPs.
From an economic perspective, the projects present a positive scenario l. The estimated ‘benefit-cost’ ratio of UPGP intervention is 8.1 while for UZGP intervention the BCR is 18.5. Given the large BCR values (it is suggested that an intervention with more than 1 BCR value is cost effective), it may be concluded that the cost effectiveness of both interventions have been satisfactory.
Furthermore, a marginal increase of 2.6% in the Gross Fixed Capital formation (GFC) has been noticed in the treatment UZP as against the control UZPs. The UP level revenue growth has also been impressive.
Average revenue growth in project UPs is 26.8 % compared to average revenue growth of 18.0 % for the control UPs. The 8.8 percentage point difference in revenue growth may be attributed to the interventions in the project areas.
The head- count poverty rates in “project” and “control” areas between baseline (2009) and the end-line (2016) has been estimated. It was observed that poverty situation has improved for the project household by about 5 percentage points. The corresponding change is half for the control households
The broad analysis of the projects reveals some important features. There are clear differences between treatment Unions and Upazilas in terms of: a. revenue mobilisation, resource generation, and expenditure/investment sector preferences. b. The treatment Unions and Upazilas have high preference to projects/programmes linked with the national policy setting like MDGs.
Results in terms of both qualitative and quantitative indicators show that the projects have brought a relative positive change in treatment areas by giving better services, connecting the people with the local government system.
The income and expenditure patterns have improved-- which significantly contributed to reduction in the local level income disparity and thereby improved the poverty scenario in the project areas.
The local level capital is being formed and the overwhelming dependence on grants is being shifted by the new revenue generation process. Based on the measurable benefits, the increased income, benefit-cost analysis show a high level of net present value and a high
10 benefit-cost ratio suggesting worthiness of the project(BCR is 18.5 for UZGP while for UPGP, BCR is 8.1).
Both at the Union and Upazila levels, the projects have brought about changes in managing fiscal resources, in enhancing the service delivery mechanism, enrolling the local people so that they can reap the benefit directly, and forming local level capital for sustainability.
The two projects have contributed to up-scaling, institutionalising and sustaining local governance mechanism on strengthened transparency, accountability and participation (standing committees, Ward Shava, open budget meetings, citizen charter).
The M&E framework strategy and vertical division of work among DLGs and DDLGs have been established and implemented according to plan, despite changes and vacancies in the project staff. The M&E data are produced being assisted by the monitoring tool kit developed under the projects.
There are specific project interventions of activating and strengthening the Ward Shava. The UPs are trained and capacitated to have their respective Citizen Charters.
A significant up-scaling of Ward Shavas has taken place in terms of holding Ward Shavas as per the UP Act 2009 (end-line: 66% vs baseline: 33%) and attendance in Ward Shavas by 5% (endline:98% vs baseline: 8%). Project UPs demonstrated greater participation of female, poor and disadvantaged people in WS compared to those in control UPs.
The concept of citizen charter has been immensely popularised and institutionalised. At the UP level, 82.5% of project and 75% of control UPs informed having citizen charter while in the baseline 58% of project and 49% of control UPs affirmed the presence of citizen charter.
Under the UZGP, use of citizen charter in project UZPs (90%) is also greater than that in control areas (60%) though the degree of citizen awareness about such charter is still extremely low in both project (8%) and control (5.2%) Upazilas. In the project areas, the UPGP has been helping the UPs in developing a MIS system. The UP Act Section 78 has also been addressed through installing a provision for dissemination of information, especially, on open budgets.
The projects have trained a large number of rural LGIs and community level leaders and women. Such training, orientation programmes, peer learning and overseas exposure visits had direct and indirect implications on their mindset, attitude and level of skills. They are more aware of their “political, administrative and social role” as public representatives.
During the community based FGDs members of the community were found to have noted that elected representatives of recent years are “socially more sensitive” and more “approachable”. The community members seem to have now relatively a higher level of “institutional trust” in LGIs, especially UP. Participation in various UP organised meetings (Ward Shava, open budget meeting) by the community people has significantly increased.
11 For instance, on an average, the percentage of people ever attending any Ward Shava in the project areas has increased to 17.2% in end-line from 8% in the baseline. More importantly, community people now feel prompted to voice their opinion in these meetings about local development. Such opinions later are reflected in the UP plans more frequently in project areas than the plans in control areas.
The most important lessons that can be drawn from both projects are: a. Community engagement is possible and it ensures quality of the implementation of the projects; b. Voices of marginalised people can be channeled through the Ward Shava; c. Open budget has significantly contributed towards “responsive attitude” and “transparency” of the LGI officials; d. Line agency officials have become more aware and also responsive to problems and issues of the grassroots on demand; e. WDF has significantly contributed to creating a platform for women leaders that has raised social awareness among them on the one hand and made them more politically conscious and empowered on the other; f. UZPs and UPs are gradually emerging as effective bodies for coordination of development intervention; contributing towards more awareness and participation.
The training programmes have considerably boosted the level of confidence and functional skills of the UP/UZP members, especially female Vice Chairs, female members of UP who appear to be more active in decision making and supported by the emerging expanding role of the WDFs.
Through training and workshops, both UPGP and UZGP appear to have been able to infuse some democratic values among the Line Department officials and the field evidence suggests they are gradually acknowledging the political reality and accepting authority of political executives.
Both projects have piloted an enhanced performance-based grant system, which has become important catalyst for the achievement of project objectives within local governance, pro-poor development planning, MDG service delivery, and increased own-source revenues (OSR) mobilisation. The performance elements are highly appreciated by LGIs and have instilled a stronger awareness on good governance and a sense of competition.
The projects have been able to generate and facilitate broad policy discussions across the country on the future directions of local government institutional issues. Through its action research and review of policies and legal framework, the projects have been effective in initiating amendments to the local government regulatory framework (legal instruments, guidelines).
12 In broad terms the projects have contributed towards the development of institutionalising and sustaining local governance through strengthened transparency, accountability and participation practices through standing committees, WDF, Ward Shavas, open budget meetings, citizen charter etc.
At the policy level, based on the outcome of project results, the LGD has been able to successfully convince the Government of creating additional staff positions like UP Accountant- cum Computer Assistant to enhance the operational and managerial efficiency of the UPs.
Policy initiatives have been taken to transfer funds of the Line Ministries under the custody of UZPs to ensure a higher degree of institutional accountability of the LM staff to the UZPs. However, it is yet to be operationalised.
LGIs both at Union and Upazila levels face a broad range of institutional, functional and capacity constraints, strained relationship with LD officials and absence of detailed and clearly spelled out rules. At the functional and institutional level there is still a low level of coordination between Upazila Parishads and the LDs which have strong implications on the development planning, management and ensuring service delivery.
Lessons Learned • Community engagement supports better public service delivery and governance at the Upazila and Union Parishads. • Performance Based Grants have stimulated improvement of UP and Upazila governance and created a healthy competition among the Upazilas and UPs. • Scope for democratic space and downward accountability has been accelerated through the induction of Ward Shava, Open budget and proactive disclosure of information. • Sustained focus on social sector projects has helped gender mainstreaming and contribute to developing local women leadership • Role of standing committees can reinforce public accountability of local governments and compliance of citizens. • Capacity building and changing mind- set of Upazila and UP functionaries enhance Own Source Revenue mobilisation • Web-based MIS system improves operational capacity, dynamism and oversight mechanism. • Regular monitoring visit to the Upazilas and UPs by the controlling authorities promotes financial transparency, accountability and compliance with the rules. • Model Upazila and UP Plans and Budget books encourage other Upazilas and UPs to make their plans effectively.
13 • Co-financing has a significant potential provided adequate legal framework, technical support as well as devising an appropriate strategy to generate trust of potential partners are there. • By stimulating the demand side the fullest potentials of the initiatives for ensuring transparent and accountable governance can be achieved.
Recommendations a) Training centres to overcome capacity constraints of local governments should be established in districts and Upazilas. Regional training hubs are also needed to sustain and institutionalise capacity building; b) Effective Fiscal Decentralisation for LGIs requires a local government finance policy framework where finance should follow functions. Local government budget should be aligned with national budget for strengthening the LGIs; c) “Basket funding” approach should be adopted by LGD which will help avoid overlapping and duplication of interventions at the local level; d) Delay in receiving EPBG hampers timely implementation of schemes. Therefore, EPBG should be transferred in time; e) Study findings reveal that all stakeholders are in favor of continuing the present audit system. Therefore, for ensuring transparency and accountability, the present audit system should continue. The audit period should be a bit longer and be done by experienced personnel/firms; f) In order to stimulate co-financing, appropriate legal frameworks are to be enacted detailing the modalities of partnership/engagement among the potential local partners; g) Performance grant system has produced outstanding dividends, so the system should be further reviewed, streamlined and continued in future interventions; h) Further attention should be given to effective use of performance grants; i) Direct transfer of funds to the Upazila and UPs was found to be effective and this should continue; j) UFF grant should be strengthened for co-financing and ownership building in local development; k) Experiences of UPGP and UZGP have strongly provided solid evidence of institutional achievements. Both projects should continue at least for one more phase and undertake few more innovative approaches to good governance and consolidate the tested tools with new interventions and experimentations; l) In line with the policy framework of the 7th Five- Year Plan, a uniform legal framework needs to be enacted and implemented;
14 m) Gradation of Upazila Parishad needs to be set up based on the number of voters and other local development indicators; n) Mechanism should be developed to address interference of the external actors into the LGI development planning and management; o) In order to strengthen the planning capacity, a planning cell should be formed at Upazila Parishad for formulating short, mid and long- term Upazila and inter-UP regional plans. A system is to be designed to develop a synergy between the Upazilas and other LGIs; p) Additional technical, financial and institutional support should be provided to LGIs with a view to making the Ward Shava more participative, effective and inclusive; q) Standing/Upazila committees have immense potential to democratise the LGIs, and therefore, it is important that these are activated properly to ensure internal accountability, local service delivery, responsive and integrated planning of LGIs; r) Participation of citizens in Ward Shavas and Open Budget Meeting is limited and women’s participation was found to be less than men. Therefore, more emphasis is to be given to encourage and motivate citizens in general and women in particular to participate in those fora s) Monitoring, Inspection and Evaluation system for LG Division should be strengthened. Monitoring of LGIs by DDLGs and other relevant agencies should be ensured. Coordination meetings between DDLGs and the LGD need to be organized on a regular basis for feedback and follow-up measures should be undertaken by the MIE wing of the LGD. The manpower and other related capacities of the MIE wing need to be enhanced to perform its monitoring role; t) Coordination meeting at UP should be conducted in the presence of UP members and UP based line agency field extension workers in order to make the local level services more responsive to community needs;
More care and emphasis be given to make the female UP members engaged, enthusiastic and participative in the UP affairs: Intensive training/orientation for the male members to support women’s participation in decision making needs to be organised; In order to enhance women empowerment capacity building initiatives should ensure increased participation of women.
15 Foreword
The Impact Assessment Study of the UPGP and UZGP took place between October 2016 and March 2017 and was conducted by a team of nine Core Consultants including Local Government Expert, Performance Grant Expert, Economist, Gender Expert, Statistician, Communication Expert and Data Analyst, eight Field Supervisors and forty two data collectors. The Techniques of data collection methods and tools comprised of pre-testing, survey, KII, FGDs, Exit Meeting and workshops. The data collection was carried out in twenty one districts and field level data collection was conducted in six weeks.
The two projects are implemented in accordance with separate project documents. They are, however, mutually interlinked both in terms of approach, intervention areas, technical assistance and management. Hence, the report integrates end-line impact assessments where interventions are jointly managed by the two projects, while interventions unique to each of the projects are assessed in separate sections.
The impact assessment team would like to thank for the warm hospitality and open discussions during all meetings and for sharing of valuable experiences in a structured manner at the inception workshop. The assessment team has done its best to reflect the views expressed during discussions with project stakeholders, although findings and recommendations in this report ultimately represent the assessments of the team and are subject to further review and discussions among project stakeholders.
The assignment would not be accomplished without substantial logistical support from the GoB officials, UNDP and UNCDF staff. The team would like to extend a special thanks to all concerned for their all- out institutional support to the arrangement of meetings, workshops, transportation and accommodation.
Salahuddin M. Aminuzzaman
16
PART - I OVERVIW OF PROJECT CONTEXT
1.1 Introduction Bangladesh is going through a transition process of democratisation. Multiple actors and factors have been emerging in the process. New processes are also emerging to strengthen democratic practices both at the national and local levels. Various policy documents including the most recent Five-Year Plan envision a stronger, effective, participative and accountable local government system as the basis for democratic transformation and consolidation. The Seventh Five- Year Plan has also recognised the need for strengthening local governments as one of the 11 priority sectors of intervention for better governance outcomes. Policy documents also emphasized the need for activating local government institutions based on a well-defined legal framework that assigns responsibilities along with commensurate financial autonomy. The local government system in Bangladesh has evolved within a three-tier framework - Union, Upazila (sub-district) and District. The institution at the primary tier i.e. the Union Parishad (UP) has had the most robust presence by virtue of institutional continuity as an elected body. The body at the secondary level i.e. the Upazila Parishad (UZP) has a much smaller history as an elected body while an elected body at the apex level i.e. district, is yet to appear. At present, Local Government Institutions (LGIs) of Bangladesh include 11 City Corporations, 313 Pourashavas (Municipalities), 61 Zila Parishads (Districts), 487 Upazila Parishads (sub-districts) and 4545 Union Parishads (rural local governments). Besides, there are three Hill District Councils, an additional system in force for the Chittagong Hill Tracts (Bandarban, Rangamati and Khagrachari districts). Alongside the statutory LGIs, these District Councils are legally empowered to exercise customary laws as well as act as the Regional Council. These are currently being run by nominated persons rather than by elected representatives. The LGIs in Bangladesh are served by nearly 77,000 elected representatives and 75,000 support staff. Since inception, these local government institutions were given responsibilities for the maintenance of law and order, infrastructure development, promotion of health, education, and some other basic social services. But the Constitution provides a wider agenda. The Article 59 narrates the overall functions of LGI as (a) “administration and the work of public officers; (b) the maintenance of public order and (c) the preparation and implementation of plans relating to public services and economic development”.
17 1.2 Relevance of Project interventions and Government of Bangladesh Policy framework Constitutional Commitment: The Constitution of Bangladesh describes the local government system and the Article 59(2) narrates the overall functions of LGI as (a) “administration and the work of public officers; (b) the maintenance of public order and (c) the preparation and implementation of plans relating to public services and economic development”. In addition, there are a number of supplementary Acts and Rules regulating different tiers of the local government system in Bangladesh.1 The local government has been repeatedly identified as a key strategic sector for improving governance and development in Bangladesh. The election manifesto of the Awami League (AL), the present ruling party, has expressed strong political commitment to establish an effective local government system in the country. The AL manifesto stressed that "every union will be made the headquarters for the development and administration of the area and be developed as a planned rural township."
Seventh Five- Year Plan In line with a strong electoral and political mandate, the Government of Bangladesh has formulated the 7th Five- Year Plan. The Government of Bangladesh places special emphasis on strengthening the local government in order to bring service delivery closer to people and also to make sure local people's preferences are well reflected in the planning process12. At the policy level, it is agreed that a devolved and effective local government system is crucial for making development agenda pro-poor, while widening participation in decision making and ensuring that resources are directed to where they are most needed through increased allocative efficiency. The 7th Five Year Plan has acknowledged that the reflection of good governance should be manifested by “better service delivery” system. The Plan, therefore, has strongly emphasized the need and importance of better service for rural and urban areas through the local government system. The Plan noted the weakness in public administrative capacity and the absence of strong local governments have limited the implementation of government policies. The focuses on strengthening public administration and local governments have been considered “a key priority” for the Seventh Plan. It specifically highlighted the need for improving efficiency and service delivery of local government through a series of structural reform interventions including “institutional and process engineering”3.
1 Local Government (City Corporation) (Amended) Act, 2011; Local Government (City Corporation) Election Rules, 2008; Zila Parishad Act, 2000; Upazila Parishad (Amended) Act, 2011; Upazila Parishad (Service) Rules, 2010; Upazila Parishad (Program Implementation) Rules, 2010; Local Government (Paurashava)(Amended) Act, 2010; Local Government (Paurashava) Election Rules, 2008; Local Government (Union Parishad) Act, 2009; Union Parishad (Development Planning) Rules, 2013; Union Parishad (Tax Schedule) Rules, 2012; Union Parishad (Property) Rules, 2012; Union Parishad (Agreement) Rules, 2012; Union Parishad (Accounts & Audit) Rules, 2012; Village Court Act, 2006. 2 Government of the People’s Republic of Bangladesh, Seventh Five Year Plan (SFYP) 2016-2020 Accelerating Growth, Empowering Citizens, General Economics Division (GED), Planning Commission, 2015, p. Xliv., 421, 422 3 Seventh Five Year Plan (SFYP) , p.123
18 It also identifies the local government as one of the four critical areas for intervention for good governance. The SFYP noted that at the process level LGIs should be: i. strengthened to engage in rural economic activities (in addition to conventional development activities. ii. capable of dealing with climate change, disaster management and planning, infrastructural management; iii. equipped with a results-based M&E system; and iv. capable improving financial management of local government bodies. Furthermore, at the policy level, the SFYP emphasized the need for introducing a Local Government Legal Framework; designing a comprehensive tax sharing formula between national and local governments; and evolving a strong management framework including creation of a ‘Local Government Service’. More specifically the SFYP put emphasis on: i. building the capacity of local governments through the assignment of appropriate officials, technical assistance and training programmes; ii. developing Planning and budgeting capacities at the local level to help design and implement local level programmes; iii. fostering initiatives to provide technical assistance to link local level plan to the national medium to long- term planning.
Vision 2021 According to Vision 2021, local government will be given due importance with a view to effecting a radical change in the political system. Vision 2021 envisages devolution of power, functions, and fiscal authority to local government in accordance with Constitutional provisions44. The Vision 2021 further noted that “the potential of local government bodies, particularly the Union Parishad, to coordinate a streamlined institutional strategy needs to be actively explored”. In addition, “establish powerful autonomous local government bodies for coordinating public and private development initiatives”. Bangladesh has strong Constitutional and political commitment, elaborate policy framework on Local government and governance – which have set a solid structural foundation for the Government of Bangladesh to attain the goals and aspiration of governance.
1.3 UZGP and UPGP at a glance – objectives and result framework The UZGP intends to build Upazilas as active and vibrant LG units, bringing all service providers at Upazila level under the accountability framework of UZP and creating a mechanism of participatory, democratic and accountable body corporate as envisioned in the Local Government (UZP) Act. The activities that are being carried out under the first output of the project aim at building the capacity of the UZP mainly to transform it into a fully functional institution and to promote democracy and empower people, in particular the women representatives to participate in the policy debate of the UZP. The capacity building efforts include making the UZP functionaries
4 General Economics Division, Planning Commission., Government of the People’s Republic of Bangladesh, Perspective Plan of Bangladesh: 2010-2021 – Making Vision 2021 Reality, April 2012. P102
19 understand their basic roles and functions. This is contributing to enhancement of their skills to perform within the provisions of the existing legal framework and fulfilling expectations of both their male and female constituencies. The second output intends to create a simple and viable planning and budgeting system under the existing legal framework and support improvements to the management and coordination of the infrastructure support and local services assigned to all the Upazila Parishads of Bangladesh under capacity development support. The project will ensure the preparation of five- year plans for all Upazilas following the guidelines prepared by LGD. The project under this output also provided support to an MDG oriented participatory planning and budgetary framework for 14 UZPs of Bangladesh. This will include a fiscal facility intended to support both development planning and actual delivery of gender sensitive and inclusive local services and infrastructure to a selected number of UZPs spread over all the seven divisions of the country. It also provides support to the Upazila Parishads as a means to improve basic service delivery within the areas assigned to them with an aim to make a difference in development and contribute to achievements of the MDGs.
The third output is common to both the projects. It includes support to the development of national policies and systems including rules which will enable implementation of the Upazila Parishad Act. The project provides assistance to the government to prepare and operationalise manuals including those for planning, human resources and administration. This output targets the capacity for policy development and national systems supporting local governance comprising both the LG training institutions, technical support and monitoring by the DLG, and DDLGs at divisional and district levels and relevant sectoral entities. The UPGP project has been piloting innovations to improve a. functional and institutional capacity, b. democratic accountability of Union Parishads and, c. to increase citizen involvement in order to achieve effective, efficient and accountable delivery of pro-poor infrastructures and services. The initiatives include a significant gender mainstreaming effort, looking at local women leadership empowerment, participation and voice. The three outputs of the UPGP projects together intend to achieve a comprehensive development outcome by maximising effectiveness of UPs as the lowest tier of democratically elected service providers. The output addresses two dimensions of improving democratic accountability at UP level. The first dimension is the functional dimension of UP affairs with a focus on ensuring that elected officials of UP fulfill statutory provisions and functions more effectively with regard to their stipulated roles, responsibilities and obligations. The project is carrying out activities to build the capacity of UP Chairpersons and Ward Members including Women Members to implement new responsibilities, identifying and removing bottlenecks and ensuring that they are supported with the right skills and capacities in line with the provisions of powers, functions and responsibilities stipulated in the UP Act 2009.
20 The second dimension is that of equitable and inclusive engagement between citizens and the UP and ultimately deepening values of local democracy as well as ensuring pro-poor service delivery. Activities that are being implemented under this output include activation of the Ward Shavas, strengthening of the Standing Committees, which provide a forum for citizens to represent specific interest groups in areas such as health, education and agriculture as well as the interests of women and poor members of the community, and support to Women‘s Development Forum. The activities being carried out under the second output intends to empower 564 UPs in seven districts to exercise their mandate in planning and delivery of services that contribute more effectively to the achievement of MDGs through strategic local development planning, equitable and improved financial management and local revenue mobilisation. The performance-based grants, being provided from the project, are instead being based on a fiscal―topping up of the exis ng GoB annual block grants (supported by the LGSP II) to pilot promotion of performance improvements in specific core areas and provide additional funding to the well-performing UPs within the geographical coverage of the programme. The grant will focus on targeted areas of UP performance within cross-sectoral performance areas (as the UP grants are not sector specific) such as: development planning, accountability, project implementation capacity, own source revenues, poverty targeting/equity (e.g. the extent to which the development plans target the poor),gender and (perhaps) environment/climate change adaptation. Another important area is to enhance UP own revenue mobilisation. The third output targets the capacity for policy development and national systems supporting local governance comprising both the LG training institutions, technical support and monitoring by the DLG and DDLGs at divisional and district levels and relevant sectoral entities. This output also attempts to create continuous citizen-state collaboration, including the engagement of civil society organizations (CSOs), experts, politicians and elected and non-elected officers of local government. Under this output, the projects also support the LGD to conduct high profile policy research with an aim to develop policy framework and institutions supporting local governance.
21 1.1.1 Objectives and Result framework of UPGP and UZGP Objectives and results UPGP UZGP
Overall objective Capacities of local governments Government institutions at the and other stakeholders are national and sub-national levels strengthened to foster are able to more effectively carry participatory local development out their mandates, including service delivery for the MDGs delivery of public services, in a more accountable, transparent and inclusive manner
Specific objective Piloting and evaluation of Capacities of local governments innovations to improve the and other stakeholders are functional and institutional strengthened to foster capacity, as well as the democratic participatory local development accountability of Union Parishads, services for the MDGs thereby increasing citizen involvement leading to more effective, efficient and accountable delivery of pro-poor infrastructure and services
1. Strengthened democratic 2. Strengthened Upazila accountability of the Union Parishads as more functional,
Parishads through Citizen transparent and accountable Results Engagement 1. Innovations in Pro-Poor and 2. Strengthened Planning and MDG-oriented Planning and Budgetary system at UZP with Financing of Service Delivery MDG orientation and pro- by Union Parishads. poor service delivery mechanism Share result of both 2. Strengthened national capacity for effective policy review, projects monitoring, Lesson learning and capacity development of LGIs for enhanced Local Governance
2.4. Background of the assessment The UZGP and the UPGP are part of the programmatic support from UNDP and UNCDF to the Government of Bangladesh for Local Governance strengthening and reforms. The support is in line with the United Nations Development Assistance Framework (UNDAF) outcome level, which states that by 2016, all Bangladeshis including vulnerable groups are better represented
22 and participate more in democratic processes and civil service and local government institutions are more responsive and better able to deliver public services.
Both are the Nationally Implemented Projects (NIMs) implemented by the Local Government Division. The UNDP and UNCDF are the technical partners and implementing UN agencies. The projects are financially supported by the European Union (EU), the Swiss Agency for Development and Cooperation (SDC) and DANIDA. The UZGP and UPGP are interlinked through shared output of policy research, have shared staff and draw on each other’s work in the field. Both the projects have heavy focus on capacity building and institutional strengthening of LGIs especially for fiscal decentralisation.
1.5 Reflections from other studies having bearing upon UPGP and UZGP A number of relevant and important research on local government and local governance have been sponsored by the UPGP and UZGP. These research / studies have been conducted by nationally and internationally reputed scholars/ experts and consultants. The following sections provide a snapshot summary of the researches. Mobasser Monem (2016) in his research entitled, “An action research on the process and quality of budgeting and planning of Upazila Parishads” examines the quality and the processes of planning and budgeting of Upazilas in Bangladesh. The specific objective of the study is to examine the extent to which the existing legislations are being followed in preparing plans and budgets of UZPs. The study observed that Upazila budgets show following limitations and or deviations: a. in most cases the budget is prepared without consulting the government guidelines and thus, the documents seemed to have lacked the consistency, coherence and logical sequence; b. The budget focuses on multi-sectoral schemes but still many UZPs suffer from traditional mindsets of resource management; c. Consequently the plan documents do not generally found to be gender sensitive or truly inclusive, d. the plan and budget documents hardly show any linkage between the FYPs and annual plans; e. the vertical and horizontal integration of the Upazila plan with other tiers was found to be almost non-existent; f. There was no effective mechanism in place for soliciting citizen’s opinions or feedback on the draft plan; g. interestingly, all sampled Upazilas prepared surplus budget; h. Upazilas were generally found to have used the budget formats envisaged in the law; i. most Upazilas violated the important clauses of the guidelines for the utilization of revenue and development funds. Monem thus offers some broader and specific recommendations – some of those specific recommendations include: developing a comprehensive guideline on the plan; b. a system of sharing of budget of LGIs with other tiers of local government; c. an open budget meeting should be made mandatory for Upazilas; d. stern administrative measures must be taken against the Upazilas for non-compliance of standard practices, e. a strong central monitoring to be developed to ensure the linkage between the five- year and annual plans; f. capacity be
23 developed on planning and budgeting; g. setting up of a separate and dedicated plan and budget wing at the Upazila . Nazmuzzaman Bhuian (2015) in his study entitled “Review of Local Government Laws of Bangladesh- Towards Prospect of a Local Government Uniform Framework Legislation” attempts to make a comprehensive assessment of the laws that govern all LGIs in Bangladesh. He notices some anomalies and legal concerns that affect role and institutional performance of LGIs. Based on arguments and observation drawn from the legal review, he strongly suggests that instead of piece-meal legislation for each unit, “both rural and urban tiers of Local Government may be regulated by a single law”. The research suggests, among others, that “LGIs should follow uniform and complementary rules as regards finance, audit and procurement under “single framework legislation”. Nizam Ahmed and others (2014) observed in their studies that on a comparative scale LGIs in Bangladesh are increasingly looking to play a meaningful role both as a platform of deepening democracy and as an improved service delivery mechanism. After a thorough institutional analysis with five dimension perspectives (effectiveness, the dynamics of local leadership, local control over administration and service delivery, fiscal autonomy, and participation and accountability), the researcher notes that : a. assignment of functions and expenditure responsibilities to LG institutions is unclear due to the fragmented legal framework; b. The ability of the local political leadership to steer their local government jurisdiction is considerably limited due to limited local control over the local government’s core administration team and other structural factors; c. LG institutions are under-staffed and have little or no control over local-level staffing. In the absence of meaningful control over local human resources, it is almost impossible for local political leaders to influence the quality of local public services; d. Local governments are given limited revenue discretion, and they do not effectively collect the revenues assigned to them. The intergovernmental transfer system generally provides inadequate and unpredictable funding, e. Limited opportunities for meaningful participation and accountability exist in the local government system. While many committees exist (especially in rural LGs) to encourage participation, many of these committees are “paper- based” and lack power over the entities they monitor. With such findings, the researcher recognised the need for: a. unified local government framework; b. there is a strong need to strengthen the deconcentration to the Upazila to check coordination failure and improve service delivery; c. balancing the functional assignment between LGI executive functions versus LGI monitoring functions; d. necessary reform of the local political system to further strengthen the LGIs; e. ensuring local control over local functionaries; develop a consolidated local government grants system; f. strengthen the local level budgeting, PFM and revenue collection system and administration; g. strengthening local participation and accountability.
24 Jamie Boex (2014) in his study entitled “Decentralization and localization in Bangladesh – the role of Local Government and Local Administration in Ensuring Efficient and Equitable Health and Education Services” has raised some fundamental concerns about local government and local administration as regards service delivery with particular focus to Education and health. The research noted that Bangladesh, as compared to other developing countries, has made considerable progress in achieving Millennium Development Goals including in the realm of health and education. He raised few fundamental questions whether, and if so how reforming some of the vertical relationships in the public sector in Bangladesh could improve public service delivery in Bangladesh. The findings further suggest that Upazila level local governments are well positioned in the long run to become the main platform for localized service delivery in Bangladesh. Upazila jurisdictions are sufficiently large to take advantages of most scale economies in service delivery, while they are sufficiently close to the people for downward responsiveness and it would be possible in a meaningful way. Thus he suggests that Upazila Parishads, in the long run would be in a position to effectively provide health and education services. He, therefore, strongly suggests that Upazila level ought to become the main platform for localised service delivery in Bangladesh. In that case, Union Parishad has an important role to play. He also observed that Upazila Parishads are positioned closer to the people, and therefore have an opportunity to support the localized delivery of public service without having to work through different administrative tiers. He, therefore, recommends: i. a gradual, sequenced approach to decentralization and localization of reforms”; ii. strengthening sectoral deconcentration by improving transparency, discretion and accountability of deconcentrated line ministry officials at the Upazila level. The research reminds that lack of sectoral expertise of the Upazila to deliver health and education services on their own could be a critical concern. However, one possible way to leverage the role of Upazila Parishad is to introduce a formula and performance based Upazila Parishad Human Development Grant. Mahfuz Kabir (2015), in his study Mapping of Fiscal Flow and Local Government Financing in Bangladesh observed that effective decentralization through inclusive and sensitive inter- governmental transfers as well as sensible resource sharing are the key to strengthening LGIs. In Bangladesh, it is widely believed that all the LGI tiers have been suffering from resource deficiency that hinders delivering quality services. In general, financing LGIs have not shown any particular pattern in the recent past years. There are oscillations in direct transfer of funds in all the LGIs in terms of both amount and rate of change. Rural LGIs are seen to experience more oscillation than other type of LGIs. The aggregate direct allocation to LGIs also shows frequent fluctuation of fund transfer. Compared to other tiers of LGI, the UPs get a stable and constantly increasing unconditional block allocation along with a more stable annual change over the years. The allocation to Upazila, however, shows a typically fluctuating pattern. The existing laws provide the areas where the LGIs are sharing resources generated locally by the national government. Revenue
25 coming from the local level is supposed to be generally predictable and sustainable, but there is paucity of data as regards revenue generated from LGIs to the national government. Most of the municipalities, including ones of A-category depend heavily on the development receipts from the government and donor projects to finance their total expenses. No particular pattern could be seen in direct development allocation by the Government. Unlike other tiers of the LGIs, the analysis of fiscal flow from and to Upazilas is quite complicated. This is because there are three types of institutions functioning on a parallel basis at Upazilas, viz. (1) UZP having revenue and development expenditures, (ii) 17 departments of 12 ministries that are transferred to LJZP but their finances are yet to be transferred, and (iii) about 17 additional non- transferred departments operating in Upazilas earn revenue from and spend at Upazilas. The transferred departments have been found to be spending more in non-ADP developmental areas than the revenue spending in most of the Upazilas. Union Parishad, on the other hand, despite many innovations and practices along with legal bindings of performance-based allocation mechanism are still dependent heavily on the government support directly from the national level and other tiers of local government, viz. Upazila. Analysis of the budget data from selected unions reveals that with very few exceptions, government support is an overwhelming majority (ranging from 80 to about 97 per cent) of total revenue in most of the UPs. The researcher suggests few policy reform strategies: a. Undertake analysis of medium- term financing requirements for all LGIs and then include it in the mainstream long- term planning; b. Introduce an index based financing mechanism for LGIs. The index could be either simple or composite, but sub-indices must have the indicators of financial performances, service delivery, fund requirement depending on quantitative analysis of the resource gap to deliver desired services to citizens; figure out the financial transfer to and spending of LGIs by functions and economic classifications. However, in the mid-term, GOB may consider establishing notified area for a reasonable period to assess the financial viability before declaring an area as a municipality. And for the long- term policy intervention the GoB may consider bifurcating the Local Government Division in two – Rural LGI Division and Urban LG Division.
Summary of the research papers Core observations • Non-existence of a unified local government legal framework covering both rural and urban tiers • LGIs do not follow single and unified framework legislation and complementary rules as regards finance, audit and procurement. • UZPs suffer from traditional mindsets of resource management;
26 • Upazila budgets are generally prepared without consulting the government guidelines; • The plan and budget documents of UP and UZP do not have any linkage with the FYPs • The vertical and horizontal integration of the UP and Upazila are non-existent; • Upazilas tend to violate guidelines for the utilization of revenue and development funds. Also, Upazilas tend to prepare surplus budget; • None of the LGI laws in any section outlines the nature of institutional interface between various tiers of LGIs. • Upazila has the potential and ought to become the main platform for localized service delivery in Bangladesh. • Financing LGIs have not shown any consistent particular pattern in the recent past years. UPs get a stable and constantly increasing unconditional block allocation. While Upazila shows a typically fluctuating pattern. Fiscal flow from to and Upazilas is also complicated.
Suggested way- forward • Both rural and urban tiers of Local Government may be regulated by a unified local government legal framework • LGIs should follow uniform and complementary rules as regards finance, audit, and procurement under “single framework legislation”. • Developing a comprehensive guideline for the multi-tier planning; • A system needs to be developed for sharing of budget information of LGIs with other tiers of local government; • An open budget meeting should be made mandatory for all levels of LGIs; • Devise mechanism and process for ensuring local control over local functionaries; • Develop a consolidated local government grants system; • Strengthening sectoral de-concentration by improving transparency, discretion and accountability of deconcentrated line ministries officials at the Upazila level. The LGD has reviewed all such study reports and developed policy papers based on the reports for relevant authorities including the Planning Commission. The project through the active support from the LGD have been able to draw attention of the Planning Commission to some of the major observations of the studies along with the empirically validated models and approaches of programme design, implementation and social monitoring processes have been well accommodated in the SFYP. Furthermore, the projects have been able to add substantive inputs to the mainstream professional and academic literature on local government and governance.
27 1.6 Scope and objectives of the assessment This study attempts to make a rigorous impact evaluation of the both projects using both qualitative and quantitative methods including econometrics techniques. Broadly, the study covers following key areas of post-results: To measure the impact of UPGP and UZGP on the lives of people in the targeted districts To assess intended and unintended results of the project both at beneficiary households level and beyond; To assess which approaches, interventions and activities have proven to be most effective and why; To assess the extent to which the project has achieved value for money, and Make recommendations of what further efforts are required for sustainable improvement of UP and UZP.
1.7 Components of the study With the above general focus of the assessment, specifically the study has following components of the projects under purview:
Component 1: Capacity building, gender and policy advocacy for UZGP and UPGP Assess the impact of backstopping services provided to Upazila Parishads and Union Parishads Examine the improvements and changes accrued from various trainings provided to Upazila Parishad & Union Parishad Chairpersons, Members and Officers Examine improvements achieved in the Upazila Parishad andUnion Parishad Planning through financing of civic engagement Examine improvements in quality and availability of Upazila Parishad andUnion Parishad Plans and the contributions of support to the planning exercise, including mentoring support Examine the impact of support provided to Ward Shavas and Standing Committees in the performance of Union Parishads Assess the status and performance of Women Development Forum at the Upazila and Union Parishad levels
Component 2: Direct fiscal support and performance assessment system of UZGP and UPGP Examine time series data on grants, revenues and investments in a representative sample of local governments, including control units, along with qualitative analysis, to identify the changes in resource base of local governments
28 Estimate the increase in availability of discretionary funds for the target local governments Identify changes in the pattern of utilization of funds, and changes in profile of service delivery by local governments Impact on how investments benefit communities, including poor households and gender profile Estimate the impact on local fiscal space, fixed capital formation and economic return Identify complementarities arising from the provision of these grants in co-financed schemes Impact of climate-change related schemes Examine sustainability of innovations in grant financing including co-financing and local platforms such as trusts Assess impact of performance assessments for grant making on key performance areas of local governments and how the good practices have been replicated in other interventions Assess changes in the financial management practices including the new accounting software, and its impact on management of grants and other resources
Component 3: Strengthened national capacity for effective policy review, monitoring and capacity development of local government institutions This component is a shared output of UZGP and UPGP. The two projects are jointly supporting the development of the policy framework and institutions supporting Local Governance. The component also supports the Policy Advisory Group, undertake research on policy issues, capacity development support to LGD (MIE Wing), DLG (Division Level) and DDLG (District Level) for backstopping and monitoring of local government (UZP and UP); and support R&D for knowledge generation.
29
PART - II EVALUATION METHODOLOGIES
2.1 Survey Methodology of the study 2.1.1 Overview of survey design The evaluation method comprised of both quantitative and qualitative data and information in order to analyse progress, achievements and changes over time. The study adopted a rigorous impact evaluation method based on econometric techniques. Quantitative data were collected through structured questionnaires at household and institutional levels and the qualitative data were gathered through in-depth or intensive Key Informant Interviews (KIIs) including information checklist for Union Parishad and Upazila Parishad activities, Focus Group Discussions (FGDs) and review of documents.
For the quantitative part of the study, a quasi-experimental design with case and control in both baseline and endline was used to measure progress of both the UZGP and the UPGP projects. To obtain an optimum benefit from the study design, the study followed the baseline sampling methodology as much as possible. This is expected to produce an estimate of change with minimum disturbance of sampling error i.e., the standard error of the estimated difference was minimized. Accordingly, two sets of households: “Treatment” group, selected from the UPGP project area and another “control” group outside the project area were selected.
Two independent Baseline Surveys for UPGP and UZGP projects employed two separate samples for UPGP and UZGP interventions and control areas. However, the evaluation survey employed single sample covering both interventions in the same intervention districts. It created a future scope of comparing and linking between Upazilas and unions across areas.
The UPGP proj ect has been implemented in 564 UPs in 65 Upazilas (UZs) in seven districts of seven divisions, where UZGP project has been implemented. The districts are Kishoreganj, Brahmanbaria, Sunamganj, Sirajganj, Khulna, Barguna and Rangpur. Treatment groups were selected from these districts while the control groups were selected from 14 adjacent districts having similar socio-economic features as project districts.
30 2.2 Study area and target population of the study 2.2.1 Study area Study area comprised of seven intervention districts: Kishoreganj, Brahmanbaria, Sunamganj, Sirajganj, Khulna, Barguna and Rangpur, and fourteen control districts: Jamalpur, Netrokana, Comilla, Chandpur, Pabna, Natore, Satkhira, Bagerhat, Barisal, Patuakhali, Dinajpur, Gaibandha, Habiganj and Moulavibazar under seven administrative divisions of Bangladesh.
2.2.2 Target population Following study population was targeted in the evaluation study: • Household beneficiaries: men and women; poor (low income, poor and hardcore poor) and non-poor; and socially marginalized population (fishermen, sweepers, hizras (transgender), poor widows, day- laborers, landless farmers, indigenous etc.) • Elected UP Chairmen, Members-(male and female) and UP Secretaries, members of the Standing Committees • Upazila Chair, Vice- chairs, UNOs, Upazila level officers, Members of Women Development Forum, members of the Standing Committees • Project/program officials: DLGs, DDLGs, District Facilitators • Community level: Selected persons attending Ward Shavas, open budget meetings, members of the community, informed local citizens. • UP and Upazila service seekers
2.2.3 Matching methodologies The major challenge in designing the sample for this evaluation was to strike a balance between different methodologies in baseline surveys for the UPGP and UZGP and hence find a single sample design for the evaluation, which would produce data comparable to those from the baseline studies. Based on the baseline reports and discussion with UPGP & UZGP management some criterion were set to finalise the evaluation methodology- 1. Strike a balance between the two methodologies in two baselines (UPGP and UZGP); 2. At least half of the sample be repeated from (same area of) both baselines; 3. Controls in UZGP baseline to be reviewed and half of them to be selected for endline 4. It is a blended study where data for both UPGP and UZGP to be collected from the same source, wherever it was applicable, e.g., household questionnaire to address both UPGP and UZGP whatever was applicable for that household (either or both) Based on such points, the sample was designed in a way that all the Upazilas of UZGP baseline plus half of the remaining 65-14=51 Upazilas were selected for the endline, i.e., 40 Upazilas in total (61.5% of all under intervention) and 3 UPs from each selected UZs. Finally, a total of 120
31 unions were selected for the endline as against 188 unions in UPGP and 14 unions in UZGP baseline, and 1560 households for both UPGP and UZGP in endline as against 1880 households for UPGP and 560 households for UZGP data in baseline. Further, the matching and selection of control UPs were determined on the basis of some background indicators set for the evaluation survey. Based on availability of data, the indicators selected from the list were as follows- education, income, livestock holding, landholding, membership of production cooperative/group, land type, current practice of agricultural production and livelihood items etc. From the 7+7=14 ‘control districts’ of UPGP and UZGP baselines, seven were selected as control in the evaluation (mentioned in sub section 2.8.2.1), considering the geographical proximity to the intervention districts and other comparability. These are: Jamalpur, Comilla, Pabna, Satkhira, Barisal, Dinajpur and Habiganj.
2.3 Matching results In the evaluation survey same union parishads, covered in the baseline survey, were selected as far as possible. Sample households were selected from different socioeconomic categories, viz., non-poor, low income group and hardcore poor, and data analysis indicates a close similarity among categories between two selected samples of treatment and control areas (see Table 2.8.4 below). The matching and selection of Control UPs were finalised with support and in consultation with UPGP project management team.
Table 2.4: Coverage of sample households from different socioeconomic categories
Endline Type of Area Treatment Control Well- off/non-poor 38.3 33.7 Low income/poor 45.6 45.1 Hard core poor 16.1 21.2 N 1541 1562
2.3.1 Methods of data collection To meet the objectives of the survey, data were collected following two methods: Quantitative data collection methods o Household survey o Upazila Parishad survey for assessing their activities and performances o Union Parishad survey for assessing their activities and performances o Exit point (of service recipient) survey of service seekers at UPs and UZPs
32 Qualitative data collection methods o Key informant interviews with Upazila and UP chairmen/Upazila Vice Chairmen (both male and female), UP members (both male and female)/UP Secretary o Key informant interviews with NBD officials/UNOs/others o Focus group discussion with Upazila Parishad Officials/Elected representatives o Focus group discussion with members of the community o Case studies
Secondary data or review of documents o Reviews of local government programmes as well as monitoring products of the project including progress reports and reports of review meetings o Outputs of the project including manuals, system design works and concept papers o Political and development reports, Economic Sector Reports of the World Bank, thematic sector reports, relevant legislations and national plans, strategy papers of UNDP and o Selected articles published in the print media
2.4 Quantitative survey The household survey was conducted using a pre-tested structured questionnaire through in- depth interview with the target beneficiaries/audiences to collect information on the selected indicators along with their responses to their knowledge, practices and attitudes regarding UPGP and UZGP interventions on a sample basis. Information was collected from the household head or responsible persons following gender ratio (male: female=1:1) of the selected households through face- to- face interviews.
2.4.1 Sample size Sample size for household survey Since the endline study has attempted to compare the estimates of the indicators at baseline with those at endline, the sample size was determined using the well-known statistical formula for the test of a hypothesis of equality of two proportions from baseline and endline (H0:P1=P2 1 Vs. HA:P1P2) as follows (Lwanga and Lemeshow, 1991 ): Sample size for testing of a Hypothesis of equality of two population proportions (For a two- sided test)