ANNEXES TO THE EVALUATION REPORT
EVALUATION OF UNICEF’S CONTRIBUTION TO THE NORMATIVE POLICY FRAMEWORK OF THE SOCIAL CARE SERVICES REFORM 2013-2019 IN ALBANIA
APRIL 2020 Authored by: Camille Massey and Merita Poni, supported by Elayn Sammon.
The findings, interpretations and conclusions expressed in this document are those of the authors and do not necessarily reflect the policies or views of UNICEF. Readers are encouraged to use material from this report for their own publications. As copyright holder, UNICEF in Albania requests due acknowledgement and a copy of the publication.
For further information, please contact: UNICEF Albania Country Office Skenderbej Street, UN House Bld, 3rd floor Tirana, Albania Telephone: +355 4 45 48 400 Email: [email protected]
Attribution: Please cite the work as follows: Massey, C. Poni, M. Sammon, E. (2020) “Evaluation of UNICEF’s Contribution to the Normative Policy Framework of the Social Care Services Reform 2013-2019 in Albania”. UNICEF in Albania ANNEXES TO THE EVALUATION REPORT
EVALUATION OF UNICEF’S CONTRIBUTION TO THE NORMATIVE POLICY FRAMEWORK OF THE SOCIAL CARE SERVICES REFORM 2013-2019 IN ALBANIA
APRIL 2020 4 TABLE OF CONTENTS
Annex 1: Terms of reference for the evaluation...... 6
Annex 2: Terms of reference for the evaluation reference group...... 24
Annex 3: Visual Aids to the Theory of Change...... 28
Annex 4: Extended evaluation matrix...... 36
Annex 5: Ethical Considerations...... 38
Annex 6: Primary data collection tools...... 42
Annex 7: List of data collection sites and interviews/focus groups...... 48
Annex 8: Survey results...... 50
Annex 9: Municipal case studies’ profiles...... 58
Annex 10: Correspondence of evaluation criteria with findings, lessons learned and recommendations...... 64
Annex 11: References...... 66
Annex 12: Information on the evaluators...... 75
UNICEF in Albania ANNEXES TO THE EVALUATION REPORT 5 ANNEX TERMS OF REFERENCE 1 FOR THE EVALUATION Terms of Reference for the International Consultant for the Evaluation of UNICEF’s Contribution to the Normative Policy Framework of the Social Care Services Reform 2013-2019 in Albania
UNICEF IN ALBANIA
UNICEF is the agency of the United Nations mandated to advocate for the protection of children’s rights, to help meet their basic needs and to expand their opportunities to reach their full potential. In Albania, the overarching goal of UNICEF’s engagement with the Government and partners is to achieve, by 2021, the progressive realization of rights of all children in Albania, accompanied by the reduction of equity gaps, based on effective child-focused policies, systematically enforced accountabilities with adequate public resources, and strengthened respect for and protection of children’s rights. Specifically, UNICEF works with the government and other partners towards strengthening governance for equity and social inclusion and ensuring protection and justice for all children.
The current Country Programme Document (2017-2021), as part of the “Delivering as One” United Nations Development Assistance Framework (UNDAF), will contribute to progress towards the achievement of the Sustainable Development Goals in Albania. The country programme is also fully aligned with national priorities of EU integration which includes social cohesion based on a modern educational system, universal and quality health care coverage, expanded employment opportunities, a stronger social protection system, gender equality and social inclusion.
♦♦ UNICEF results framework (2017 – 2021) contributes to the following results specific to children: ♦♦ By 2021, health care policymakers and service providers are equipped with knowledge, guidance, tools and mechanisms to effectively scale up and implement the new MNCH normative framework, focused on the most marginalized. ♦♦ By 2021, education sector policymakers and practitioners at central and local levels are equipped with knowledge, guidance, tools and mechanisms to effectively implement and scale up the reformed, Early Learning and Development Standards-based and equity- sensitive, early learning education framework ♦♦ By 2021, social protection policymakers and practitioners are equipped with knowledge, guidance, tools and mechanisms to effectively implement the reformed social protection policy (combining cash assistance with decentralized care services). ♦♦ By 2021, social and child protection practitioners have increased capacity to assist families in situations of particular vulnerability/at risk of separation to provide care for their children, preventing institutionalization. ♦♦ By 2021, child protection practitioners are empowered with legal and normative frameworks,
6 operational standards and tools to effectively prevent and address situations of child abuse, violence, neglect and exploitation. ♦♦ By 2021, justice sector policymakers, practitioners and independent human rights institutions have the capacity to fully align the Justice for Children normative framework to international standards and to effectively protect the rights of children in conflict and in contact with the law.
BACKGROUND AND CONTEXT
COUNTRY CONTEXT Albania is an upper-middle-income country situated in South-Eastern Europe. Rapid demographic changes have taken place in the past years, with a significant decline in the size of the working- age population due to rapid aging and continuous high levels of emigration. Data from the Institute of Statistics (INSTAT) show the shrinking trend in the total population size, from 2.89 million in 2014, to 2.87 million in 2018. The child population (younger than 19 years) has also fallen, from about 28 per cent (2014) to 25 per cent (2018) of the total1.
The country has a high human development index, ranked as 68th out of 189 countries in 20172. Over the last quarter of a century, living standards in Albania have seen significant improvement, with per-capita Gross Domestic Product (GDP) quadrupling from 1992 to 2017. In 2018, this positive trend in economic growth continued with GDP growth in real terms at 4.3 per cent for the first three-quarters of the year, attributed mainly to Foreign Direct Investment projects and revival of construction, and tourism3.
Yet, despite this economic progress, wealth distribution has not been equal, and vulnerable groups continue to be underserved in the areas of health, education, social care and child protection. Albania has high unemployment rates especially among youth and women4. International organizations estimate the share in “vulnerable” employment (those engaged as unpaid family workers or own-account workers) to represent just under 60 per cent in Albania5 and that the country still remains one of the world’s 35 largest remittance-receiving economies. Albania is one the poorest country in Europe. Latest poverty published data are from Living Standards Measurement Survey (LSMS 2012), which indicate the urbanization of poverty due to internal migratory flows, confirmed that children are most affected by poverty: while 14.3% of the population lived in absolute poverty, the figure for children was 17.4 per cent (living in families with less than USD125 a month), and almost two per cent (1.8% or 15,252 children) were living in families with zero income.
The Government of Albania (GoA) has signed and ratified various international conventions, documents and resolutions related to human, including child rights, and has been successful to have in place a robust national legislation in line with international conventions. However, it lags behind in the implementation of the laws, policies and strategies6.
State’s financial allocations for social sectors—education, health and social protection—make up the largest share of public expenditure as a percentage of GDP; but they are not comparable to international standards for an upper middle-income country and fail to provide the adequate
1. INSTAT, Population Statistics, 2018 2. UNDP, Human Development Indices and Indicators, 2018 Statistical Update 3. International Monetary Fund, European Department, Albania 2017 Article IV Consultation 4. Mainstreaming, Acceleration and Policy Support for Achieving the Sustainable Development Goals in Albania (2018) 5. UNDP (2016) Regional Human Development Report 2016 Progress at Risk: Inequalities and Human Development in Eastern Europe, Turkey and Central Asia. Chapter 2. 6. EU Albania 2018 Report, pg. 78; https://ec.europa.eu/neighbourhood-enlargement/sites/near/files/20180417-albania-report.pdf
UNICEF in Albania ANNEXES TO THE EVALUATION REPORT 7 human, technical and financial resources needed for improved services for children and for access to those services7. Investments in health care, at 2.9 per cent of GDP, have remained relatively constant over the last five years, but substantially less than in other countries in the region. The low- level allocation of public funds in education—3.1 per cent of GDP one of the lowest in the region8. Since 2015, for social protection, social assistance represents only 1.2–1.3 per cent of GDP. The largest financial allocations within social protection scheme cover pensions and unemployment benefits.
The National Social Protection Strategy 2015 - 2020 sets the GoA’s vision for an integrated social protection system, based on universal principles, capable to protect all Albanian people from falling into poverty, to better manage risks and shocks and enhance their capacity throughout their life-cycle. In the past years, the GoA has been leading three major reforms and has made positive achievements to: improve the efficiency and effectiveness of the poverty targeted cash assistance, improve the situation of persons with disability and establish an integrated system of social care services at decentralized level. Yet, the current social protection programmes fall short of addressing in a comprehensive manner the multiple vulnerabilities faced by children and families. Social care is limited to cash transfer schemes, subsidies and tax exemptions, while services are fragmented, and coverage limited. No specific programmes exist to ensure income security for children, though many are covered by programmes targeting poor and vulnerable households.
BACKGROUND ON SOCIAL CARE SERVICES REFORM AND UNICEF’S CONTRIBUTION Starting from 2010, discussions about the necessity to establish an adequate and effective system of social care services emerged at the National Conference for Social Sector Reform, supported by UNICEF. The conference clearly outlined that a proper system of social care systems in Albania was missing. A national mapping identified that existing social care services were provided mainly by NGOs, were fragmented and not harmonised to respond to the needs and being donor driven, totally unsustainable. Further in-depth analysis conducted in 2012-2013, identified the critical bottlenecks of the existing system, listing a) dearth of policy guidance around integrated social protection systems which include both cash and care; b) lack of clarity on what constitutes social services; c) missing clear accountability lines and weak monitoring and inspection; and d) absence of any financial mechanisms to fund social care services.
Following GoA’s decision to undertake a thorough reform to transform social protection system, UNICEF managed to mobilise donors support around such complex reform. In 2012, UNICEF raised about 3.3 million USD from the Swiss Agency for Development and Cooperation (SDC), which matched with UNICEF funds of at least USD 1 million, were meant to support a four-year programme (2012 – 2016) aiming towards the following results:
♦♦ A national comprehensive normative policy framework exists and is operationalized by local government units and service providers empowered to effectively reach and fulfil the social and economic rights of the most marginalized populations, and ♦♦ Situations of extreme marginalization in impoverished and minority (Roma and Egyptian) communities are effectively addressed by local duty-bearers, converting the provisions of the national social care reform into immediate practical improvements in people’s lives.
The UNICEF interventions which aimed to build the new system of social care services were mainly planned under outcome 1 and the financial contribution related to that would be estimated to 1,6 million from SDC and about 250,000 USD from UNICEF. As part of 2012-2016 Government of Albania - UNICEF Programme of Cooperation, which was embedded in the wider Government – United Nations’ Programme “Delivering as One”, UNICEF was mandated by the
7. Mainstreaming, Acceleration and Policy Support for Achieving the Sustainable Development Goals in Albania (2018) 8. INSTAT, Statistical Yearbook, 2018
8 GoA to support the design of an integrated social service system at the decentralized level to fulfil one of the most fundamental rights of Albanian children and families, the right to social protection, care and support.
The new Government in September 2013, renewed the political attention to launch and start working on the Social Care Reform. This was not smooth anyway: the political shift of the Government was associated with new structures and staff, requiring additional time to transit in a more or less normal regime of operation; most importantly, Government initiated a fundamental reform of the country’s territorial and administrative division, aspiring to have a more compact and efficient local governance system ready for the elections of 2015.
In parallel to the social care reform, since 2013, the GoA has been leading two other reforms to: i) improve the efficiency and effectiveness of the poverty targeted cash assistance; ii) improve the situation of persons with disability, including the revision of disability allowance financial support scheme. Recognizant of the comprehensive scale of transformation, GoA is supported by the donors and international organization for financial and technical support in planning and implementing the reforms. World Bank remains the primary support of the cash assistance related reforms; whilst, over the years, the social care reform has been supported by Swiss Agency for Development and Cooperation (SDC) and United Nation (UN) Agencies.
Overall, the SCS reform did not follow a strictly-defined path, rather a long and challenging one. As outlined at the National Social Protection Strategy document, the reform of social care services went through two main stages/phases:
PHASE 1 – YEARS 2013-2016 During 2012-2016, UNICEF worked to put in place the major “building blocks” of the SCS reform in Albania, working to achieve one of the outcomes of the SDC supported programme: A national comprehensive normative policy framework exists and is operationalized by local government units and service providers empowered to effectively reach and fulfil the social and economic rights of the most marginalized populations.
The work done was in line also with UNICEF global and regional strategic goals, and in the belief that any effective social protection system should ensure that both cash entitlements and social services are provided to those most in need. Based on a combination of UNICEF’s advocacy, capacity building efforts, continuously generated evidence and extensive technical assistance offered to the GoA, UNICEF in Albania contributed for the realization of the following, but not limited, components (outputs):
Vision and Planning ♦♦ The development of a long-term vision of the national SCS system, validated by an extensive national consensus-building process and made explicit in the national development agenda. For the first time in Albania, the SCS notion, principles and approach were included in and endorsed by the second National Strategy for Development and Integration (NSDI-II) for 2015-2020; ♦♦ The adoption of the National Social Protection Strategy (NSPS) for 2015-2020, outlining the main elements of the future SCS system. The NSPS is the document that sets the vision for social care services development and provision. It also clarifies the roles of the central and local authorities and sets the frame of cooperation between the public and the non-profit sectors to ensure provision of relevant and efficient social care services. The specific objectives of the SCS reform have been grouped into 6 general categories: 1) establishing of an integrated system of social care service at local level; 2) reviewing of functions and processes for service delivery at all levels; 3) putting in place a functional integrated information system; 4) ensuring the quality of provided services; 5) strengthening capacities of human resource; 6) raising awareness and educating the public on prevention of social problems.
UNICEF in Albania ANNEXES TO THE EVALUATION REPORT 9 Institutional Set-up ♦♦ Accountabilities and roles of the newly formed Local Government Units (LGUs) (reorganised into 61 Municipalities from the previous array of 373 Municipalities and Communes in 2015), in relation to the SCS planning in their territories, were clearly articulated and integrated in the National Inter-sectorial Decentralization Strategy for 2015-2020 and the new Law on Local Self-Government, adopted in 2015. The Needs Assessment and Referral Units (NARUs) are introduced as a separate unit in the local government administration to serve as key entry points enabling a prompt needs assessment and referral of cases to the appropriate forms and channels of social support. The Law on Social Care Service brings clear provisions detailing the roles and accountabilities of the key duty-bearers, including at the local level. ♦♦ Two new laws - On the Order of Social Workers and On the Order of Psychologists - were developed and approved by the Parliament in December 2014 and April 2016, respectively, laying the foundation for institutionalization of the “social work” profession, delivered according to the established standard for quality and ethics.
Human Resource capacities ♦♦ Tools were developed for Municipalities to budget, finance and manage the delivery of social care services. A standard package (“basket”) of services, with appropriate costing estimates, was developed, agreed upon and used as a core instrument by the first seven Municipalities in developing their SCS plans.
Data supply for service planning and monitoring ♦♦ Technical recommendations on establishing a comprehensive Management Information System (MIS) encompassing cash assistance and social care services in Albania were developed and presented to the Ministry of Social Welfare and Youth (MoSWY), with necessary financial resources already budgeted by the Ministry for the fiscal year of 2016.
Learning from Service modelling ♦♦ The work at the policy-making level was enriched by UNICEF supported services directly benefitting children and families, through its Child Protection and Early Childhood Development programme interventions. Such work did provide technical insights on development of some of the tools. ♦♦ The culmination of the work was the approval of the new Law on Social Care Services approved by the Parliament of Albania in December 2016 providing the foundations for both national and local level to take the proper actions and institute the necessary structures for allowing planning and delivery of social care services at decentralized level. Among others, the new Law on Social Care Services does provide for the proper mechanisms for the SCS financing. It envisages a combination of funding sources from: (i) the central State Budget, (ii) the local government budget, (iii) locally generated income of Municipalities, and (iv) fees that can be introduced for some social services. The Law allows the Municipalities to establish a dedicated “social fund” budget line, protected against utilisation for purposes different from social care services.
PHASE 2 – 2017-PRESENT As per projection of the National Social Protection Strategy, the SCS system should have started the piloting stage at municipality level from 2017. A stagnation of policy implementation occurred in 2017, due to major restructuring of central apparatus, accompanied with review of functions, transfer and re-assignment of staff. In confluence with the vertical (i.e. ongoing decentralization, territorial- administrative) governance reform, these changes introduced significant uncertainties into Albania’s overall government structures. On the positive note, the country’s integration in the European Union has become a topical agenda. This ambition has been a shared political priority and a vital development driver of comprehensive reforms spanning across different sectors. Progress has been swift and recognised by the European Commission, which, in April 2018, recommended considering
10 accession negotiations be opened in June 2019, deepening the reform momentum in the country. The second phase of the SCR corresponds to the second formal agreement signed between the SDC and UN Agencies in Albania, named ‘Promoting Social Inclusion – Leave no one behind’. Aligning efforts with other UN agencies, UNICEF would make use of 700,00 USD, to support the implementation of the policies and strategies at local level to enable ‘all women and men, boys and girls are exercising their entitlements to equitable quality services’ (project vision). Regardless the broad scope of social inclusion, the interventions planned under the project, for UNICEF, represent the continuation of the first phase implementation of the social care reform, focusing at operationalization of the normative policy framework.
The ‘Leave no one behind’ reform-supporting programme is based on three Outcomes envisaging that, by the end of 2021: ♦♦ The vulnerable population receives adequate social services from local authorities; ♦♦ Municipalities effectively manage the provision of social services; ♦♦ National institutions implement the relevant social policy framework.
UNICEF is committed to contribute to the three main objectives, attending to the main following, but not limited, pillars of interventions (outputs):
Mechanisms for social care provision ♦♦ Ministry of Health and Social Protection (MoHSP), former MSWY, has been supported in strengthening the normative framework with secondary legislation that will guide implementation of the laws and development of social care services at both the national and the local level. Fourteen by-laws were drafted and validated by national stakeholders, while nine have been approved and the rest awaiting approval. ♦♦ Agreement between Ministry of Finance and Economy and MoHSP has been achieved in defining the financing formula – Social Fund – for social services at municipality level.
Human Resource capacities ♦♦ Staff capacities of selected municipalities have been built in planning and delivering social services. That includes municipal officials trained and receiving on-the-job mentoring on the new legislation on social care services, the skills and necessary tools to develop and implement social care plans for poor and vulnerable families and children.
Awareness raising ♦♦ Engagement with policy-makers and opinion influencers to create champions for social care services will be key. Awareness raising activities, parental education programmes and public hearings on issues related to social care services planning and management to be organised at national and/or municipal levels will establish the demand for services.
The programme has been designed to establish the missing institutional framework for a system of social care services in Albania, but this has been a means to address one of the fundamental human and children rights; the right of men and women, boys and girls to social protection. At the beginning of the programme, Albania did not have the proper vision, not the policy framework for comprehensive, functional social protection system that would be able to provide systematic assistance in both care and cash to marginalized communities to escape the conditions that perpetuate poverty and vulnerability. Without a functional social protection mechanism that also addresses some key causes of perpetuating poverty, inequity is likely to continue to increase including here gender issues. The programme design was underlined by the assumption that the policies, procedures, structures, systems, capacities required to realise efficiency savings while protecting the rights of adults and children, men or women, boys or girls, are largely the same though there will be differences in some services. Overall, UNICEF contribution has attempted to follow the policy cycle perspective, integrating interventions at the design, delivery and monitoring components of the SCS Reform.
UNICEF in Albania ANNEXES TO THE EVALUATION REPORT 11 SCS REFORM STAKEHOLDERS’ ANALYSIS The key stakeholders involved in the SCS range from central to local apparatus, from state to non-state entities, include, but are not limited to:
Ministry of Social Welfare and Youth (MoSWY) re-configured as Ministry of Health and Social Protection (MHSP) in 2017, is the key stakeholder and the “owner” of the Social Care Services reform agenda. On the one hand, the MHSP in charge of the country’s social protection support (extended to the population either through in-kind services or as cash allowances), while on the other hand the MoSWY is expected to act as a national coordinator of social inclusion initiatives undertaken by other sectors.
Since 2015, a new mechanism of Integrated Policy Management Groups (IPMG) was introduced in Albania, with a view of improving policy coordination and monitoring in selected sectors9. The IPMG on Employment and Social Policy remains under the management of the MHSP, and the role of the IPMG chair is believed to benefit the Ministry’s work in social protection and inclusion too, through the spill over effect of the improved internal processes and mechanisms.
Ministry of Finance’s (MoF) role is central in the preparation of the public finance management analysis of Albania’s social care system and then in the development of technical options for financing social care services at local level.
The role of Local governmental units (LGUs) as key stakeholders is central, especially during second phase of SCR implementation, when is expected to put on scale the local action for effective and efficient service planning, monitoring and delivery. Other social sector ministries (Ministry of Education, Sports and Youth, Ministry of Justice) and their subordinate entities, have been also impacted by the SCS reform, as far as their mandates cover locally-offered services to the vulnerable and/or marginalised populations. It is important to underscore the role played by civil society organizations (CSOs) as a major group of stakeholders which has accumulated valuable technical expertise, delivering up to 80% of all social services in the country. Many CSOs possess unique networks on the ground to underpin the necessary support at the grass roots levels and have been very vocal in lobbying for the objectives of the reform, with a particular emphasis on the need to develop financial mechanisms allowing social services to be outsourced to non-governmental service providers.
PURPOSE & OBJECTIVES, EXPECTED RESULTS
In full collaboration and support to the GoA, UNICEF in Albania has been closely engaged with the SCS reform, since its initial design in 2010. The Reform has been implemented in several phases, and most importantly shaped by several happenings at country context, e.g. the parallel reforms of cash assistance, the territorial and decentralization reform, the EU’s accession negotiation, the re-configuration of governance apparatus, etc.
UNICEF in Albania intends to conduct an evaluation of UNICEF’s Contribution to the Social Care Services Reform in Albania, aiming to capture the interventions supported by UNICEF assistance during 2013-2019. The interventions, subject to the evaluation, are part of a broad collaboration of UNICEF with Ministry of Health and Social Protection, as embedded in UNICEF’s country programme documents and operationalized through UNICEF’s own resources and other fledged projects supported by SDC. The expertise of an international consultant is required through the publishing of this Terms of Reference (ToR) to lead the work and undertake all the necessary activities to complete the
9. Albania Prime Minister’s Order No. 129, dated 21 September 2015.
12 evaluation. The international consultant will be supported by a local expertise/evaluator/consultant, which ToR will be defined by the international consultant at the inception stage of the evaluation.
The object of Evaluation: The object of evaluation is UNICEF’s contribution to the normative policy framework of the Social Care Services Reform in Albania during 2013-2019.
The focus of the evaluation has been deemed necessary recognizing: ♦♦ the multidimensional (and complex) nature of the reform and UNICEF’s contribution to it. The evaluation of the entire components of UNICEF’s support to the reform (design, implementation, monitoring) would have transformed the evaluation into a complex and cost-demanding exercise, risking the timely delivery of the report for the intended use; ♦♦ the current status of the Reform corresponding to the start of the normative framework’ operationalization; Therefore, the evaluation findings and recommendations would be relevant/useful to foster this process; ♦♦ the crucial importance of the policy development/normative framework for the successful realization of reform. This component entails the institution of the guiding legal, regulatory and functional mechanisms/instruments, that constitute the cornerstone for the integrated social care services in Albania; ♦♦ the continuance of UNICEF’s contribution to the normative policy framework throughout the planning and implementation of the SCS reform (2013-2019).
Furthermore, the scope of the evaluation is defined by: ♦♦ Level of change: UNICEF’s initiatives towards policy development/establishment of the normative policy framework for the SCS Reform, including the entire set of laws, strategies, regulations formulated, and capacities build in respect to this framework; ♦♦ Programmatic framework: UNICEF’s contribution as part of the two SDC-supported projects (outlined in the Context section of the ToR). ♦♦ Theory of change: the evaluation will examine UNICEF’s results against the activities, objectives and goals of the two projects supported by SDC and UNICEF’s Country Programme Documents Results Frameworks. A tentative theory of change for the purpose of this evaluation is put together by UNICEF County Office (Appendix A), attempting to conceptualise the intended change processes starting with inputs towards impact-level contribution, in respect to policy development. ♦♦ Implementation time period: UNICEF’s initiatives undertaken during the years 2013-2019; ♦♦ Geographic coverage: UNICEF’s contribution to the establishment of normative policy framework at central/national level, and targeted municipalities supported by UNICEF (UNICEF has supported 11 municipalities – Tirana, Durres, Shkodra, Korca, Librazhd, Maliq, Mallakaster, Vlora, Saranda, Shijak, Malesia e Madhe). ♦♦ Evaluation criteria used to guide the evaluation thematically: relevance, effectiveness, efficiency, impact (to some extent) and sustainability;
The purpose of this evaluation is multi-folded, including to: ♦♦ Fulfil UNICEF’s accountability towards national government and institutions, donors, children and families, and other stakeholders by providing information on the use of resources and achievement of results that are measurable to-date in regard to the component of normative policy framework development of the SCS Reform. ♦♦ Identify missing opportunities and remaining challenges to inform / recommend decision- makers at national and local levels on actions to be undertaken in the future for effective and sustainable implementation of the normative policy framework so that it produces outcomes for children and families. ♦♦ Generate knowledge and evidence on achievements, lessons learned and recommendations in establishing and maintaining a human/child rights, equity and gender sensitive approach throughout the SCS Reform in Albania.
UNICEF in Albania ANNEXES TO THE EVALUATION REPORT 13 The specific objectives of the Evaluation are to: ♦♦ Identify the progress made so far in the implementation of the SCS reform, and overall UNICEF’s contribution to the Reform; ♦♦ Assess against UNICEF’s programmatic commitments and overall progress of the reform the relevance, effectiveness, efficiency, sustainability and, to the extent possible, progress towards impact of the UNICEF’s contribution to the establishment of the normative policy framework for the SCS reform during 2013-2019; ♦♦ To identify the extent to which cross-cutting strategies such as human rights-based approach, results-based management, analysis of child vulnerability and gender equity have contributed and have been mainstreamed in the policy development; ♦♦ To identify lessons learned and make recommendations for interventions’ adjustments required to improve and accelerate the effective and sustainable implementation of the normative policy framework in the next years to ensure that children and families are cared and supported.
The evaluation has both summative and formative purpose. With the policy framework component being at place to date, the evaluation will tend to take the summative approach. However, whilst the reform has embarked in 2018 in an important stage of policy implementation at both national and local level, the formative approach to evaluation will be handy to identify what could be mitigated in this regard, and the indication of the current bottlenecks may be associated with recommendations towards policy improvements. The duality of summative and formative elements is also present in the perspective of evaluating two consecutive projects, with one being already completed in 2016. However, the object of evaluation remains the support provided for the policy development spanned in both projects, rather than the projects per se.
Users of the evaluation will be: ♦♦ Implementing institutions of the reform – Ministry of Health and Social Welfare, State Social Services, Municipalities, NGO and civil society partners and other development partners (including UN agencies). ♦♦ UNICEF in Albania would make benefit of the evaluation results and overall knowledge for future support to the SCS Reform, and its internal planning, reporting and advocacy efforts informing the implementation of the 2017-2021 Country Programme. ♦♦ Donors, such as SDC, being the major financial contributor to the SCS reform, as well as other entities, such as European Union (EU), in the framework of its support for the country’s accession to the EU. ♦♦ The most significant use of the evaluation will be to inform: ♦♦ The revision of the National Social Protection Strategy by GoA, namely MHSP, and UNICEF, warranting that this central policy instrument will integrate actions, leading towards the realisation of a fully responsive social care system. ♦♦ Municipal social care plans development, so that actions are taken at local level to further accelerate the reform and enhance results for vulnerable groups, including children. ♦♦ The 2010-2024 EU budget support planning for the country, through the scheme of Instruments of Pre-Accession Assistance II, with a focus at implementing social care services system at local level.
EVALUABILITY ASSESSMENT Early in 2018, UNICEF Albania undertook a light evaluability assessment exercise, but targeting overall UNICEF’s contribution solely during 2013-2016. At that time, the intention was to perform the evaluation as specified in the UNICEF’s Country Evaluation Plan covering interventions until 2016. The main conclusions of the evaluability assessment report have been: ♦♦ UNICEF’s contribution for the policy development of the SCS Reform is evaluable, regardless of the limitations in data availability or unclarities in the theory of change. ♦♦ Any evaluation to be conducted should limit its intend for the impact assessment, due to the current status of the Reform’s implementation and the limitations in measuring the impact component.
14 The theory of change (appendix A) has been developed retroactively by the UNICEF Country Office only for the purpose of providing a general framework for this evaluation, assembling the relevant elements from the result frameworks of both projects. The evaluator is expected to further revise it during the inception phase.
To mitigate the above-mentioned limitations, partial evaluability assessment and incomplete theory of change, the evaluator is expected to perform an evaluability assessment exercise, including on reliability of disaggregated data, in light of the object of this evaluation. The findings will further inform the inception report and alert for any actions to be taken to improve availability of data.
EVALUATION CRITERIA The evaluation criteria of 1. Relevance; 2. Effectiveness; 3. Efficiency; 4. Progress towards Impact; 5. Sustainability will be applied, selected as a) the standard international criteria for development evaluation, as reflected in United National Evaluation Group (UNEG) standards, to which UNICEF adheres, b) appropriately geared to the Purpose and Objectives of the evaluation, as set out above.
The list is not exhaustive and should be considered subject to revision (addition/omission), once work for the compilation of the inception report kicks off, in close communication with UNICEF in Albania. Already, coordination and coherence dimensions/criteria of assessment have been integrated into the defined evaluation criteria.
TABLE 1.1: EVALUATION CRITERIA
Main guiding enquiry questions: To what extent UNICEF’s interventions to establish the normative policy framework have addressed the country context - the underlying systemic bottlenecks and the situation of the children and families in Albania? Have these issues sufficiently analysed and clarified to justify the choice of policy priorities? To what extent has UNICEF’s contribution been relevant in the context of European and internationally agreed human-rights commitments ratified by the GoA (e.g. SDG, CEDAW, CRC), with special focus on those promoting child rights advancement and gender equality?
Relevance Relevance To what extent are the main changes brought through UNICEF’s contribution to policy development of the SCS reform relevant/aligned to the overall country’s priorities, policies and strategies, as well as those of EU and UNICEF’s regional and global agenda in advancing a child sensitive social protection system? What actions need to be taken to ensure that the normative policy framework maintains ongoing relevance and adopts to the changing environment of the country and the dynamics of the situation of children and families?
Main guiding enquiry questions: What is the achievement level of planned results compared to the stated objectives/outcomes of UNICEF’s interventions for the development of the normative policy framework of the SCS reform? To what extent was UNICEF’s contribution to policy development effective in addressing specific human right’s needs, gender inequalities and inequity in service delivery, particularly for children and families? What were the main factors which contributed or hindered the achievement of the intended outputs/outcomes to date? Has the implementation of the Programme produced any additional, unplanned effect (positive or negative)? How effectively have they been addressed?
Effectiveness To what extend has been the coordination and synergy between activities (in context of normative policy framework) at different levels of governance (central/national and local/subnational)? How can that be improved in the future? What actions need to be taken to ensure effective implementation of the normative policy framework to deliver a functional and sustainable social care system for children and families?
Main guiding enquiry questions: To what extent are the resources (human and financial) used appropriately and strategically? Were there the major delays, if any, in the implementation of the interventions that affected the efficient use of resources? How has the project ensured co-ordination with other similar interventions to encourage synergy and avoid overlaps and influence any agendas so that social care services work for children? What are the areas and ways of cooperation with other agencies? What actions need to be taken to efficiently implement the normative policy framework of the SCS Reform, so that it is Efficiency relevant to the realities of children and families? To what extent were/are human/child rights and gender equality a priority in the overall budget and implementation of UNICEF’s interventions?
UNICEF in Albania ANNEXES TO THE EVALUATION REPORT 15 Main guiding enquiry question: Are there signs of UNICEF’s contribution to policy development leading to the establishment of a quality (functional and sustainable) social care system accessible by children and families and the most socially disadvantaged population
Impact (disaggregated by sex, age, and other criteria to be defined as result of evaluability assessment), and helping them escape the conditions that perpetuate child poverty and social exclusion?
Elements of sustainability have been mainstreamed along the above-mentioned criteria/components. However, the evaluation will assess specifically the extent to which the results achieved so far in policy development are likely to be maintained over time. Main guiding enquiry questions: To what extent are national and local stakeholders ready for implementation of the normative policy framework developed under UNICEF’s contribution? What factors may hinder or advance the sustainability of results benefitting children and families? What is the level of ownership of the reform process within the MHSP and local self-governments? What actions need to be taken to foster governmental ownership, so that it addresses the needs of families, and children and women, in particular? Sustainability To what extent has UNICEF promoted strengthening of already existing partnerships and establishment of new ones and the strengthening of inter-sectoral and cross-sectoral cooperation both at the national and local level for better outcomes for children? What actions need to be taken to improve in this regard? Based on lessons learned what specific recommendations could be given to each key partner under the project that would contribute to the sustainability of the intervention?
At this stage, a number of limitations have been identified, as described in the table below. Additional limitations are expected to be identified by the International Consultant at the inception phase.
TABLE 1.2: LIMITATIONS OF THE EVALUATION APPROACH
Potential limitations Proposed mitigation measures
A clear and comprehensive theory of change linking results In close collaboration with UNICEF in Albania, main counterparties, with UNICEF’s contribution over the years is missing. This is and based on preliminary desk review, the theory of change is to be partly due to the fact, that in 2012/13, a theory of change was retro-actively re-constructed, including baselines and targets. In case not an institutional requirement for developing a programme. of persistent information gaps, a more qualitative appraisal will be In addition, there is a dearth of indicators, including baseline used, applying contribution analysis at different levels. A mixed- and target data. method approach will be utilized to make use of available qualitative and quantitative sources and ensure cross-checking/triangulation of information. The desk review will be dependent on explicit evidence contained The most significant shortcomings and gaps in the documentary in planning and reporting documents of UNICEF and GoA. evidence to be explicitly identified during the inception phase. Originally, the compilation of these documents has not taken Primary data collection to cover gaps and add multiple informant into account the evaluation criteria, so the degree of explanation perspectives. of UNICEF’s contribution may vary and not always satisfy the requirements for information in the framework of this evaluation.
Validity and reliability of interview data, particularly in a Strict application of ethical standards, ensuring interviewees are political sensitive environment (the evaluation’s undertaking thoroughly briefed/ informed at the start of interviews about the will correspond with local elections in Albania). scope of the evaluation, and the elements of confidentiality and anonymity. At inception stage, the likely limitations from certain sources should be recognized, in close communication with UNICEF staff. The Reference Group established for the Evaluation will also play as an intermediator with institutions during the data collection phase.
METHODOLOGICAL APPROACH The detailed methodology will be fully designed by the International Consultant as part of the inception phase (report). The methodology should aim to utilize the best mix of data gathering tools to yield the most reliable and valid answers to the evaluation questions and generate maximum learning within the limits of resources and availability of data.
In order to serve its purpose, the evaluation may apply contribution analysis, and have a utilization focused approach, including, but not limited to: stakeholder mapping; mapping of situation and
16 contextual analyses, barriers and bottlenecks; in-depth documentary review and structured desk analysis of Programme/projects/interventions design, implementation approaches, documenting of results and processes; structured desk analysis of policy documents and legislative frameworks (preliminary list to be found at Appendix B); reconstruction of the Theory of Change; analysis of results from M&E systems; contribution analysis to determine factors which promoted or impeded the progress against intended results and attribution analysis to the extent possible; financial analysis; analysis of sustainability strategies and systemic barriers to sustainability; in-depth interviews and focus groups. Sampling criteria for the municipalities will be proposed by the Consultant at the inception report.
The evaluation will assess the relevance, efficiency, effectiveness, impact (to some extent) and sustainability of the UNICEF’s interventions based on the revised ToC, the Results Framework and indicators used for monitoring performance and attainment of estimated results. A core analytical framework, containing criteria of analysis, will be developed for the evaluation questions at the inception phase (report), against which data will be gathered and analysed. Evaluation questions outlined in the ToR are expected to be reformulated, regrouped and streamlined on the basis of the logic model reflected by the revised Theory of change and will be assigned to specific evaluation criteria (relevance, effectiveness, efficiency, impact and sustainability).
Methods for ensuring validity and reliability at analysis stage will include: ♦♦ Triangulation – to confirm and corroborate results reached by different methods. ♦♦ Complementarity - to explain and understand findings obtained by one method by applying a second. ♦♦ Interrogation - where diverging results emerge from the application of different methods – these will need to be interrogated to either reconcile, or explain, the differences apparent.
HUMAN RIGHTS AND ETHICAL CONSIDERATIONS The evaluation should be guided by principles the Results-Based Management10 and Human Rights Based Approaches to programming and evaluation. The entire process and deliverables of the evaluation should be sensitive to gender equity, and human rights aspects, mostly child rights11.
The international consultant will commit to fully adhere to the required ethical principles/standards for data collection, as specified in the UNICEF’s policy and Albanian ethical regulation for research. Ethical approval by UNICEF’s ethical review board may be required for the inception report, in case data collection will involve sensitive questions or have vulnerable categories of population, including children as participants in the evaluation.
The international consultant will ensure to exercise independent judgement and operate in an impartial and unbiased manner. The final report should be credible, based on reliable data and observations, and provide a comprehensive and balanced presentation of the evidence. During data collection, any sensitive issues or concerns, including conflicts of interest, will be raised with the UNICEF evaluation manager as soon as they are identified.
During data collection, attention will be paid to ensure that the evaluation process is ethical, by ensuring to: ♦♦ Identify the need for and securing necessary approval. ♦♦ Avoid any harm to participants and respect full confidentiality. The privacy and safety of respondents should be a priority. Respondent participation should be voluntary, without undue burden, free from any duress, consensual and furnished with written consent where possible. For the safety and security of evaluator, the necessary training and on-the-ground
10. The Common Understanding on HRBA to Development Cooperation and Programming (United Nations Development Group, 2003). 11. Guidance on Human Rights and Gender Equality in Evaluation
UNICEF in Albania ANNEXES TO THE EVALUATION REPORT 17 support will be provided by UNICEF in Albania, to avoid any issues of safety. ♦♦ Respect respondents’ dignity, vulnerabilities and cultural sensitivities. Development and use of data collection instruments should be performed to take into account the variances in ethnicities, gender, disability, age and other variables like socio-economic categories of those engaged in any form during the implementation of the evaluation. ♦♦ Select fairly and in a representative manner the respondents. The evaluator should define and document the criteria for selections of any participants in the evaluation.1213
STEPS OF THE EVALUATION
TABLE 1.3: EVALUATION ROADMAP Preparatory Work/Inception Phase Tasks Responsible Timeline in 2019 Establishment of the Evaluation Reference Group UNICEF in Albania March Create a google drive folder with all materials for UNICEF Albania March desk review Agree on the roadmap for the undertaking of the UNICEF and International Consultant Following the signature of evaluation (with timeline) the contract Conduct thorough desk review of all relevant International Consultant April materials (reports, strategies, policies, legislation, etc.) in light of evaluability assessment. Prepare Draft Inception Report12, including re- International Consultant April – May (mission in the constructed Theory of Change and TOR for National country for International Evaluator Consultant) Quality Assurance/Validation for the Inception UNIVERSALIA + UNICEF in Albania + Reference May Report Group + Regional Office Consultant Finalize the Inception Report International Consultant End of May Field Work Data collection – interviews/focus groups International Consultant + National Consultant June – July 2019 (mission in the country for International Consultant) Report Preparation Data Analysis and Processing International Consultant + National Consultant August – September Preliminary findings summary International Consultant September Validation UNICEF in Albania + Reference Group October Draft Final Report13 International Consultant October Quality Assurance/Validation for the Draft Final UNIVERSALIA + UNICEF in Albania + November Report Reference Group Validation of the report (recommendations) with International Consultant + Local Consultant + December (mission in the various stakeholders UNICEF in Albania + Reference Group + various country for International stakeholders Consultant) Completion of Final Report International Consultant December Preparation of the Management response and UNICEF in Albania + Reference Group January 2020 dissemination of the Report
12. The inception and final report will adhere to UNICEF’s Publication Toolkit, to be shared by UNICEF in Albania team at the start of the contract. Proposed Outline for the Inception Report: 1. Introduction; 2. Context/ object of Evaluation; 3. Objectives and Scope; 4. Theory of Change; 5. Stakeholders analysis; 6. Evaluation framework/analytical framework with a detailed matrix (questions, data sources, judgement criteria, etc.); 7. Methodology (limitations + ethical considerations + data collection); 8. Preliminary findings from desk review (list of materials reviewed and planned to be reviewed); 9. Workplan, including detailed field work with responsibilities and timelines (TOR for the national Consultant); 10. Proposed structure of the Final Report (including how the recommendations will be presented). 13. The structure of the Final Report should follow UNICEF-Adapted UNEG Evaluation Reports Standards.
18 KEY DELIVERABLES AND TIMEFRAME
DELIVERABLE 1 – INCEPTION REPORT OF THE EVALUATION
Tasks: ♦♦ Thorough desk review / secondary data analysis of relevant materials; ♦♦ Conduction of (Skype) interviews with UNICEF team and other key informants as deemed necessary; ♦♦ Drafting of the Inception Report and the ToR for the National Evaluator; ♦♦ Completion of the Inception Report, after incorporating the feedback received.
To note: Acceptance of the Inception Report will be subject to receiving at least ‘satisfactory’ grading by the UNICEF’s hired company for evaluation reviewing.
Duration: 22 working days (10 working days in-country mission) Actor: International Consultant Completion Date: end of May 2019
DELIVERABLE 2 – FINAL REPORT OF THE EVALUATION
Tasks: ♦♦ Provide orientation/guidance for the national evaluator during data collection; ♦♦ Conduct interviews with key stakeholders during the in-country mission; ♦♦ Desk review of additional materials; ♦♦ Draft Preliminary Findings (narrative and power-point version); ♦♦ Draft the Final Report of Evaluation, incorporating the feedback received; ♦♦ Support in the conceptualisation/organization of the validation workshop; Prepare power- point presentation and 6 pages summary document for the participants of the workshop; Present findings and recommendations at the workshop (to be determined later if via skype or in-country mission). ♦♦ Complete the Final Report of Evaluation.
Duration: 43 working days (15 working days in-country mission) Actor: International Consultant Completion Date: end of December 2019
To note: Acceptance of the Draft Final Report will be subject to receiving at least ‘satisfactory’ grading by the UNICEF’s hired company for evaluation reviewing.
QUALITY ASSURANCE
The Evaluation should fully adhere with the United Nation Evaluation Group Norms and Standards for Evaluation (2016). UNICEF retains an external company to review the Inception Report and Final Report, using quality review matrixes (which will be shared by UNICEF at inception phase). None of the deliverables will not be cleared by UNICEF until assessed at least satisfactory by the external facility.
The international consultant will have a number of check points for quality assurance: ♦♦ A discussion with UNICEF Albania team following the desk review, which will help uncover sources of usable secondary data and identify additional sources of secondary data as well
UNICEF in Albania ANNEXES TO THE EVALUATION REPORT 19 specific methodological difficulties that may be encountered through the evaluation process. The discussion at this stage will help to identify those areas where primary data is needed to complement limitations in terms of quality and availability of secondary data. ♦♦ A discussion over of the Inception Report to ensure that the international consultant’s understanding of what is required corresponds to the evaluation standards and UNICEF & Reference Group’s expectations. The international consultant will interact at this stage with stakeholders while in the country. ♦♦ Presentation and discussion of preliminary findings with UNICEF and Reference Group. ♦♦ A validation workshop with the larger audience of relevant stakeholders. ♦♦ A review of the draft and Final Report by UNICEF and Reference Group.
Adjustments will be made to reflect feedback at each of these points. This process ensures that multiple opportunities are provided to resolve issues and challenges throughout the evaluation exercise.
RESPONSIBILITIES
The evaluation is commissioned and will be managed by UNICEF in Albania, namely the Monitoring and Evaluation Specialist, overseen by the Representative of Country Office. UNICEF has the responsibility to manage and support the consultancy and provide overall quality assurance to process and deliverables, as well as develop an explicit communication strategy for the evaluation, including the dissemination plan and expected use of findings for advocacy. Support to the consultant would include regular communication, feedback and discussion on the progress of the assignment. UNICEF will be responsible to share and collect feedback in due time on the deliverables.
A dedicated Reference Group will be established for the evaluation to foster/enable the participation of relevant stakeholders in the design and scope of the evaluation, to raise awareness of the different information needs, to provide information quality assurance throughout the process and support the dissemination of the evaluation results. The Reference group will be composed, but not limited to, by representatives of MHSP and its subordinate entity State Social Services, at least two local government units, SDC, EU and other UN Agencies.
The international consultant will be responsible to undertake all agreed tasks in a timely manner and delivering quality results as per those specified in this ToR. The international consultant will be responsible to guide and coordinate the work of local consultant, whose ToR will be compiled and agreed upon during the inception phase.
REPORTING REQUIREMENTS
♦♦ From the onset of the work, the international consultant and UNICEF’s evaluation manager will agree on monthly updated work-plan to keep up to date with the process of evaluation implementation. ♦♦ Deliverables/Reports to be submitted to UNICEF according to the timelines indicated in the section of the Deliverables above, all in English language. ♦♦ All materials developed will remain the copyright of UNICEF. UNICEF will be free to adapt and modify them in the future. ♦♦ The Inception Report should contain, but not limited to the following components/chapters: (i) Introduction; (ii) Context/ object of Evaluation; (iii) Objectives and Scope; (iv) Theory of
20 Change; (v) Stakeholders analysis; (vi) Evaluation framework/analytical framework with a detailed matrix (questions, data sources, judgement criteria, etc.); (vii). Methodology (limitations + ethical considerations + data collection); (viii) Preliminary findings from desk review (list of materials reviewed and planned to be reviewed); (ix). Workplan, including detailed field work with responsibilities and timelines (TOR for the national Consultant); (x) Proposed structure of the Final Report (including how the recommendations will be presented). ♦♦ Structure of the Final Report should be guided by the UNICEF-Adapted UNEG Evaluation Reports Standards.
LOCATION AND DURATION
♦♦ Country mission trips are expected as per indicative schedule outlined in this TOR. ♦♦ All travel plans, and duration of the stay in the country will be based on the agreed implementation plan, updated on monthly basis by UNICEF’s evaluation manager and International Consultant. ♦♦ UNICEF will make the international travel arrangements for the consultants, based on economy class travel, regardless of the length of travel. ♦♦ The consultant is responsible for assuming costs for obtaining visa(s) and travel insurance. ♦♦ Daily subsistence allowance (DSA) will be paid, when applicable, as per respective UN rates. Travel days are not included as working days. ♦♦ Travelling in other cities outside Tirana for data collection will be covered by UNICEF. ♦♦ Office space during country missions, as well as local travel and translation, when needed, will be provided by UNICEF; ♦♦ Office space, IT solutions, communication costs, etc. are the responsibility of the International Consultant when not present in Albania.
The exact dates for starting and ending the contract will be specified upon contract award.
Tentative starting period: April 2019 Foreseen finishing period or duration: February 2020
QUALIFICATION REQUIREMENTS
The qualifications and skill areas required for the international consultant include:
♦♦ At least 10 years of documented progressive evaluation work experience, including in the area of social protection, preferably in the area of social care services. Additional assets: Having conducted evaluation for UNICEF, Having evaluation positively rated by UNICEF’s quality assurance system ♦♦ Experience in working in developing countries, including South-East Europe countries. Additional asset: Previous work in Albania ♦♦ Solid Knowledge on child rights/social inclusion; ♦♦ Strong knowledge on gender equality and human rights-based approach to programming (theory of change); ♦♦ Excellent report writing skills in the English Language; ♦♦ Solid communication and presentation skills; ♦♦ Excellent written and spoken English; ♦♦ Ability to keep with strict deadlines.
UNICEF in Albania ANNEXES TO THE EVALUATION REPORT 21 APPENDIX A: Preliminary constructed theory of change for UNICEF’s contribution to the normative policy framework of the SCS Reform 2013-2019
OUTPUT OUTCOMES IMPACT Overall vision, objectives and process of the reform A national comprehensive normative policy All women and men, boys articulated, formalised in national legislative and framework exists and is operationalized by and girls are exercising their administrative documents, and largely publicised. local government units and service providers entitlements to equitable empowered to effectively reach and fulfil the quality social care services. The Ministry Social Welfare makes a decision on rights the most marginalized populations. structural design and starts establishing a system of public authority entities to plan, manage, monitor and National Institutions implement their policy deliver social care services (based on accumulated framework for ensuring social inclusion and evidence and validated through stakeholder adequately fund social services through consultations). improved data collection system, developed capacities and empowered target groups. In close collaboration with the Ministry of Finance, legislative and normative provisions are developed Municipalities effectively manage the allowing the utilization of social protection budgets provision of social services and promote for care services (including, sub-contracting of NGO social inclusion. providers).
The profession of “social work” is fully institutionalised, including job profiling, education curricular and standards, licensing procedure and in-service training.
Ministries are supported in their implementation of policies and strategies and in plans for funding policies and strategies based on monitoring data.
The establishment of funding mechanisms for development and scaling-up of innovative social services is supported.
22 APPENDIX B: List of Materials related to the SCS Reform
A. Hoxha et al., Social Care Services at the Regional Scale: Analysis and recommendations, UNICEF, 2013 E. Jorgoni et al., Rescaling Social Care Services: Proposing Responsibilities and Functions for Local Government Units, UNICEF, 2014 G. Matković et al., Supporting Planning of Social Care Services: Proposal on Establishing a Package of Services in Albania and Financing Mechanisms, UNICEF, 2015 Government of Albania, National Pact on Social Welfare, 2016 Government of Albania, Social Protection Strategy 2015-2020, 2015 J. Miluka et al., Supporting Planning of Social Care Services: Vulnerability Mapping, UNICEF, 2016 L. Joshua, Support to sustainable reform processes in the employment, skills and social policy sector Albania, 2009 Law on Local Self-Governance, 2015. Law on social care services (https://www.parlament.al/wp-content/uploads/2016/11/ligj-nr.-121- dt.-24.11.2016.pdf) Law on the Organization & Functioning of Local Government No 8652, dated 31.07.2000 Mel Cousins and Associates, Documenting the process of Local Government Units (LGUs) capacity building in Social Care Planning, UNICEF 2017 (to be published) P. Evans, A vision for social care in Albania, UNICEF 2014 P. Evans, Charting Social Care Reform in Albania: Explanatory Note, UNICEF 2014 P. Evans, Deinstitutionalisation: a Concept Note, UNICEF 2015 P. Evans, Encouraging social services reform in Albania, UNICEF 2015 P. Evans, Social Care Decentralisation framework & action plan - Explanatory note, UNICEF 2014 P. Evans, Social Care Decentralisation framework & action plan – summary, UNICEF2015 P. Evans, Why reform social services in Albania? UNICEF 2015 Project Proposal: SCS reform in Albania 2012-2016 to address vulnerability and marginalization of children, women and families (2012). Project Proposal: Leave No One Behind – Promoting Social Inclusion (2017) UNDP, integrated community based social, services in Albania to address the needs, of Roma and Egyptian communities, 2017 UNICEF, Draft Policy & Framework for Regional & Local Social Services Planning, UNICEF 2011 UNICEF, Social Care Service Reform in Albania (2012-2016) to address vulnerability and marginalization of children, women and families (with focus on integration of Roma communities) Funding Proposal submitted by UNICEF Albania Country Office to the Swiss Development Cooperation, UNICEF 2012 UNICEF, Social care services at the regional scale: Analysis and recommendations, 2013 UNICEF, Social Care Services Reform in Albania (2012-2016) to address vulnerability and marginalization of children, women and families (with focus on integration of Roma communities) Final Donor Report submitted by UNICEF Albania to Swiss Agency for Development and Cooperation (SDC), UNICEF 2017
UNICEF in Albania ANNEXES TO THE EVALUATION REPORT 23 ANNEX TERMS OF REFERENCE FOR THE 2 EVALUATION REFERENCE GROUP
BACKGROUND
In full collaboration with and support to the Government of Albania, UNICEF in Albania has been closely engaged with the Social Care System (SCS) reform, since its initial design in 2010. In 2019–2020, in alignment with the Joint Work Plan of the Government of Albania and United Nations Agencies, UNICEF intends to conduct an evaluation of its contribution to the Normative Policy Framework of the SCS Reform, aiming to target the interventions it has assisted during the years 2013–2019.
The interventions, subject to the evaluation, are part of a broad collaboration of UNICEF with the Ministry of Health and Social Protection, and local government units as embedded in UNICEF’s country programme documents and operationalised through its own resources and other fledged projects supported by the Swiss Agency for Development and Cooperation.
EVALUATION PURPOSE AND OBJECTIVES
The purpose of this evaluation is multifolded, including the following:
♦♦ To fulfil UNICEF’s accountability towards the national government and institutions, donors, children and families, and other stakeholders, by providing information on the use of resources and achievement of measurable results to date with regard to the component of normative policy framework development of the SCS reform. ♦♦ To identify missing opportunities and remaining challenges to inform and recommend to decision makers at national and local levels of actions to be undertaken in the future for effective and sustainable implementation of the normative policy framework so that it produces outcomes for children and families. ♦♦ To generate knowledge and evidence on achievements, lessons learned and recommendations in establishing and maintaining a human and child rights, equity and gender-sensitive approach throughout the SCS reform in the country.
24 EVALUATION MANAGEMENT AND FUNCTION OF THE REFERENCE GROUP
The evaluation will be a consultative, inclusive and participatory process, and will be managed by the UNICEF Albania Office with the close support of the UNICEF Regional Evaluation Advisor. Under the guidance of the UNICEF Global Policy of Evaluation, the evaluation will be carried out by an external team comprising one international and one national expert. The process will undergo several steps of quality assurance, as specified in the policy. Presently, only the international consultant is selected.
The establishment of a Reference Group for this evaluation is deemed important to strengthen the quality and ownership of the evaluation, by facilitating discussion and solicitation of contributions of relevant stakeholders around:
♦♦ the findings and recommendations of the evaluation ♦♦ information needs for the evaluation and access to materials for desk review and key informants for interviews ♦♦ steps and activities for validation and quality assurance of the evaluation ♦♦ dissemination of the evaluation results.
By being part of the Reference Group, the members will be invited to participate in meetings and other activities, such as validation workshops organised with a broader range of stakeholders. The members might also be asked for bilateral in-depth interviews as key informants for the evaluation.
COMPOSITION AND CONTRIBUTIONS OF THE REFERENCE GROUP
The evaluation Reference Group is intended to be a joint composition of UNICEF in Albania and relevant government representatives. It will be co-chaired by the UNICEF Representative in Albania and one appointed representative of the Ministry of Health and Social Protection, with a managerial function. The composition of the Reference Group will enable representation of relevant stakeholders from national and local authorities, as well as the development partners and other relevant stakeholders in the field of SCS reform. The members of the group are expected to have a technical background with roles and responsibilities linked to the SCS reform.
Proposed Composition of the Evaluation Reference Group
Institution Number of representatives UNICEF in Albania 3 Ministry of Health and Social Protection 2 Ministry of Finance and Economy 1 Academia 2 European Union Delegation 1 Municipality of Durres 1 Municipality of Tirana 1 National Council of NGOs 1 Swiss Agency for Development and Cooperation 1 UNDP 1
UNICEF in Albania ANNEXES TO THE EVALUATION REPORT 25 It is expected that the Reference Group will meet periodically to be informed of the progress of the evaluation and to discuss its major deliverables. Additional updates and other matters for consensus within the Reference Group will be followed up by email communication.
All evaluation deliverables (inception report, draft evaluation report and final evaluation report) will be sent for review and feedback to the Reference Group members and co-chairs at least ten working days prior to the Reference Group meetings. Reference Group members and co-chairs are therefore expected to review the evaluation deliverables prior to each meeting and provide their comments, either in writing or orally, during the meeting.
Planned Reference Expected date Deliverables planned Group meeting of the meeting for review Introductory meeting June 2019 Inception report July 2019 Inception report Meeting on the preliminary findings October 2019 First draft evaluation report Final Reference Group meeting November 2019 Draft final evaluation report
The quorum for the meeting is reached when at least one of the chairs and at least half the other members are present.
UNICEF in Albania will act as a secretariat and provide the logistical support needed for the organisation of the meetings. Elda Hallkaj, Child Rights Monitoring Specialist (ehallkaj@unicef. org), will be the contact focal person for this evaluation.
26 TABLE 2.1: REFERENCE GROUP MEMBERS AND THEIR RELATION TO THE OBJECT OF EVALUATION
Name Institution Relation to SCS reform Relation to evaluation Ms. Merita Xhafa Ministry for Health and Policy maker for the social care Potential user of lessons learned and Social Welfare services reform recommendations
Ms. Denada Seferi Ministry for Health and Policy maker for the social care Potential user of lessons learned and Social Welfare services reform recommendations
Ms. Meme Xhaferraj Durres Municipal Social Practitioner, implementing the Representative of sampled Protection Department normative policy framework municipality. Potential user of lessons learned and recommendations/ Mr. Roberto de Bernardi UNICEF Albania Supervised UNICEF interventions Commissioner of the evaluation. Potential user of lessons learned and recommendations. Ms. Elda Hallkaj UNICEF Albania Supported the monitoring of UNICEF Commissioner of the evaluation. interventions Potential user of lessons learned and recommendations. Ms. Entela Lako UNDP Albania Implementer of Leave No One Potential user of lessons learned and Behind (programme of which the recommendations. object of the evaluation is a part) Ms. Enkelejda Bregu EU Delegation in Albania Donor Potential user of lessons learned and recommendations.
Ms. Silvana Mjeda Swiss Development Donor Potential user of lessons learned and Cooperation recommendations.
Ms. Rudina Rama Tirana Faculty of Social Representative of the academia, Potential user of lessons learned and Sciences policy maker for the social care recommendations. services reform Ms. Anisa Ruseti Tirana Municipality Representative of pioneering Potential user of lessons learned and municipality in social care services. recommendations.
Mr. Valion Cenalia Ministry of Finance Policy maker for the social care Potential user of lessons learned and services reform, particularly as recommendations. regards financial mechanisms. Mr. Blendi Dibra Albanian National Council Civil society representative, policy Potential user of lessons learned and of Civil Society maker for the social care services recommendations. reform
UNICEF in Albania ANNEXES TO THE EVALUATION REPORT 27 ANNEX VISUAL AIDS TO THE 3 THEORY OF CHANGE
These diagrams are used to illustrate and facilitate the reading of the presentation of this process. The following key is being used:
Figure 3.1: Key to theory of change and SCS system diagrams
The diagram below presents an overview of UNICEF’s theory of change (TOC) for its support to SCS in Albania:
Figure 3.2: Theory of change of UNICEF’s contribution to the policy normative framework of Albania’s SCS system
28 The diagram below provides an overview of the criteria as relates to the theory of change:
Figure 3.3: Evaluation criteria and UNICEF theory of change
At the heart of this theory of change lays the emergence of an SCS system that would mainstream UNICEF’s key principles.
The first level of construction of the SCS system, developed by the GoA with, inter alia, UNICEF’s support comprises the definition of the SCS building blocks:
♦♦ The social cares services themselves, which form a “basket of services”: ‐‐ Services delivered directly to beneficiaries at the municipal level (home-based services, and services provided in community centres), and at state level (specialized services for specific categories of beneficiaries such as children, persons with disabilities, and ageing persons); ‐‐ Residential services, administered by the State Social Service; ‐‐ Para-social services: all sub-services at municipal level that make delivery to the beneficiaries possible (e.g. needs assessments); ♦♦ The administrative flows and processes which support the system: ‐‐ Financial flows and processes; ‐‐ Decision-making lines and processes; ♦♦ The tools which operationalize the system: ‐‐ ICT tools; ‐‐ Municipal Social Plans (MSPs), themselves supported by municipal needs analysis, and which enable tailored SCS at the municipal level; ‐‐ Social pacts, which crystallize the mutual commitments of the State and Municipalities in the implementation of the SCS system; ♦♦ The staff of the Ministry, State Social Service and Needs Assessment and Referral Units (NARUs) at municipal level, who represent human capacities which embody and operate the
UNICEF in Albania ANNEXES TO THE EVALUATION REPORT 29 system. Key to the success of the theory of change, this staff needs to be: ‐‐ Ready: possessing the adequate knowledge and know how to implement the policy, legislative, regulatory and institutional framework, and to operate the system; ‐‐ Able: having the decision-making power, infrastructures and resources to implement the policy, legislative, regulatory and institutional framework, and to operate the system; ‐‐ Willing: sharing the essential values and principles, and having the motivation to implement the policy, legislative, regulatory and institutional framework, and to operate the system.
Figure 3.4: Building blocks of the SCS system
The second level is the legal framework surrounding these building blocks. Chronologically, the Government of Albania and UNICEF started with the National Strategy for Social Protection, and the Law on Social Protection (although it should be noted that these documents wend beyond sole SCS, and also encompassed cash assistance). At this point, the Government of Albania and UNICEF seized the opportunity of the ongoing decentralization reform, and of the drafting process of the Law on Local Self Governance, to integrate into this law, the attribution of SCS functions to the various levels of government. The diagram below shows how these documents first came to crystallize the vision and its building blocks:
30 Figure 3.5: The SCS system building blocks within the national policy/strategic, and legislative framework
According to UNICEF’s theory of change, this legal framework is expected to convey UNICEF’s guiding principles:
Figure 3.6: Mainstreaming of UNICEF principles into the policy and legislative framework of the SCS system
UNICEF in Albania ANNEXES TO THE EVALUATION REPORT 31 Within this basic framework, specialized regulatory texts (by-laws) needed to regulate the various building blocks. ♦♦ The basket of services ♦♦ The financial flows ♦♦ The decision-making processes ♦♦ The training of the staff, to ensure its readiness to implement the policy normative framework and operate the system for the benefit of the users – as there existed no diploma or curricula for social workers, and no framework for their lifelong professional training.
Figure 3.7: Coverage of the SCS system and its building blocks by the relevant policy, legislative, regulatory and institutional framework (i.e. normative policy framework)
This framework, in addition to the content of each building block, devolved the various processes of the SCS system to the various level(s) of governance: centralized national level (e.g. the MoSWY or the State Social Service), deconcentrated national level (e.g. representations of the State Social Services at regional level), municipal (e.g. NARUs), or multi-level:
32 Figure 3.8: Governance level of the SCS system’s building blocks
Again, UNICEF invested considerable efforts to mainstream its principles through each of these building blocks, through support to, advocacy for, and contributions to these regulatory documents. UNICEF’s internal documents bear the traces of these efforts.
Figure 3.9: Mainstreaming of UNICEF principles in to the SCS system’s regulatory documents
UNICEF in Albania ANNEXES TO THE EVALUATION REPORT 33 In addition, the Government of Albania and UNICEF had to establish and regulate the profession of social workers, which was a relatively recent development (1992), and whose role in the SCS system, and governance structures, needed to be defined. This was a precondition for the relevant staff to be able to perform their duties. Once again, in keeping with its mandate, UNICEF strove to mainstream its key principles into the draft law. The same would be done later for the profession of psychologists.
Figure 3.10: Insertion of the Law on Social Workers into the SCS system’s normative policy framework
Finally, UNICEF mainstreamed child-sensitiveness and the targeting of children and their families, mainly through the specifications of specialized services to particular categories of beneficiaries (children, the elderly, and persons with disabilities):
34 Figure 3.11: Mainstreaming of child-sensitiveness and targeting into the SCS system through specialized services
Through these steps and efforts, UNICEF expected the SCS system, its vision, as well as the entire normative policy framework establishing it, to respond to the internal assumptions, and be compliant with UNICEF principles, as well as sensitive for, and targeted towards children and their families:
Figure 3.12: UNICEF’s vision for the SCS system
UNICEF in Albania ANNEXES TO THE EVALUATION REPORT 35 ANNEX EXTENDED 4 EVALUATION MATRIX EVALUATION EVALUATION QUESTIONS SUB-QUESTIONS INDICATOR DATA COLLECTION METHODS DATA SOURCES CRITERIA Relevance, added To what extent did UNICEF’s theory of change How coherent are UNICEF’s theory of change, and implemented action, with its guiding principles, its Regularity of UNICEF Albania communication with and contribution Document review UNICEF and GoA strategic and programmatic documents. Applicable treaties, value match UNICEF’s priorities and standards, and overall strategic documents and with the recommendations of relevant treaty bodies? to GoA policy, regulatory and legislative drafting Survey EU agreements, and other commitments of Albania. Treaty bodies’ reports, Albania’s priorities and vision? What were UNICEF’s and the GoA’s approaches for the policy normative framework of SCS reform? Recurrence of common terminology between UNICEF country reports, shadow reports. How aligned were these approaches with each other, and with identified needs? organization-wide strategic guidance, and UNICEF in Albania Semi-structured interviews Survey responses strategic documents Responses of complementary informants Evidence of social cleavages articulated in international (WB, Written input and interviews with UNICEF social protection experts and UNDP, UNICEF) and national studies (governmental and CSO reports). international expert on SCS Focus group with policy makers To what extent has UNICEF’s theory of change Has UNICEF contributed to policy, legislation, regulations and institutions with sufficient and quality Treaty bodies’ reports, country reports, shadow reports. addressed the need for, and barriers to the social analysis of barriers to social? Dynamics of poverty and inequality indicators as evidenced by Survey responses inclusion children and families? To what extent have UNICEF’s interventions in support of the normative policy framework contributed UNICEF and other international and national reports. Responses of complementary informants to mainstreaming and targeting the needs of the children and families in Albania? Programming documents International expertise/ peer review Monitoring reports Written input and interviews with two international social protection Thematic and municipal case studies experts How relevant is UNICEF’s Theory of Change To what extent has UNICEF’s action been coherent with its theory of change? The degree of alignment of the priorities of UNICEF in Albania UNICEF guidelines for interventions in middle-income countries within the current action to the emerging needs How has UNICEF’s theory of change evolved over time? with national priorities as articulated in national strategic Treaty bodies’ reports, country reports, shadow reports. and priorities of Albania? What are the possible scenarios to optimize this relevance in the long run? documents National strategic documents The perception of alignment between the national and UNICEF EU strategic documents in Albania priorities by decision makers and stakeholders. Survey responses Responses of complementary informants Programming documents Monitoring reports Effectiveness What were the results of UNICEF’s action in terms Was UNICEF’s action instrumental in setting a policy agenda for social care services’ reform and if so Self-reported understanding by stakeholders of the building Document review Applicable policy normative framework of establishment of building blocks of the SCS in what way? blocks as constitutive of a systems approach. Focus groups International standards system? To what extent was UNICEF’s contribution to policy development effective in mainstreaming vulnerable Recurrence of key terminology and constitutive elements of the Semi-structured interviews Projects’ documentation groups and their needs (e.g. gender inequalities and inequity in service delivery, particularly for children SCS system, both in UNICEF strategic documents and in the International expertise/ peer review Projects’ publications and families)? normative policy framework as adopted by Albanian authorities Thematic and municipal case studies Proceeds of events organized by the projects. Was UNICEF’s action instrumental to the design and /or adoption of the policy normative framework of Recurrence of semantic similarities between UNICEF guidance Informants’ responses and experiences To what extent has UNICEF succeeded in SCS, and if so in what way? and/or communication documents, and the normative policy Written input and interviews with two international social protection mainstreaming of UNICEF principles? How has UNICEF mainstreamed its key expertise and principles for SCS during the design and/r framework experts adoption of the policy normative framework of SCS? Albania’s normative and policy framework provide adequate resources for meeting commitments on social care services Efficiency, added To what extent has UNICEF optimized its assets What are UNICEF’s comparative advantages to SCS support, in terms of expertise, standards/principles The mix of assets used by UNICEF in Albania to for Document review Project documents of UNICEF and other international stakeholders value and comparative advantages? worldwide, vision, resources, and assets in the country? development of normative framework as compared to Thematic case studies Agreement documents with stakeholders How did UNICEF mobilized these advantages to influence the stakeholders? traditionally used/available assets. Semi-structured interviews Informants’ responses How has UNICEF coordinated with key actors of SCS or connected areas? The progression shown in the outcomes of internal and Focus groups To what extent has UNICEF harnessed synergies external monitoring with other actors, in particular with the GoA’s The regularity, breadth and depth of cooperation and joint vision and theory of change, as well other planning with other actors. international actors and civil society initiatives? Factual evidence and perception of critical stumbling blocks in reform addressed through such collaboration. Impact What are the broader changes resulting from How has UNICEF influenced the compliance of the policy normative framework of SCS in Albania with Municipal case studies Informants’ responses and experiences (especially in sampled UNICEF’s contribution? key principles? Expectations expressed by stakeholders are self-reportedly Thematic case studies municipalities, and at national level) What institutional settings and behaviours have resulted from UNICEF’s contribution, and how increasing Document review compatible are they with UNICEF’s principles? Documentary and perceptive evidence of child care services Survey Can the contribution of UNICEF-supported policy adequately integrated within social care services Semi-structured interviews normative framework to changes in boys, girls, Prioritization of social care services in government’s strategic Focus groups women and men’s access to SCS as rights documents holders, be traced? Traceable objective change in access to/quality of social services received by targeted categories Sustainability How likely is it that UNICEF’s contribution to the What are the conditions for this impact to last? Perceived subjective change in access to/quality of social Municipal case studies Applicable policy normative framework policy normative framework of SCS in Albania will How likely are these conditions to last? services Thematic case studies Draft policy framework, in particular legislation, if any last in the long run? What can UNICEF and its partners do to ensure lasting impact? Document review Public reports of relevant CSOs, NHRIs, international organisations, as Survey well as Albania’s reports and shadow reports to the monitoring bodies of Semi-structured interviews relevant international treaties and standards Focus groups Responses of informants International expertise/ peer review Written input and interviews with two international social protection experts
36 EVALUATION EVALUATION QUESTIONS SUB-QUESTIONS INDICATOR DATA COLLECTION METHODS DATA SOURCES CRITERIA Relevance, added To what extent did UNICEF’s theory of change How coherent are UNICEF’s theory of change, and implemented action, with its guiding principles, its Regularity of UNICEF Albania communication with and contribution Document review UNICEF and GoA strategic and programmatic documents. Applicable treaties, value match UNICEF’s priorities and standards, and overall strategic documents and with the recommendations of relevant treaty bodies? to GoA policy, regulatory and legislative drafting Survey EU agreements, and other commitments of Albania. Treaty bodies’ reports, Albania’s priorities and vision? What were UNICEF’s and the GoA’s approaches for the policy normative framework of SCS reform? Recurrence of common terminology between UNICEF country reports, shadow reports. How aligned were these approaches with each other, and with identified needs? organization-wide strategic guidance, and UNICEF in Albania Semi-structured interviews Survey responses strategic documents Responses of complementary informants Evidence of social cleavages articulated in international (WB, Written input and interviews with UNICEF social protection experts and UNDP, UNICEF) and national studies (governmental and CSO reports). international expert on SCS Focus group with policy makers To what extent has UNICEF’s theory of change Has UNICEF contributed to policy, legislation, regulations and institutions with sufficient and quality Treaty bodies’ reports, country reports, shadow reports. addressed the need for, and barriers to the social analysis of barriers to social? Dynamics of poverty and inequality indicators as evidenced by Survey responses inclusion children and families? To what extent have UNICEF’s interventions in support of the normative policy framework contributed UNICEF and other international and national reports. Responses of complementary informants to mainstreaming and targeting the needs of the children and families in Albania? Programming documents International expertise/ peer review Monitoring reports Written input and interviews with two international social protection Thematic and municipal case studies experts How relevant is UNICEF’s Theory of Change To what extent has UNICEF’s action been coherent with its theory of change? The degree of alignment of the priorities of UNICEF in Albania UNICEF guidelines for interventions in middle-income countries within the current action to the emerging needs How has UNICEF’s theory of change evolved over time? with national priorities as articulated in national strategic Treaty bodies’ reports, country reports, shadow reports. and priorities of Albania? What are the possible scenarios to optimize this relevance in the long run? documents National strategic documents The perception of alignment between the national and UNICEF EU strategic documents in Albania priorities by decision makers and stakeholders. Survey responses Responses of complementary informants Programming documents Monitoring reports Effectiveness What were the results of UNICEF’s action in terms Was UNICEF’s action instrumental in setting a policy agenda for social care services’ reform and if so Self-reported understanding by stakeholders of the building Document review Applicable policy normative framework of establishment of building blocks of the SCS in what way? blocks as constitutive of a systems approach. Focus groups International standards system? To what extent was UNICEF’s contribution to policy development effective in mainstreaming vulnerable Recurrence of key terminology and constitutive elements of the Semi-structured interviews Projects’ documentation groups and their needs (e.g. gender inequalities and inequity in service delivery, particularly for children SCS system, both in UNICEF strategic documents and in the International expertise/ peer review Projects’ publications and families)? normative policy framework as adopted by Albanian authorities Thematic and municipal case studies Proceeds of events organized by the projects. Was UNICEF’s action instrumental to the design and /or adoption of the policy normative framework of Recurrence of semantic similarities between UNICEF guidance Informants’ responses and experiences To what extent has UNICEF succeeded in SCS, and if so in what way? and/or communication documents, and the normative policy Written input and interviews with two international social protection mainstreaming of UNICEF principles? How has UNICEF mainstreamed its key expertise and principles for SCS during the design and/r framework experts adoption of the policy normative framework of SCS? Albania’s normative and policy framework provide adequate resources for meeting commitments on social care services Efficiency, added To what extent has UNICEF optimized its assets What are UNICEF’s comparative advantages to SCS support, in terms of expertise, standards/principles The mix of assets used by UNICEF in Albania to for Document review Project documents of UNICEF and other international stakeholders value and comparative advantages? worldwide, vision, resources, and assets in the country? development of normative framework as compared to Thematic case studies Agreement documents with stakeholders How did UNICEF mobilized these advantages to influence the stakeholders? traditionally used/available assets. Semi-structured interviews Informants’ responses How has UNICEF coordinated with key actors of SCS or connected areas? The progression shown in the outcomes of internal and Focus groups To what extent has UNICEF harnessed synergies external monitoring with other actors, in particular with the GoA’s The regularity, breadth and depth of cooperation and joint vision and theory of change, as well other planning with other actors. international actors and civil society initiatives? Factual evidence and perception of critical stumbling blocks in reform addressed through such collaboration. Impact What are the broader changes resulting from How has UNICEF influenced the compliance of the policy normative framework of SCS in Albania with Municipal case studies Informants’ responses and experiences (especially in sampled UNICEF’s contribution? key principles? Expectations expressed by stakeholders are self-reportedly Thematic case studies municipalities, and at national level) What institutional settings and behaviours have resulted from UNICEF’s contribution, and how increasing Document review compatible are they with UNICEF’s principles? Documentary and perceptive evidence of child care services Survey Can the contribution of UNICEF-supported policy adequately integrated within social care services Semi-structured interviews normative framework to changes in boys, girls, Prioritization of social care services in government’s strategic Focus groups women and men’s access to SCS as rights documents holders, be traced? Traceable objective change in access to/quality of social services received by targeted categories Sustainability How likely is it that UNICEF’s contribution to the What are the conditions for this impact to last? Perceived subjective change in access to/quality of social Municipal case studies Applicable policy normative framework policy normative framework of SCS in Albania will How likely are these conditions to last? services Thematic case studies Draft policy framework, in particular legislation, if any last in the long run? What can UNICEF and its partners do to ensure lasting impact? Document review Public reports of relevant CSOs, NHRIs, international organisations, as Survey well as Albania’s reports and shadow reports to the monitoring bodies of Semi-structured interviews relevant international treaties and standards Focus groups Responses of informants International expertise/ peer review Written input and interviews with two international social protection experts
UNICEF in Albania ANNEXES TO THE EVALUATION REPORT 37 ANNEX ETHICAL 5 CONSIDERATIONS
Letter of Approval from the Ethics Review Board
Research Ethics Approval
23 August 2019
Elda Hallkaj Monitoring & Evaluation / Childs Rights Monitoring Officer United Nations Children's Fund (UNICEF) Str. "Skenderbej", UN House building, 3rd floor Tirana, Albania
RE: Ethics Review Board findings for: Evaluation of UNICEF's Contribution to the Normative Policy Framework of the Social Care Services Reform 2013 -- 2019 in Albania
Dear Ms. Hallkaj,
Protocols for the protection of human subjects in the above study were assessed through a research ethics review by HML Institutional Review Board on 05 – 23 August 2019.
This study’s human subjects’ protection protocols, as stated in the materials submitted, received ethics review approval. Please notify this IRB of any changes in this study’s design, risks, consent, or other human subject protection protocols.
Sincerely,
D. Michael Anderson, Ph.D., MPH Chair & Human Subjects Protections Director, HML IRB
cc: Camille Massey, Merita Poni, Penelope Lantz, JD
HML Institutional Review Board 1101 Connecticut Avenue, NW Suite 450 Washington, DC 20036 USA +1.202.753.5040 [email protected] www.hmlirb.com US Department of Health & Human Services, Office of Human Research Protections IRB #00001211
38 ETHICAL CONSIDERATIONS FOR DATA COLLECTION
In this section are described the ethical standards / principles applied during data collection.
DOCUMENT REVIEW The evaluation team reviewed three types of documents, and adjusted its process to each type of document in order to fulfil ethical standards:
♦♦ Publicly available documents (e.g. legislation or by-laws, national strategies, UN system guidelines…). These documents did not pose any particular ethical questions, beyond standard evaluation due diligence in analysing them. ♦♦ Internal documents (UNICEF-internal, such as project documents and reports, or correspondence; and internal to the MoSWY, such as administrative instructions or detailed organigram). Two ethical considerations have been identified with regards to these documents: confidentiality, and potential conflicts of interest. ‐‐ Regarding confidentiality, these documents were only quoted upon authorization from their proprietor, and safely stored in a password-protected drive for the duration of the evaluation and will be completely deleted upon completion of the evaluation. ‐‐ Some of these documents could only be identified, located and extracted by the action’s staff, as was already the case during the inception phase. This situation is quite commonplace in evaluations, but created a level of dependency of the evaluation team towards the action’s staff, for access to data – which in turn created a perception of limitation of the evaluation team’s independence, and of conflict of interest of the staff acting both as evaluees and as providers of documentation. The evaluation team however believes that, owing to the purposes of the evaluation (all geared towards usefulness) acting as strong motivator for the staff to obtain a truthful evaluation, to the diligence of the staff concerned, and to the trust-based approach to evaluation which has been fostered throughout the inception phase, there was no real concern on this front. ♦♦ For the case studies, the evaluation team reviewed small samples of locally-produced documentation on social care services (e.g. municipal social plans, forms and templates, sample of decisions/notifications). In order to protect the right to privacy, as well as data protection rights and legislation, the evaluation team requested these documents to be cleared of names and only shown (not handed over) to them. The evaluation team did not take or keep any copy of these documents, whether hard or electronic.
Interviews/focus groups with stakeholders within the UN system or from the donor’s administration and with Albanian SCS stakeholders at national and municipal level (relevant Ministries, states services and NARU)
The evaluation followed the UNEG Norms and Standards as well as the UNEG Ethical Guidelines for Evaluation. The evaluation team applied human rights and gender equality-sensitive processes and maximise stakeholders’ participation in the evaluation, in particular through these interviews. The evaluation team paid particular attention to the gender distribution of interviewees, to ensure equal participation of men and women, as much as possible. These interviews only took place on a voluntary basis. Given the fact that they did not involve persons who could be perceived as vulnerable, and only officials in their professional capacity, no informed consent form was required.
However, participants may have perceived that some opinions or information shared during interviews and focus groups could expose the informants to repercussions – or could simply be sensitive. Interviews were therefore held in confidence, focus groups operated under “Chatham House rules”. Interviews did not put together supervisor and supervisee. Focus groups’ notes and interview notes were only shared within the evaluation team, and not further. They will be deleted/destroyed upon completion of the evaluation. Quotes maintained in data analysis tools, or mentioned in the reports, were anonymized.
UNICEF in Albania ANNEXES TO THE EVALUATION REPORT 39 Interviews with SCS beneficiaries at municipal level (within the three municipal case studies)
The ethical standards foresaw an ethical review board for “Evidence generation that involves vulnerable cohorts whose personal agency is limited due to age, situation or capabilities and for whom an additional duty of care is required. This includes research, evaluation and data collection and analysis that undertakes primary data collection and: (…) Targets and involves a group that may be perceived as vulnerable within the local context(examples include; women, minority groups, persons with HIV/AIDS, the economically and educationally disadvantaged, persons in institutions) as participants.” The interviews with SCS beneficiaries, although they did not aim to generate data on their possible vulnerabilities, did involve primary data collection with people whose needs for SCS may relate to vulnerabilities, such as, for instance, an economic disadvantage, single parenthood, etc.
These interviews were envisaged because they are the only way to gather, without risk of conflict of interest, evidence on how the mainstreaming of UNICEF’s principles into the SCS system affects SCS beneficiaries. It was viewed as an indispensable data collection mechanism to answer impact questions, and ensure the principle of justice, outlined above.
These interviews were confidential and anonymous. Anonymous, non-attributable quotes were used in the final evaluation report, if a statement of an interviewee was representative of a widely shared opinion among SCS beneficiaries interviewed. Upon permission of the interviewees, interview notes were taken verbatim on computer during the interviews, to ensure maximum fidelity without resorting to audio or video recording which may be viewed as more intimidating. These notes were not shared beyond the evaluation team (International and National Consultant). They were stored in a password-protected Drive accessible only to the consultants, only for the duration of the evaluation. They will be deleted permanently once the evaluation contract comes to an end.
The interviews did not seek to gather private data, but instead the opinion, ideas, and recommendations of the interviewees based on their experience with SCS.
The protocol, aimed to minimize stress or inconvenience, as well as to protect privacy, confidentiality and ensure respect for the interviewees as autonomous agents, was as followed:
♦♦ Both consultants requested permission to spend one day, in the respective municipalities sampled for case studies, within one or more of the SCS facilities receiving beneficiaries in the sampled municipalities (community center, or NARU within the municipal administration). In the pilot municipality, they attended this occasion jointly. On these occasions, they were supplied by UNICEF with small snacks and non-alcoholic refreshments for themselves and interviewees. ♦♦ Adult beneficiaries visiting the facility were offered orally, by the relevant consultant introduced by the SCS professional, to take the interview as they complete their initial endeavor with the SCS. This offer specified: ‐‐ The name and function of the UNICEF consultant and of the interpreter if applicable (with an emphasis on confidentiality, independence and impartiality), ‐‐ The statement of the purpose of the evaluation (a learning exercise meant to ensure UNICEF delivers the best support possible to SCS in Albania and elsewhere), ‐‐ The statement of the purpose of the interview (understanding how the SCS, and in particular its key principles as championed by UNICEF, affect its beneficiaries; making sure the voice, opinions and ideas of the SCS beneficiaries are heard within the evaluation), ‐‐ The complete anonymity and confidentiality of the interview and its notes, ‐‐ The logistical aspects of the interview (duration, use of simultaneous interpretation as may be required, children care by SCS if applicable, provision of refreshment),
40 ‐‐ The absence of financial compensation, ‐‐ The fact that the interview is fully voluntary, and that the person(s) is/are entirely free to accept or refuse to take it, without any consequences for themselves or others; ‐‐ That if they have any complaints or further questions, they may contact UNICEF country Office – the Office’s generic phone number and email address were given to them in writing: in this way, even persons who declined the interview and therefore did not receive any Informed Consent form, still had an opportunity to bring any complaint forward. ♦♦ Interested interviewees were given a written Informed Consent form to read and, if agreed to, sign.14 ♦♦ The interviews only involved adults. If they were accompanied by children at the time of their visit, SCS professionals invited the children to play while the interview was being held with their parent/caregiver. In facilities where these were not available already, the Evaluation Team provided basic sets for children to draw or play (paper, markers/crayons, stickers, figurines, books, supplied by UNICEF. The children were able to bring their artwork home). ♦♦ The interviewees were offered refreshments/snacks, but no financial compensation. This was to ensure that the interview was comfortable, but that interviewees did not experience it as work/as a paid duty, which may imply that they are expected to express certain views and not others about the SCS system and/or UNICEF. ♦♦ The length of interviews were kept to a minimum (30 minutes approximately). ♦♦ Interviews started with a short introduction, and gave the interviewee(s) an opportunity to ask any question about the process they were participating into. The interviewees were requested not to share any private data/confidential information. The interviewer(s) requested permission to take notes, but reiterate that these were anonymous, would not be shared beyond the consultants, and only serve as aide-memoire for the consultants, and possibly a source for anonymous quotes, if a statement was representative of a widely shared opinion among interviewees. ♦♦ The interviews and focus group followed the relevant interview questionnaires.15 ♦♦ Signed consent forms were collected by the evaluation and provided to UNICEF for safekeeping. UNICEF will follow its rules for protection of data, its storage and destruction. The evaluation team did not record identifiers – the views and opinions expressed in the interviews were not attributed to individual participants.
SURVEY WITH SCS PROFESSIONALS The online survey was anonymous. Identifying information of individual subjects (e.g., name, address, email address, etc.) were not collected. Surveys were anonymous and administered through Survey Monkey. The invitation to participate including links to the SurveyMonkey were sent to the respondents via email by the MoSWY, but MoSWY or UNICEF staff did not have access to the answers provided through Survey Monkey website. Nobody inside or outside of the project was able to connect individual subjects with their responses. The surveys’ electronic tables and results summarized information at the aggregate level, so that UNICEF or outside parties were not able to access identifiable information. Draft electronic tables were protected with individual passwords. Once the survey was closed, the individual answers were deleted.
14. Annex 6 15. Annex 6
UNICEF in Albania ANNEXES TO THE EVALUATION REPORT 41 ANNEX PRIMARY DATA 6 COLLECTION TOOLS
INTERVIEW TEMPLATE FOR ALBANIAN (NATIONAL LEVEL) STAKEHOLDERS
Interviewee(s)
Function(s)
Date of the meeting
Interviewer(s)
Location
In confidence/shareable
INTRODUCTION
♦♦ Presentation of the project title, timeline and dates, and framework ♦♦ Self-presentation by expert (name, function; specify that independent expert hired to assess the results of UNICEF’s contribution; not UNICEF staff) ♦♦ Purpose of the interview: to understand UNICEF’s contribution to the policy, legal, regulatory and institutional framework of SCS in Albania, what has worked well, what has not worked well, and how to improve upon this in the future ♦♦ The exercise is NOT an evaluation of any particular person’s individual performance; it is NOT a control, NOT an audit, but, rather, a learning exercise. ♦♦ Of great need and value is the interviewee’s personal experience and opinion. They are encouraged to speak in a personal capacity, not as a representative of an institution. ♦♦ The process ensures anonymity. Interview notes will not be shared and the results will be synthesised into a general evaluation report, without attributable statements. The interviewer must request authorisation to take notes on a computer.
The following are some examples of questions to be asked: ♦♦ What was your involvement in the action? ♦♦ How relevant do you think the action was? Was its focus on legislative, regulatory and institutional changes relevant to the situation of social care services? ♦♦ What did the action intend to change in this situation, and how?
42 ♦♦ For you, what were the key phases of the project? And what was its story? ♦♦ What are the main outcomes of the action, in your opinion? ♦♦ If you could change something in the topic, scope, content or organisation of UNICEF’s action, what would it be? ♦♦ Any additional comments or ideas?
INTERVIEW TEMPLATE FOR INTERNATIONAL STAKEHOLDERS AND CSOS
Interviewee(s)
Function(s)
Date of the meeting
Interviewer(s)
Location
In confidence/shareable
INTRODUCTION
♦♦ Presentation of the project title, timeline and dates, and framework ♦♦ Self-presentation by expert (name, function; specify that independent expert hired to assess the results of UNICEF’s contribution; not UNICEF staff) ♦♦ Purpose of the interview: to understand UNICEF’s contribution to the policy, legal, regulatory and institutional framework of SCS in Albania, what was useful, what worked well, what did not work well, and how to improve upon this in the future. ♦♦ The exercise is NOT an evaluation of any particular person’s individual performance; it is NOT a control, NOT an audit, but, rather, a learning exercise. ♦♦ Of great need and value is the interviewee’s personal experience and opinion. They are encouraged to speak in a personal capacity, not as a representative of an institution. ♦♦ The process ensures anonymity. Interview notes will not be shared and the results will be synthesised into a general evaluation report, without attributable statements. The interviewer must request authorisation to take notes on a computer.
What was your involvement in the action? Since when have you been involved in cooperation with UNICEF over social care services? In which capacity, or capacities in the event your functions have changed over time? What were the key activities and key topics of the two programmes in which you participated?
How relevant do you think the action was? Was its focus on legislative, regulatory and institutional changes relevant to the situation of social care services? How would you describe the situation of social care services before the action started? What needed to be done? What was the vision of your institution for social care services at that time? Your vision?
What did the action intend to change in this situation, and how? What did your institution expect from UNICEF at the start of the action? What were your expectations, personally, of UNICEF?
UNICEF in Albania ANNEXES TO THE EVALUATION REPORT 43 For you, what were the key phases of the action? And what was its story? Please describe the key moments in the reform, and the role played by UNICEF at that time. Please describe the most meaningful activities conducted by UNICEF over the period (events or publications that have had a major outcome, or a move made by UNICEF that was particularly important in your eyes).
What are the main outcomes of the action, in your opinion? What did this programme change for your interlocutors in the relevant Albanian institutions, as far as you can observe? What do you think of the National Strategy? Of the Law? If you participated in the drafting process of these documents, do you remember some aspects that UNICEF pushed for, which were not initially part of the discussions? Were any actors included in the consultations for these documents thanks to UNICEF’s advice? As far as you can observe, what are the obstacles in the implementation of the Law, the relevant regulations and the Strategy, and could they have been anticipated or fixed?
If you could change something with regard to the topic, scope, content or organisation of the action, what would it be? Do you remember an activity, a move, or a position of UNICEF that you thought was not helpful enough throughout these programmes? Was UNICEF a predictable partner or reliable actor? Were UNICEF’s positions and discourse coherent and meaningful? Would it be more effective for UNICEF to work through other actors (NGOs for example)?
Any additional comments or ideas?
INTERVIEW TEMPLATE FOR ALBANIAN (MUNICIPAL LEVEL) STAKEHOLDERS
Interviewee(s)
Function(s)
Date of the meeting
Interviewer(s) Location In confidence/shareable
INTRODUCTION
♦♦ Presentation of the project title, timeline and dates, and framework ♦♦ Self-presentation by expert (name, function; specify that independent expert hired to assess the results of UNICEF’s contribution; not UNICEF staff) ♦♦ Purpose of the interview: to understand UNICEF’s contribution to the policy, legal, regulatory and institutional framework of SCS in Albania, what has worked well, what has not worked well, and how to improve upon this in the future ♦♦ The exercise is NOT an evaluation of any particular person’s individual performance; it is NOT a control, NOT an audit, but, rather, a learning exercise. ♦♦ Of great need and value is the interviewee’s personal experience and opinion. They are encouraged to speak in a personal capacity, not as a representative of an institution. ♦♦ The process ensures anonymity. Interview notes will not be shared and the results will be synthesised into a general evaluation report, without attributable statements. The interviewer must request authorisation to take notes on a computer.
44 What has been your role so far in SCS provision? How familiar would you say you are with the legal, regulatory and institutional framework on SCS? In particular, what trainings, briefings or other learning opportunities have you had in this regard, and how would you assess their usefulness? From your perception, what are the key principles and the main approaches adopted by this framework? What key principles guide you in your work and your decision making? Among your SCS beneficiaries (direct or indirect), what proportion are children (persons under 18 years of age)? How conducive is the framework for you to provide them with sufficient, quality SCS? How easy or difficult is this framework to apply for you as a professional? Are there issues of compatibility among the various laws, by-laws or instructions? Do you have the necessary resources, infrastructure or means to implement this framework? What could be done to improve this situation, if necessary? In your opinion and from your personal experience, what has this framework changed in the provision of SCS in your municipality? To the situation of the beneficiaries? What feedback have you received from beneficiaries? Any additional comments or ideas?
INTERVIEW GUIDELINES FOR SCS BENEFICIARIES
Interviewee(s)
Function(s)
Date of the meeting
Interviewer(s)
Location
In confidence/shareable
INTRODUCTION
♦♦ Presentation of the project title, timeline and dates, and framework ♦♦ Self-presentation by expert (name, function; specify that independent expert hired to assess the results of UNICEF’s contribution; not UNICEF staff) ♦♦ Purpose of the interview: to understand UNICEF’s contribution to the policy, legal, regulatory and institutional framework of SCS in Albania, what has worked well, what has not worked well, and how to improve upon this in the future. ♦♦ The exercise is NOT an evaluation of any particular person’s individual performance; it is NOT a control, NOT an audit, but, rather, a learning exercise. There will be no consequences for any individual. ♦♦ Of great need and value is the interviewee’s personal experience and opinion; they are encouraged to speak freely. ♦♦ The process ensures anonymity. Interview notes will not be shared and the results will be synthesised into a general evaluation report, without attributable statements. The interviewer must request authorisation to take notes on computer.
For how long have you benefitted from SCS? What kind of services do you receive? How did you come to benefit from them? Was it mostly at your own initiative, or somebody
UNICEF in Albania ANNEXES TO THE EVALUATION REPORT 45 else’s? How easy or difficult was it to arrange? What were the main obstacles, if any? Overall, how satisfied would you say you are with the services you have received? How suited are they to your needs? Are they as you expected? In your opinion, how clear are your SCS rights and entitlements? And the rules around them? In your view, what is the Needs Assessment and Referral Unit (NARU) trying to achieve with its offer of services; i.e. what is the reason? Any additional comments or ideas?
GUIDELINES FOR FOCUS GROUPS WITH SCS REFORM DECISION-MAKERS
Participant(s)
Function(s)
Date of the meeting
Facilitator
Notes taker
Location
In confidence/shareable
INTRODUCTION
♦♦ Presentation of the project title, timeline and dates, and framework ♦♦ Self-presentation by expert (name, function; specify that independent expert hired to assess the results of UNICEF’s contribution; not UNICEF staff) ♦♦ Purpose of the interview: to understand UNICEF’s contribution to the policy, legal, regulatory and institutional framework of SCS in Albania, what has worked well, what has not worked well, and how to improve upon this in the future ♦♦ The exercise is NOT an evaluation of any particular person’s individual performance; it is NOT a control, NOT an audit, but, rather, a learning exercise. ♦♦ Of great need and value is the interviewee’s personal experience and opinion. They are encouraged to speak in a personal capacity, not as a representative of an institution. ♦♦ The process ensures anonymity. Interview notes will not be shared and the results will be synthesised into a general evaluation report, without attributable statements. The interviewer must request authorisation to take notes on a computer.
What was the original vision of the SCS system in your understanding as key actors, when the reform started?
Guiding sub-questions: ♦♦ How would you characterise the situation regarding the need for SCS in 2012, and the services that were available back then? ♦♦ What was your intention when you started working on SCS reform? How did you approach this situation, and how did you address it? ♦♦ What were your objectives and priorities as a group of policymakers and decision makers? Your key principles?
46 ♦♦ What were the key components of the SCS system you intended to build? Were they to be introduced in any particular order, and if so why? ♦♦ What was UNICEF’s contribution to defining the Normative Policy Framework of the SCS system, especially in terms of guiding principles?
Guiding sub-questions: ♦♦ At what stage did UNICEF become involved in the SCS reform? And how did it become involved at that stage? ♦♦ What were the key messages of UNICEF at the time? What did UNICEF advocate for in terms of objectives and priorities? In terms of key principles? ♦♦ How did UNICEF communicate these to you? Through which activities or inputs? How relevant or useful did you think these activities and inputs were? ♦♦ Which aspects of the SCS reform and of the key documents (e.g. the National Strategy on Social Protection, the Law on Social Protection, some of the by-laws) did you define jointly with UNICEF, if any? What influence did UNICEF have on your own approach? ♦♦ What other actors were involved at that time? What were their key messages, priorities and principles?
What has been the impact and sustainability of the reform in terms of system building? Guiding sub-questions: ♦♦ How satisfied are you with the SCS system as it stands now? What feedback have you receive from the NARUs and the State Social Service professionals regarding the Normative Policy Framework? ♦♦ What needs to be added, refined or changed, if anything? What is the next ‘frontier’? ♦♦ Any additional comments or ideas?
UNICEF in Albania ANNEXES TO THE EVALUATION REPORT 47 ANNEX LIST OF DATA COLLECTION SITES AND 7 INTERVIEWS/FOCUS GROUPS
♦♦ UNICEF Office Albania (7 people) ♦♦ UNICEF Regional Office, (1 person) ♦♦ United Nations Albania (1 person) ♦♦ UNDP Office Albania (3 people) ♦♦ Swiss Development Agency SDA, Swiss Embassy (1 person) ♦♦ EU Delegation Albania (1 person) ♦♦ Ministry of Finance (3 people) ♦♦ Department of Development Programme For Health and Social Protection, MoHSP (4 people) ♦♦ Peoples’ Advocate (3 people) ♦♦ Member of Parliament, Parliament/Social Affairs Com (2 people) ♦♦ Inspectorate of Labour and Social Services (1 person) ♦♦ State Agency for Children (1 person) ♦♦ State Social Service Gjirokaster (1 person) ♦♦ Tirana Municipality (2 people) ♦♦ Malsesi e Madhe Municipality (9 people) ♦♦ Durres Municipality (9 people) ♦♦ Permet municipality (8 people) ♦♦ University of Tirana (4 people) ♦♦ UNICEF expert, Euro Partners (1 person) ♦♦ UNICEF expert/social fund, freelance (1 person) ♦♦ Expert on SCS/former SSS Head, freelance (1 person) ♦♦ Agency for support to LGUs/MoI (1 person) ♦♦ ESA consulting (1 person) ♦♦ Bethany Christian Services Albania (1 person) ♦♦ World Vision Albania (2 people) ♦♦ Save the Children Albania (2 people) ♦♦ Terre des Hommes (1 person) ♦♦ Solidaritet Now (1 person)
48 ♦♦ World Vision/Durres (1 person) ♦♦ Multifunctional Day Center Nish Tulla/Durres Municipality (3 beneficiaries; staff) ♦♦ Disabled Children center (1 person) ♦♦ Disabled Children center (9 beneficiaries) ♦♦ Community Center for children with disabilities (1 beneficiary) ♦♦ Center for mental health for children (1 beneficiary) ♦♦ Community Center for children with disabilities (2 people) ♦♦ 1 Parent of child with Down syndrome ♦♦ SSS Shkoder (1 person) ♦♦ Municipality center for children with disabilities (focus group 8 people and 6 separate interviews)
LIST OF SITE VISITS:
♦♦ 16-18 September 2019, Tirana (evaluation team) ♦♦ 19-20 September 2019, Dures (evaluation team) ♦♦ 23-24 September 2019, Permet (national consultant) and Malesia et Made (international consultant) ♦♦ 25-27 September 2019, Tirana (evaluation team)
UNICEF in Albania ANNEXES TO THE EVALUATION REPORT 49 ANNEX 8 SURVEY RESULTS The survey with SCS professionals was designed and administered in SurveyMonkey (TM), in Albanian language, by the evaluation team. The questionnaire was elaborated based on insight gained through initial interviews and document review, to look into the gender and academic composition of the social work policy makers and practitioners in Albania; their knowledge, understanding and appreciation of the normative policy framework on social care services; their experience and appreciation of UNICEF’s contribution to this framework; and their aspirations for social care services in the future. These areas of inquiry were validated with UNICEF staff and during a focus group with academia and policy makers. The survey questionnaire comprised common core questions and two subsets of questions for policy makers and field practitioners respectively. In average, it took respondents six and a half minutes to complete. The survey was sent to all social workers registered by the Ministry of Health and Social Protection and Ministry social protection staff, complemented by social workers registered with the Social Workers’ Association of Albania (with elimination of double entries). It was open from 26 December 2019 to 26 March 2020 (with almost all responses provided in December-January), and was answered by 128 persons out of 219 addressees, representing approximately a 58% response rate, which is significantly more than usual for similar surveys, especially considering the end of year period and earthquake related challenges.
QUESTIONS RELEVANT FOR ALL RESPONDENTS
Question: You are… Question: Which is your current employer? Answered: 128 Skipped: 0 Answered: 128 Skipped: 0
A municipality 67.19% 11.72% Another state/gov entity 15.63% Female Male An Albanian NGO 8.59% Other 4.69% 88.28% An international NGO 3.91%
Question: What is your main qualification? Question: Have you been involved in the design of the national Answered: 128 Skipped: 0 policy, legislation or bylaws regarding social care services? Answered: 128 Skipped: 0 Answer choices Responses Social work master 46.09% 2.34% Other 19.53% Psychology diploma 15.63% 32.81% Yes Economics diploma 7.81% No Social work bachelor 4.69% I do not want to 64.84% disclouse Law diploma 3.13% Political science diploma 1.56% General secondary school diploma 1.56% Medical diploma 0 General high school diploma 0
50 Question: How long have you been involved with issues related to social care services? Answered: 128 Skipped: 0
Answer choices Responses 3 to 5 years 28.91% 3 to 5 years 28.9% 6 to 10 years 24.22% 6 to 10 years 24.22% 1 to 2 years 22.66% 1 to 2 years 22.66% 11 years or more 14.06% Less than 1 year 10.16% 11 years or more 14.06% I do not know 0 Less than 1 year 10.16% I do not want to disclose 0
Question: In which institution have you worked the longest during the past 10 years? Answered: 128 Skipped: 0
Answer choices Responses A municipal social care service, within the 45.31% municipal administration Other 17.97% A municipality social care services 45.31% None 7.03% Other 17.97% The Ministry of Health and Social 6.25% Protection None 7.03% The State Social Services at national or The Ministry of Health and 5.47% Social Protection 6.25% regional level The State Social Services An NGO providing social care services 4.69% 5.47% A residential institution 3.91% An NGO providing social 4.69% care services An institution for day-only social care 3.13% A residential institution services 3.91% An NGO interested in social care services, but not providing such services directly to 3.13% beneficiaries Academia 3.13% Private consultant 0
Question: What would you say are the top 5 challenges you experience in the exercise of your functions? Answered: 30 Skipped: 98
Answer choices Responses The lack of funds at all levels 73.33% The lack of clear policy direction or principles 66.67% The lack of funds at municipal level 63.33% The lack of funds at national level 46.67% The lack of political will 46.67% The lack of political will at municipal level 43.33% The lack of political will at national level 36.67% The lack of understanding of the concept of social care services among officials 33.33% The absence of legal or regulatory framework to regulate all aspects of social care services 30.00% The lack of clarity of the existing legal and regulatory framework 26.67% The lack of awareness about entitlements to social services among beneficiaries 26.67% Other 6.67%
UNICEF in Albania ANNEXES TO THE EVALUATION REPORT 51 Question: According to you, what are the top 5 objectives of the social care services system in Albania? Answered: 30 Skipped: 98
Answer choices Responses To enhance social inclusion 91.30% To target the most vulnerable categories of 69.57% citizens To enhance To assist children as directly as possible 60.87% social inclusion 91.30% To combat poverty 58.70% To target the most vulnerable To make sure no citizen is “left behind” 47.83% categories of citizens 69.57%17.97% To address the needs of citizens without placing 43.48% To assist children as directly them in residential institutions, if possible as possible 60.87% To assist the most disadvantaged as long as 34.78% possible To comb at poverty 58.70% To fulfil Albania’s international obligations 32.61% To help citizens live through moments of crisis 28.26% To make sure no citizen is “left behind” 47.83% To make sure that residential institutions can 19.57% accommodate the most vulnerable To keep families together 13.04% Other 0.00%
Question: How familiar would you say you are with the following concepts? Answered: 46 Skipped: 82
100 % 100 % 90 % 90 % 80 % 80 % 70 % 34.73 % 70 % 60 % 60 % 50 % 50 % 43.48 % 40 % 73.51 % 50.00 % 67.39 % 40 % 80.43 % 30 % 30 % 82.61 % 80.43 % 20 % 21.74 % 91.30 % 93.48 % 30.43 % 20 % 45.65% 91.30 % 10 % 10.87 % 10 % 0 % 4.35 % 4.35 % 8.70 % 6.52 % 2.17 % 0 % 19.57 % 10.87 % 17.39 % 19.57 % 8.70 % Accessibility Life-cycle approach Equity Accontability Deinstitutionalisation social care Innovation in Participation Ef ciency Sustainability Not to all A little Somewhat A lot services public services of citizens Not to all A little Somewhat A lot
Question: How familiar would you say you were 5 years ago with the following concepts? Answered: 46 Skipped: 82
Deinstitutionalisation 10.87 % 30.43 % 26.09 % 32.61 %
Accountability 10.87 % 32.61 % 56.52%
Equity 13.04 % 23.91% 63.04 %
Life-cycle approach 19.57 % 28.26 % 41.30 % 10.87 %
Accessibility 8.70 % 23.91 % 34.78 % 32.61 %
Sustainability 13.04 % 28.26 % 58.70 %
Ef ciency 4.35 % 19.57 % 26.09 % 50.00 %
Participation of citizens 19.57% 39.13 % 41.30 %
Innovation in public services 10.87% 39.13% 43.48% 6.52%
Social care services 2.17% 26.09% 41.30% 30.43%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Not to all A little Somewhat A lot
52 Question: Which of these concept would you say are entrenched in the current social care services system? (several answers possible) Answered: 46 Skipped: 82
Answer choices Responses Social care services 69.57% Social care services 69.57% Participation of citizens 52.17% Participation Deinstitutionalisation 41.30% of citizens 52.17%17.97% Accountability 41.30% Deinstitution Accessibility 32.61% nalisation 41.30% Equity 21.74% Sustainability 19.57% Accountabillity 41.30% Innovation in public services 19.57% Efficiency 17.39% Accessibillity 32.61% Life-cycle approach 13.04%
QUESTIONS RELEVANT FOR SERVICE PROVIDERS
Question: Which types of services, if any, have you delivered over the past 3 years? Answered: 30 Skipped: 98
Answer Choices Responses Answer choices Responses Provision of meals/arranging for the provision of meals 66.67% Counsel on legal or administrative 13.33% matters Referral of (potential) beneficiaries to relevant 56.67% institutions Parenting advice 13.33% Intervention with other relevant institution for the Other 13.33% 53.33% fulfilment of the beneficiaries’ needs/rights Counsel on job seeking 10.00% Advice on/administrative process towards placement 43.33% Day-care for a person with special of child in a kincare 6.67% needs Provision of in-kind assistance 40.00% Provision of petty cash/extra 6.67% Assessment and approval of financial social financial assistance 33.33% entitlements foreseen by the law Speech defect therapy 3.33% Advice on/administrative process towards placement 33.33% Developmental care/therapy 3.33% in a residential institution Psychotherapy 0.00% Advice on/administrative process towards placement 30.00% of child in foster care Physiotherapy/orthopedic care 0.00% Psychological counsel/support 23.33% None of the above 0.00% Professional training, or referral to professional training 20.00% Advice to care givers on caring for a person with 16.67% special needs
Question: Over the past 3 years, how often have you provided direct services (as listed above) to the following categories of children (0-18 years old)? Answered: 30 Skipped: 98
Never 1 to 5 times per year 6 to 12 times per year 13 to 24 times per year 25 times or more 80.00 % 63.33 % 60.00 % 40.00 % 40.00 % 36.67 % 33.33 % 33.33 % 26.67 % 30.00 % 26.67 % 26.67 % 26.67 % 16.67 % 20.00 % 20.00% 20.00 % 20.00 % 13.33 % 10.00 % 13.33 % 3.33 % 6.67 % 10.00 % 6.67 % 6.67 % 3.33 % 6.67 % 0.00 % Children Children affected by Children with Children affected by Children affected by in general poverty dissabilities domestic violence traf ck in human beings
UNICEF in Albania ANNEXES TO THE EVALUATION REPORT 53 Never 1 to 5 times per year 6 to 12 times per year 13 to 24 times per year 25 times or more 93.33 % 100.00 % 76.67 % 80.00 % 60.00 % 60.00 % 36.67 % 36.67 % 40.00 % 33.35 % 30.00 % 23.33 % 23.33 % 20.00 % 16.67 % 20.00 % 10.00 % 10.00 % 10.00 % 6.67 % 3.33 % 6.67 % 3.33 % 0.00 % Children affected by Children affected by discrimination Children affected by discrimination Children affected by Orphan children forced labour on the grounds of belonging to on the grounds of belonging to other forms of abuse/violence a national minority a nsexual minority
Never 1 to 5 times per year 6 to 12 times per year 13 to 24 times per year 25 times or more 80.00 % 70.00 % 60.00 % 56.67 % 50.00 % 40.00 % 26.67 % 23.33 % 20.00 % 13.33 % 16.67 % 13.33 % 10.00 % 6.67 % 3.33 % 6.67 % 3.33 % 0.00 % Homeless children Underage parents Children with other types (mothers or fathers under 16) of vulnerabilities
Question: Over the past 3 years, how often have you provided direct services (as listed above) to the following categories of adults (18+ years old)? Answered: 30 Skipped: 98
Never 1 to 5 times per year 6 to 12 times per year 13 to 24 times per year 25 times or more 100.00 % 80.00 % 60.00 % 40.00 % 40.00 % 26.67 % 26.67 % 30.00 % 26.67 % 33.33 % 30.00 % 20.00 % 23.33 % 23.33 % 20.00 % 20.00 % 13.33 % 13.33 % 13.33 % 16.67 % 16.67 % 6.67 % 6.67 % 6.67 % 6.67 % 0.00 % Adults in general Adults affected by poverty Adults with dissabilities Adults with other phycological needs Never 1 to 5 times per year 6 to 12 times per year 13 to 24 times per year 25 times or more 100.00 % 80.00 % 60.00 % 46.67 % 43.33 % 33.33 % 36.67 % 40.00 % 26.67 % 26.67 % 26.67 % 20.00 % 23.33 % 16.67 % 16.67 % 16.67 % 20.00 % 13.33 % 10.00 % 10.00 % 10.00 % 10.00 % 3.33 % 6.67 % 3.33 % 0.00 % Elderly people Single parents Parents or care givers for Parents or care givers for children with special needs grown ups with special needs Never 1 to 5 times per year 6 to 12 times per year 13 to 24 times per year 25 times or more
100.00 % 80.00 % 80.00 % 80.00 % 70.00 % 60.00 % 40.00 % 30.00 % 33.33 % 20.00 % 16.67 % 20.00 % 13.33 %10.00 %13.33 % 13.33 % 6.67 % 3.33 % 3.33 % 3.33 % 3.33 % 0.00 % Adults affected by Adults affected by Adults affected by Adults affected by house violence sexual violence human traf cking ethnic disrimination
93.33 % 100.00 % 73.33 % 80.00 % 56.67 % 60.00 % 43.33 % 40.00 % 23.33 % 23.33 % 16.67 % 20.00 % 13.33 % 10.00 % 10.00 % 10.00 % 3.33 % 3.33 % 6.67 % 6.67 % 6.67 % 0.00 % Adults affected by Adults affected by Homeless people Adults affected by due to sexual minorty other forms of abuse/violence other types of vulnerability
54 Question: If you are working at the municipal level, to the best of your knowledge, has your municipality conducted the following steps? Answered: 30 Skipped: 98 Answer Choices Responses Answer choices Responses Drafting of a municipal social plan 76.67% Adoption of specific municipal budget lines for the 26.67% fulfilment of the municipal social plan Municipal social needs assessment 63.33% Establishment of a full-fledged Needs Assessment and Establishment of a municipal social 26.67% 46.67% Referral Unit as defined in the Law on Social Care Services fund Take-over from an NGO, of a day-only care center for Costing of a municipal social plan 43.33% 23.33% certain categories of beneficiaries Adoption of a municipal social plan 43.33% I do not know 6.67% Funding a municipal social fund 33.33% Not applicable 6.67% Establishment of a day-only care Other 0.00% center for certain categories of 30.00% beneficiaries None of the above 0.00%
Question: If your municipality has conducted a municipal social Question: If your municipality has conducted a municipal social needs assessment/mapping, were you involved in this exercise? needs assessment/mapping, how would you assess the quality of this Answered: 30 Skipped: 98 exercise, on a scale from 1 (lowest quality) to 5 (highest quality)? Answered: 30 Skipped: 98
5 star 13.33% 10.00% 4 star 17.97% 23.33% Yes No 3 star 33.33% 56.67% 33.33% Not applicable 2 star 10.00% 1 star 20.00%
Question: If your municipality has drafted a municipal social plan, Question: If your municipality has drafted a municipal social plan, were you involved in this exercise? how would you assess the quality of this exercise, on a scale from Answered: 30 Skipped: 98 1 (lowest quality) to 5 (highest quality)? Answered: 30 Skipped: 98
5 star 10.00% 13.33% 4 star 17.97% 30.00% Yes No 3 star 23.33% 30.00% 56.67% Not applicable 2 star 6.67% 1 star 30.00%
Question: Please tick, from the list below, the legal or regulatory acts you have had a chance to read (several answers possible). Answered: 30 Skipped: 98 Answer choices Responses Law on Social Care Services 93.33% DCM (Nr. 518 of 04 September 2018) on “Obtaining community and residential services, criteria and procedures on the 76.67% amount of personal expenses of organised services” DCM (Nr.149 of 13 March 2018) on the criteria, documentation and procedures for identification of foster care families for 73.33% children without parental custody and the amount of funding for expenses related to the child care placed in foster families DCM (Nr. 394 of 09 May 2019) on the organization and functioning of the multidisciplinary commission and procedures 70.00% for implementation of the standards of alternative care services for children in foster care families DCM (Nr. 136 of 07 March 2018) on the functioning and administering of national electronic register of social care services 50.00% DCM (Nr. 111 of 23 February 2018) on the creation and functioning of social fund 46.67% DCM (Nr. 150 of 20 March 2019) On calculation of the funding of social care services 26.67% None 3.33%
UNICEF in Albania ANNEXES TO THE EVALUATION REPORT 55 QUESTIONS RELEVANT FOR NATIONAL LEVEL
Question: Please tick, from the list below, the legal or regulatory acts you have had a chance to read (several answers possible). Answered: 30 Skipped: 98
Answer choices Responses Law on Social Care Services 90.00% DCM (Nr. 518 of 04 September 2018) on “Obtaining community and residential services, criteria and procedures on 76.67% the amount of personal expenses of organised services” DCM (Nr.149 of 13 March 2018) on the criteria, documentation and procedures for identification of foster care families for 63.33% children without parental custody and the amount of funding for expenses related to the child care placed in foster families DCM (Nr. 394 of 09 May 2019) on the organization and functioning of the multidisciplinary commission and procedures 60.00% for implementation of the standards of alternative care services for children in foster care families DCM (Nr. 136 of 07 March 2018) on the functioning and administering of national electronic register of social care services 50.00% DCM (Nr. 111 of 23 February 2018) on the creation and functioning of social fund 50.00% DCM (Nr. 150 of 20 March 2019) On calculation of the funding of social care services 30.00% None 3.33% Other 3.33% Question: Please tick, from the list below, the legal or regulatory acts you have contributed to draft (several answers possible Answered: 30 Skipped: 98
Answer choices Responses None 50.00% Law on Social Care Services 40.00% DCM (Nr. 394 of 09 May 2019) on the organization and functioning of the multidisciplinary commission and 20.00% procedures for implementation of the standards of alternative care services for children in foster care families DCM (Nr. 518 of 04 September 2018) on “Obtaining community and residential services, criteria and procedures on 16.67% the amount of personal expenses of organised services” DCM (Nr. 111 of 23 February 2018) on the creation and functioning of social fund 16.67% DCM (Nr. 150 of 20 March 2019) On calculation of the funding of social care services 13.33% DCM (Nr. 136 of 07 March 2018) on the functioning and administering of national electronic register of social care services 10.00% DCM (Nr. 149 of 13 March 2018) on the criteria, documentation and procedures for identification of foster care families for children without parental custody and the amount of funding for expenses related to the child care placed 6.67% in foster families Other 3.33% Question: What would you say are the top 5 challenges you identify in the delivery of social care services? Answered: 30 Skipped: 98
Answer choices Responses The lack of political will at national level 70.00% The lack of funds at all levels 66.67% The lack of funds at national level 66.67% The absence of legal or regulatory framework to regulate all aspects of social care services 60.00% The lack of awareness about entitlements to social services among beneficiaries 56.67% The lack of political will at municipal level 53.33% The lack of political will 43.33% The lack of clarity of the existing legal and regulatory framework 33.33% The lack of understanding of the concept of social care services among officials 23.33% The lack of funds at municipal level 13.33% The lack of clear policy direction or principles 13.33% Other 0.00%
56 Question: How useful would you say UNICEF’s action in support of social care services has been? Answered: 30 Skipped: 98
100.00 %
80.00 %
60.00 %
40.00 % 40.00 % 33.33 %
20.00 % 16.67 % 6.67 % 3.33 % 0.00 % Not useful at all Somewhat useful Useful Very useful I do not know I do not want to respond
Question: In your opinion, what has been the added value, if any, of UNICEF in the development of the social care services system in Albania? (several responses possible) Answered: 30 Skipped: 98
Answer choices Responses It has provided expertise 46.67% It has provided It has caused decision makers to expertise 46.67% accelerate the process of developing the 43.33% system It has caused decision makers to accelerate the process It has helped put social care services on 43.33%17.97% 36.67% of developing the system the agenda of policy makers It has helped put social care It has helped put social care services on services on the agenda 33.33% 36.67% the agenda of policy makers of policity makers It has helped put social care It has channelled some basic principles services on the agenda 33.33% into the Government’s approach to social 33.33% of policity makers care services It has channelled some basic I do not know 13.33% principles into the Government’s 33.33% approach to social care services None of the above 10.00%
UNICEF in Albania ANNEXES TO THE EVALUATION REPORT 57 ANNEX MUNICIPAL CASE 9 STUDIES’ PROFILES
DURRES SCS MUNICIPALITY, 19 Jan 2020
SOCIAL PLAN Durres has a Social Plan 2017-2020 worked out by ESA Consulting, UNICEF. The plan is approved by the Municipal Council, costed but not budgeted. The cost of existing social protection is covered by municipal funds and government funds, through conditional and unconditional transfers.
DEMOGRAPHIC DATA According to Census, 2011 in Durrës, there are 175,110 inhabitants, whilst according to Civil Registration Office, there are 299,989 inhabitants. The new municipality occupies a territory of 338.30 km2 with a density of population of 517 inhabitants/km2. In accordance with the law Nr.115/201416, the municipality is composed of 6 administrative units: Durrës, Sukth, Ishëm, Katund i Ri, Rrashbull and Manëz17. Durres municipality administers three cities: Durrës, Sukth and Manëz and 39 villages. Durres municipality is part of Durres Region. In 2015, Durres municipality was increased with 5 new administrative units, that before were small municipalities or communes: Sukth, Ishem, Katundi i Ri, Rrashbulli and Manëz18. The new administrative units increased the number of inhabitants and administrative burden.
16. Law Nr.115/2014 on the Administrative Territorial Division of the Local Government Units in Republic of Albania, official gazette Nr. 137, date 2014.URL: http://www.vendime.al/ligj-nr-1152014-per-ndarjen-administrativo-territoriale-te-njesive-te-qeverisjes-vendore-ne- republiken-e-shqiperise/ 17. Municipality of Durres, web page: https://www.durres.gov.al/bashkia/rajonet/673-profili-i-bashkise-durres 18. “Rapid National Inventory of Human Recourses dealing with social protection issues in Local Government level” INFOCIP, sponsored by UNICEF, August 2016. URL: http://www.infocip.org/al/wp-content/uploads/2018/08/raporti-final-UNICEF-dhejtor-rezi-profilet-e-61- bashive-gerti-pa-emra.pdf
58 TYPE OF SOCIAL CARE FACILITIES AND SERVICES PROVIDED Durrës region is fourth in country for number of inhabitants and offers social care services for 262 785 inhabitants, 67% of region beneficiaries are living in Durrës municipality. In the region of Durrës, the municipality of Durrës provides 78% of region social care services. From 10 services foreseen at the basket of services only five are provided in Albania: children, women, elderly, PwD and individuals in need. And out of 5 services developed in country level, Durrës municipality offers only 4: children, women, elderly, PwD. Out of all social care services, 45% are for children19. Social care services in Durrës municipality are mainly concentrated in the urban area and represent 7% of the national percentage of SCS. SCS are offered to the neediest categories of children, elderly and PwD, in residential or day care services20. Providers are public and non-public (NGOs).
TABLE 1. SCS (SOCIAL SERVICE DURRËS MUNICIPALITY) Residential public Residential Non-public Community Public Community non-public 2 (SSS: PwD, orphans) 1 (private) 2 (municipality: Roma, PwD) 13 (NGOs)
Social care services for: ♦♦ Children are 8: 3 public residential services, 5 community public day care services (2 public and 3 non-public) as following: Elderly are 3, one nonpublic residential and 2 community (1 public and 1 nonpublic); ♦♦ People with Disabilities (PwD) are 3, one public residential and 2 community-based (2 public and 1 nonpublic)21; Roma children 2 community day care services in Nish-Tulla zone22; ♦♦ Women, 1 nonpublic day service center for Victims of Domestic Violence;
SERVICES PROVIDED AND FINANCED In total there are 17 social care services provided in municipality of Durres of different typology and capacity: 2 services are covered/offered by State Social Services, 2 from municipality, 13 by NGOs. Residential institutions are staffed and financed by State Social Service. Day care public facilities are staffed and financed by municipality. All other types of non-public services are provided by NGOs. Day services consist of food and material aid, psychosocial support, school/ educational support, rehabilitation therapy, legal assistance, information and awareness raising. Actually, municipality provides two services: elderly day care and Roma day care at Nish Tulla. Co-financed services of municipality and NGOs are those for domestic violence victims, where offered legal assistance and psycho-social support for women. The newest service is Day care for children with disabilities, where the municipality offers the environment and NGO the staff for a period of 2 years. All other services are offered by NGOs, local and international ones. None of Durres municipality administrative units has NARU and covered with social care services. All services, except for 2 (in suburb), are focused in the city of Durres (urban area).
19. Social Care Needs Assessment for 12 regions in Albania. UNDP, Leave no One Behind, June 2019, Authored: Arlinda Ymeraj. URL: https:// www.al.undp.org/content/albania/en/home/library/poverty/vleresimi-i-nevojes-per-sherbime-shoqerore-ne-12-qarqet-e-shqipe.html 20. Durrës Municipality Social Plan 2017-2020 authored ESA consulting sponsored UNICEF 2016. URL: https://www. google.com/search?q=plani+social+bashkia+durres&rlz=1C1GNAM_enAL686AL689&oq=Plani+social+&aqs=chrome.4 .69i59j69i57j0l4.7697j0j7&sourceid=chrome&ie=UTF-8 21. State Social Services, Annual Report 2018, http://www.sherbimisocial.gov.al/wp-content/uploads/2019/01/ANALIZA-E-PUNES- SHSSH-2018.pdf 22. State Social Services, Data on Durres municipality: URL: http://www.sherbimisocial.gov.al/wp-content/uploads/2016/11/Bashkia- Durres.pdf
UNICEF in Albania ANNEXES TO THE EVALUATION REPORT 59 SOCIAL CARE SERVICES IN DURRËS MUNICIPALITY
TABLE 2. STATE SOCIAL SERVICE
Typology Number of services 17 Children 8 residential institutions 2 community day care centers 6 Elderly 3 residential institution 1 day care community centers 2 People with Disabilities (PwD) 3 residential institutions 1 day care community centers 2 Women (Domestic Violence Victims) 1 residential institutions 0 day care community centers 1 Roma/gipsy 2 residential institutions 0 day care community centers 2
The staff of Department (Directory) of Social Service at Durres municipality is overcharged with social problems of existing and added population (of 5 new administrative units), which is bumping for services in Durres department and this addresses them to the existing services in the center of Durres. Lack of financial funds for social care is endemic and causes shortage of services.
WORKFORCE FOR SOCIAL PROTECTION The staff of social protection (cash transfer officers and Child Protection Unit officers) are 26 employees23. All staff has a university diploma, 18 have the status of civil servant24 and 8 with contract work (time-line undefined). It is important to highlight, that more than half25 of the academic background of this structure consist in fields of agronomy, economics, law which are far away from backgrounds of social services, even though in different articles of the Law No. 9355 dated 10.03.2005 “On social assistance and services” (amended), are clearly assigned tasks and responsibilities that require a background in Social Work or Social Services Administration. Social Services Directory (SSD) is divided into two sectors:
♦♦ Cash Payment Sector (CPS) with 15 employees: 1 Chief Sector for 12 social administrators to share: 7 for the city and 5 for new administrative units, Sukth, Manze, Rrashbull, Ishëm and Katund i Ri; 1 inspector for disability and 1 inspector of finance and statistics; ♦♦ Social Care Services Sector (SCSS) 9 staff: with 10 workers (1 Head of sector, 5 specialists for child protection, 2 specialists for gender equality and domestic violence, 2 specialists for other services, such as for disabled, elderly and other cases for residential care) and 1 general Director.
Administrative Units Social Protection Sector suffers human resources shortage (specialist for Gender Equality and Domestic Violence and Child Protection). Actually out of 10 AU, only in one has a GEDV specialist (Sukth) and 2 have CPU specialist (Sukth and Rrashbull). Unlike
23. STOCKTAKING REPORT: Plani Social 2017-2020, Bashkia Durrës, Plani social, ESA Consulting/UNICEF, 2019. 24. “Rapid National Inventory of Human Recourses dealing with social protection issues in Local Government level” INFOCIP, sponsored by UNICEF, August 2016. URL: http://www.infocip.org/al/wp-content/uploads/2018/08/raporti-final-UNICEF-dhejtor-rezi-profilet-e-61- bashive-gerti-pa-emra.pdf 25. Durrës Municipality Social Plan 2017-2020 authored ESA consulting sponsored UNICEF 2016. URL: https://www.google. com/search?q=plani+social+bashkia+durres&rlz=1C1GNAM_enAL686AL689&oq=Plani+social+&aqs=chrome.4.69i59j69i57j0l4. 7697j0j7&sourceid=chrome&ie=UTF-8
60 appear the structures of Cash Payment Sector, where all administrative units (AU) have 1 social administrator, who often performs the functions of employees of CP and GEDV26. In 201827, as per Social Plan 2017-2018, the municipality has established a new structure of Social Protection composed of gender specialists and child protection officers. In this structure are added two CPUs with social science training: one psychologist and one social worker. Cash transfers office is increased with two new staff for cash. During 2018, municipality has paid 3 staff of the Center of children with disabilities, as per agreement with Save the Children. Indeed, staff working as CPU officers in administrative units, perform CP among other tasks, because they have other functions in the official structure of administrative units, with primary function as “specialist of economic aid”, and no functions deriving from the law on child protection28. This anomaly derives from the shortage of funds for social care. The existing social administrators, whose primary function is cash delivery, cover social care as well, in administrative units.
NARU Durres Municipality has no NARU. Only one administrative unit has a similar structure with NARU, Sukth unit since 2013, composed of cash specialist of cash, CPU specialist and Domestic Violence and Gender specialist.
MALESIA E MADHE (MM) SCS MUNICIPALITY, 19 Jan 2020
SOCIAL PLAN MM Social Plan 2019-2022 is worked out with support of UNICEF (ESA Consulting); it is approved by the Municipal Council, but not costed or budgeted. The cost of existing social protection is covered by municipal funds and government funds, through conditional and unconditional transfers.
DEMOGRAPHIC DATA According to Census, 2011 in MM municipality there are 30,823 inhabitants, whilst according to civil registration office, 2018 there are 47,394 inhabitants. The new municipality occupies a territory of 951.01 km2, with a density of population 32.41 inhabitants/km2 according to civil registry. In accordance with the law Nr.115/2014, the municipality is composed of 6 administrative units:
26. Durrës Municipality Social Plan 2017-2020 authored ESA consulting sponsored UNICEF 2016. URL: https://www.google. com/search?q=plani+social+bashkia+durres&rlz=1C1GNAM_enAL686AL689&oq=Plani+social+&aqs=chrome.4.69i59j69i57j0l4. 7697j0j7&sourceid=chrome&ie=UTF-8 27. STOCKTAKING REPORT: Plani Social 2017-2020, Bashkia Durrës, Plani social, ESA Consulting/UNICEF, 2019. 28. Law on child rights and protection 18/2017, Article 49, Para. 2 on the child protection officer. URL: https://www.drejtesia.gov.al/wp- content/uploads/2017/11/04_Ligj_18_2017_23.02.2017_Per_te_drejtat_dhe_mbrojtjen_e_femijes.pdf
UNICEF in Albania ANNEXES TO THE EVALUATION REPORT 61 Koplik, Gruemire, Kastrat, Kelmend, Qendër and Shkrel. Within MM municipality territory are two cities, Koplik and Bajzë, and 56 villages. MM Municipality is part of Shkoder Region. Only 16, 5% of population lives in urban zone.
TYPE OF SOCIAL CARE FACILITIES AND SERVICES PROVIDED There are no social care services provided in MM, except for one center for children with disabilities, set up by MEDPAK (Association of People with Disabilities) which was handed over to Municipality in 2019.
WORKFORCE FOR SOCIAL PROTECTION The Economic Aid and Social Services Sector at the municipality are composed of 4 persons: 1 head of sector, 1 social administrator for poverty cash assistance (NE), 1 social worker and 1 lawyer. The sector is mainly working for social protection scheme rather than social care. There is neither CPU specialist nor DV Coordinator. Each administrative unit has 1 social administrator for economic aid. Kelmend administrative unit has 1 social worker. Recently 2 workers of the center for children with disabilities are added to the social protection work force which is 13 persons.
NARU No Naru exist in MM.
PERMET SCS MUNICIPALITY, 19 Jan 2020
SOCIAL PLAN Përmet has a Social Plan 2017-2020, worked out with support of UNDP/LNB, is approved by the Municipal Council, costed but not budgeted. The cost of existing social protection is covered by municipal funds and government funds, through conditional and unconditional transfers. Only 5% of the municipal fund goes to social protection for staff salaries, there is no budget for services.
DEMOGRAPHIC DATA According to Civil registration office there are 19.872 inhabitants. More than half of population 56% lives in the city of Përmet, while 44 % in other administrative units (former rural communes). The municipality is halved from 2001 until 2011 (National Census of 2011) because of emigration
62 and low birth rates. Permet municipality is composed of 5 administrative units: Përmet, Çarshovë, Frashër, Petran, Qendër Piskovë. Municipality of Permet administers 49 villages.
TYPE OF SOCIAL CARE FACILITIES AND SERVICES PROVIDED Permet municipality has no social services facility. The unit of municipal social protection serves as a case management unit as well. The unit is composed of social protection (social administrators) and CPU/DV officer. The DV officer works on several tasks such as public awareness raising and domestic violence cases management. The municipality has access to a school dormitory room for sheltering domestic violence victims for 72 hours until issuance of Court Protection Order. There are no specialized services not NGOs working for women and domestic violence. Specialist of DV provides legal assistance and psycho-social support to victims (36 cases treated during last 5 years). Child protection is done through identification and cases monitoring done by municpal team. Cases are managed by the CPU established in 2018 (6 cases of children in stereet situation sare terated during 2018). Actually 93% of families in poverty scheem have children. PwD with disability allowance are 200; they have no access in social care services. Children with disabilities have no acces in services either, except for primary education (31 children with disabilities). Services are nor provided or finaced.
WORKFORCE FOR SOCIAL PROTECTION Social Protection and Social Service office has 8 employees: 1 office coordinator, 1 CPU/DV and 6 social administrators (2 in municipality for poverty and disability and 4 in administrative units). The office of social protection and social services is under Economy Sector.
ADMINISTRATIVE UNIT EMPLOYEES Përmet 1 office coordinator 1 specialist poverty/NE 1 specialist PwD 1 Domestic Violence/ CPU Piskovë 1 specialist NE/PwD Frashër 1 specialist NE/PwD Petran 1 specialist NE/PwD Çarshovë 1 specialist NE/PwD Total 8 persons
Source: Municipality Permet
NARU There is no NARU in Permet municipality.
UNICEF in Albania ANNEXES TO THE EVALUATION REPORT 63 ANNEX CORRESPONDENCE OF 10 EVALUATION CRITERIA WITH FINDINGS, LESSONS LEARNED AND RECOMMENDATIONS
LESSON CRITERION FINDING RECOMMENDATION LEARNED
1: The theory of change of the contribution by UNICEF in Albania was 3.3. UNICEF in Albania, supported by the Regional Office, built progressively and organically, with minimal strategic guidance should develop a definition of key concepts and principles, a organisationally, through the learning progression of UNICEF in Albania as well as strategic guidance on social care services in a specific context. intervention. 2: Post factum, the intervention approach proved relevant to, and coherent with UNICEF standards and strategic approach, worldwide and regionally. 3: The key added value of UNICEF in Albania was to embed the prioritization of children and families within the social care services reform, in line with the UNICEF mandate. 4: The purpose of the intervention was very relevant to the needs and 3.1.UNICEF in Albania, supported by the Regional Office, barriers experienced by children and families, as identified by UNICEF in should develop guidance for gender analysis as a part of Albania and by interviewed beneficiaries. initial research and analysis of barriers to social inclusion and social care services needs. Relevance 5: The theory of change and actual intervention of UNICEF in Albania gradually identified specific barriers and bottlenecks, and incorporated their a analysis into the system, thereby building relevance into the programmes. 6: The contribution, theory of change and actual intervention of UNICEF in Albania were in line with Albania’s international human rights obligations and inspired by Albania’s national priorities deriving from these obligations. 7: The intervention has adjusted to evolving and emerging national priorities by embracing parallel reforms and a new political, policy and a legislative environment, especially decentralisation. 8: With the facilitation of an inclusive consultation process, UNICEF in Albania has contributed to keeping social care services on the agenda of policy and decision makers at national, regional and local levels. 9: At intermediate result level, UNICEF has succeeded in jointly establishing, with the Government of Albania, a concept for the system of social care services based on essential building blocks, and the corresponding normative policy framework. 10: The contribution of UNICEF in Albania was instrumental for at least 2.2. Create a community of practice among Social Care three building blocks to clearly emerge: the expected immediate results Services/Social Work practitioners of the intervention are partly met. 2.3.Promote Social Worker diplomas among men and promote their participation and contribution to social care services
Effectiveness 11: However, the stakeholders are not aware of the principles nor 1.2. Systematically support the MOHSP to finalise the of the building blocks, and the normative policy framework is not yet development of procedures, standards and guidance to e, i complete: the conditions created are not sufficient for the building practitioners, in order to facilitate the implementation of the blocks to be operational, and immediate results remain transitional. normative framework and the delivery of social care services 12: UNICEF has succeeded in mainstream most of its principles, at least to some extent, into the spirit of normative policy framework. 13: The funding and the financial flows supporting the social care 1.1. Re-engage stakeholders and decision-makers, acting services system remain an unsolved question, which seriously d as an honest broker to define the funding responsibilities endangers the intervention’s results. and mechanisms for social care services at local level
64 14: UNICEF has made an intensive use of its unique internal resources and of assets put at its disposal, but could have optimized its advocacy i, l potential to help speed up the development of the normative policy framework at regulatory level. 15: In the absence of a robust theory of change and corresponding 3.2. UNICEF in Albania, supported by the Regional Office, systematic monitoring, UNICEF in Albania has not been well equipped should implement theory of change-based planning and a to steer its intervention and the use of its inputs in a pro-active and monitoring, and train its staff accordingly documented fashion. 16: UNICEF in Albania mobilized excellent expertise, but has not given experts clear guidance or monitored the focus on the principles they d, k
Efficiency were meant to champion. 17: Thanks to a partnership-based and inclusive approach, the contribution of UNICEF was at least complementary, and often h synergetic with the vision and strategies of the Government, civil society actors and other international stakeholders. 18: In the context of partnership management, the prudent approach to advocacy adopted by UNICEF did not help overcome the important e bottleneck of funding for social care services. 19: The contribution of UNICEF in Albania to the country’s policy 1.5.Build comprehensive monitoring mechanisms of normative framework on social care services has fostered growing the delivery of the full basket of services in few select demand for social care services. municipalities. 20: The transitional implementation of the policy normative framework 1.4.Support the Government of Albania and relevant on social care services creates the risk of negative effects on child stakeholders to review, clarify and fine-tune the normative protection, as long as the assumption of public funding for social care c, e policy framework. services is not fulfilled, and child protection and social care services are not strongly integrated. 21: The social care services policy normative framework as promoted Impact by UNICEF in Albania could benefit from stronger interlinks with other ongoing areas of government action in favour of social protection or f, g other economic and social rights.
22: At this stage there is only anecdotal evidence that the policy 1.3. Systematically support municipalities to implement normative framework has had measurable impact on rights holders. b the normative policy framework on social care services, primarily through full-fledged municipal social plans 23: The social care service workforce is acting as a buffer with the 2.1. Promote with the Government of Albania an increase rights holders, in a system which is perceived as ineffective during the in the number of social workers employed in NARUs. current transitional period of the reform. This could in turn endanger the sustainability of this building block of social care services. 24: In a context of renewed international interest for social protection, UNICEF in Albania has opportunities for a renewed, strong role in the Sustainability final negotiations for the policy normative framework of social care services, and for their implementation in the rights holders’ lives.
UNICEF in Albania ANNEXES TO THE EVALUATION REPORT 65 ANNEX 11 REFERENCES
Apland, Kara and Carolyn Hamilton, ‘Final Evaluation of the Justice for Every Child Project: December 2013 –November 2017’, Coram International, Commissioned by UNICEF Bosnia and Herzegovina, May 2017. Babajanian, Babken and Rebecca Holmes, ‘Linkages between social protection and other social interventions for children in CEE/CIS’, Policy analysis paper for UNICEF, 14 February 2013. Bamberger, Michael and Marco Segone, ‘How to design and manage Equity-focused evaluations’, Evaluation Working Papers (EWP), UNICEF Evaluation Office, New York, 2011. Berens, Anne E., and Charles A. Nelson, ’The science of early adversity: is there a role for large institutions in the care of vulnerable children?’ The Lancet, 2015, p. 386, pp. 388-398. Bilson, Andy et al., Childonomics: A Conceptual Framework, Oxford Policy Management, Oxford, 2017. BMZ, ‘Social Protection for Equitable Development’, Position Paper 09/2017, Federal Ministry for Economic Cooperation and Development (BMZ), Germany, 2017. Bronfenbrenner, Urie, ‘Contexts of child rearing: Problems and prospects’, American Psychologist, vol.34, no.10, 1979, pp. 844-850. Browne, Kevin, The risk of harm to young children in institutional care, The Better Care Network and Save the Children, 2009. Centre for the Developing Child, Applying the Science of Child Development in Child Welfare Systems, Harvard University, Boston, n.d. Council of Europe Development Bank, ‘Social Care systems in Europe’, Technical Brief, Council of Europe Development Bank (CEB), 2018. Council of Europe, ‘Report by Dunja Mijatović, Commissioner for Human Rights of the Council of Europe, Following her Visit to Albania from 21 to 25 May 2018’, Strasbourg, 13 September 2018. Dahlberg, Linda, L.and Etienne. G. Krug, ‘Violence-a global public health problem’ ch.1 in World Report on Violence and Health, edited by Etienne. G. Krug et al., World Health Organization, Geneva, Switzerland, 2002, pp. 1–56. Davis, Rebecca, ‘Technical Support to UNICEF Europe and Central Asia Regional Office (ECARO) for a Regional Conference on Social Service/Social Work Workforce’, Draft, Albania Case Study, Oxford Policy Management Limited, Oxford, United Kingdom, 13 November 2018. Delaney S., P. Quigley and M. Shuteriqi, Understanding and applying a systems approach to child protection: a guide for programme staff, Terre des hommes/Child Frontiers, 2014. Edwards, Nancy., ‘How can systems’ approaches be optimally used to advance the primary prevention research agenda?’, Presentation to the UKPRP Workshop, Faculty of Health Sciences, University of Ottawa, Ottawa, Canada, 18 May 2017. ESA Consulting, ‘Udhërrëfyes për Bashkitë: për planifikimin dhe administrimin e shërbimeve të përkujdesit shoqëror’(Guidelines for Municipalities: planning and managing social care services), UNICEF, Tiranë, Mars 2019. ESA Consulting, ‘Programme Progress: Building the capacities of Local Government Units (LGUs) to develop social care services plans in their communities’, Final Report, UNICEF, Tirana, February 2017.
66 ESA Consulting, ‘Udhëzues: Instrumentet për planifikimin e shërbimeve të kujdesit shoqëror’, UNICEF, Tiranë, dhjetor 2016. Fajth, Gaspar, ‘Child Sensitive Social Protection’, Strategies for the extension of Social Protection, Power Point presentation, Chief of Social Policy and Economic Analysis at UNICEF Policy and Practice in New York, Turin 5 October, 2009. Gheorghe, Camelia and Ajsa Hadzibegovic, ‘Evaluation of the Programme “Montenegro – Investment case on early Childhood Development”, Final Evaluation Report, Vol.1, commissioned by UNICEF Montenegro, 18 april 2017. Gheorghe, Camelia and Ajsa Hadzibegovic, ‘Evaluation of the Programme “Montenegro – Investment case on early Childhood Development”,Final Evaluation Report, Vol.1, Annexes, commissioned by UNICEF Montenegro, 18 april 2017. Gheorghe, Camelia and Ajsa Hadzibegovic, ‘Evaluation of the Programme “Montenegro – Investment case on early Childhood Development”, Final Evaluation Report, Vol.2, Annexes, commissioned by UNICEF Montenegro, 18 april 2017. Gheorghe, Camelia, ‘Summative Evaluation of the Project “Promoting Access to Justice for Children in Azerbaijan through national capacity building for State and non-State Actors”, Evaluation Report (vol.1), UNICEF Country Office in Azerbaijan, 8 December 2017. Gray, Mel, James Midgley and Stephen A. Webb, ‘Introduction’, ch.1 in The Sage Handbook of Social Work, edited by Mel Gray, James Midgley and Stephen A. Webb, Sage Publications, London, 2012, pp. 1-15. HelpAge International Global Age Watch Insights, The right to health for older people, the right to be counted, HelpAge International, London, 2018. Hoxha, Artan, et al., ‘Social Care Services at the Regional Scale: Analysis and recommendations’, UNICEF, 2013 Humanity & Inclusion UK, News, ‘People with Disabilities are still excluded’,
UNICEF in Albania ANNEXES TO THE EVALUATION REPORT 67 & Homes for Children, Institute of Development Studies, International Labour Organization, Overseas Development Institute, Save the Children UK, UNDP, UNICEF, the World Bank and World Vision, August 2009. Jorgoni, Elira, and Sabina Ymeri, ‘Supporting planning of social care services’, Policy brief on roles and functions for the institutions involved with social care services, Institute of Contemporary Studies and UNICEF, Tirana, 2016. Joronen, Katja and Anja Rantanen, ‘Family Life Cycle Stages’ in Encyclopaedia of Quality of Life and Well-Being Research edited by Alex C. Michalos, Springer, Dordrecht, 2014. Joshua, Laurie, ‘Outline of National Sector Programme for Employment, Skills and Social Policy: A Results Based Approach’, 17 November 2014. Joshua, Laurie, ‘Support to Sustainable Reform Processes in the Employment, Skills and Social Policy Sector’, 17 November 2014. Joshua, Laurie, and Elira Jorgoni, ‘Strategic Agility: Road Map for the Integrated Policy Management Group and Development of a National Sector Programme in the Employment, Skills and Social Policy (ESSP) Sector, 8 December 2014. Joshua, Laurie et al., ‘Sector Budget Support on Social Inclusion in Albania (2019-2022)’, PowerPoint presentation at the Development Partners Meeting, EU Delegation Albania, 22 February 2019. Joynes, Chris, and Jacqui Mattingly, ’A systems approach to child protection’, K4D Helpdesk Report, Institute of Development Studies, Brighton, UK, 2018. Joynes, Chris, and Helen West, ‘A systems approach to child protection’, K4D Helpdesk Report, Institute of Development Studies, Brighton, UK, 2018. Joynes, Chris, and Roisin Plunkett, ‘Issues to consider in designing a Child Protection System’, K4D Helpdesk Report, Institute of Development Studies, Brighton, UK, 2018. Kovalevskaya, Antonina, ‘Review of financial regulations and financial information systems for the management of social protection budgets’, Report, UNICEF, Tirana, August, 2013. Lalor, Kevin and Perry Share, ‘Understanding Social Care’, ch.1 in Applied Social Care: An Introduction for Students in Ireland, 2nd ed., edited by Perry Share & Kevin Lalor, Gill & Macmillan, Dublin, 2009, pp. 3-21. Maclean, Siobhan, and Rob Harrison, System Theory’, ch. 42 ‘in Theory and Practice: A straightforward guide for social work students, 3rd ed., edited by Siobhan Maclean and Rob Harrison, Kirwon Maclean Associates Limited, Glasgow, 2015. MacKenzie, Alex and Sabina Ymeri, ‘Evaluation Report GoA-UN Program of Cooperation 2012- 2016’, Final Report, 29 June 2015. Martins, Deolinda and Elena Gaia, ‘Preparing for an Uncertain Future: Expanding Social Protection for Children in Eastern Europe and Central Asia’, UNICEF Policy and Strategy Social Protection Series, Social and Economic Policy Working Briefs, August 2012. Matković, Gordana, ‘Strengthening Planning of Social Care Services, Proposal on Establishing a Package of Services in Albania and Financing Mechanisms’, Institute of Contemporary Studies and UNICEF, Tirana, 2017. McKee, N., et al., ‘Involving People, Evolving Behaviour: The UNICEF experience’, ch. 12 in Approaches to Development Communication, edited by Jan Servaes, UNESCO, Paris, 2002. Mel Cousins & Associates, ‘Documenting the process of Local Government Units (LGUs) capacity building in Social Care planning’, Project supported by UNICEF Tirana Country Office, Consultant’s Report, 24 March 2017. Merrick, Melissa T, et al., ‘Prevalence of Adverse Childhood Experiences from the 2011-2014: Behavioral Risk Factor Surveillance System in 23 States’, JAMA Pediatrics, vol.172, no.11, 2018, pp. 1038-1044. Munday, Brian, Integrated Social Services in Europe, Council of Europe, Strasbourg, 2007.
68 Nelson, Charles, et al., Caring for orphaned, abandoned and maltreated children: Insights from the Bucharest Early Intervention Project, Washington DC, 2007. NHS Health Scotland, Adverse childhood experiences in context, NHS Health Scotland, Edinburgh, 2019. Peters, Guy B., ‘What is so wicked about wicked problems? A conceptual analysis and a research program’, Policy and Society, vol. 36, no.3, 2017, pp. 385-396. QURES, ‘Summative evaluation of the Minimum Package of Services component of the “Social inclusion through the provision of integrated social services at community level” modelling project in Romania, 2014-2018’, UNICEF, Bucharest, 2019. Rees, Nicholas, Jingqing Chai, David Anthony, ‘Right in Principle and in Practice: A Review of the Social and Economic Returns to Investing in Children’, UNICEF, Social and Economic Policy Working Paper, June 2012,
UNICEF in Albania ANNEXES TO THE EVALUATION REPORT 69 June 2017- May 2018’, Submitted By UNITED NATIONS, United Nations Country Office Albania, June 2018. United Nations and the Government of the Republic of Albania (GoA), ‘Programme of Cooperation 2012-2016’, UN Sustainable Development Framework, United Nations Albania, Tirana, Albania, 2012. United Nations and the Government of the Republic of Albania (GoA), ‘Programme of Cooperation for Sustainable Development 2017-2021’, United Nations Albania, Tirana, Albania, 24 October, 2016. United Nations and the Government of the Republic of Albania (GoA), ‘Programme of Cooperation for Sustainable Development 2017-2021: Progress Report 2017’’, United Nations Albania, Tirana, Albania, 2017. United Nations and the Government of the Republic of Albania (GoA), ‘Programme of Cooperation for Sustainable Development 2017-2021: Progress Report 2018’’, United Nations Albania, Tirana, Albania, 2018. United Nations Children’s Fund, ‘Social Profile of 15 municipalities’. nd. United Nations Children’s Fund, ‘UNICEF’s Global Social Protection Programme Framework’, UNICEF, New York, 2019. United Nations Children’s Fund, ‘Every Child Learns’, UNICEF Education Strategy 2019-2030, UNICEF, New York, 2019. United Nations Children’s Fund, ‘Guidelines to Strengthen the Social Service Workforce for Child Protection’, UNICEF, New York, 2019. United Nations Children’s Fund, ‘Strengthening child protection systems: evaluation of UNICEF strategies and programme performance’, E/ICEF/2019/23, Executive Board, 3 July 2019. United Nations Children’s Fund, ‘State of the Social Service Workforce in South Asia’, UNICEF, Kathmandu, 2018. United Nations Children’s Fund, ‘Strengthening the Social Work and Social Service workforce in Europe and Central Asia as an Investment in Our Children’s Future: A Call to Action’, UNICEF, Geneva, 5 December 2018. United Nations Children’s Fund, ’UNICEF Strategic Plan 2018–2021: Executive Summary’, UNICEF, New York, 2018. United Nations Children’s Fund, ‘Thematic evaluation of UNICEF contribution to reducing vulnerabilities, strengthening resilience and promoting the rights of adolescents in Belarus’, (Internal document), Spring 2018. United Nations Children’s Fund, ‘Evaluation of the Government of Moldova – UNICEF 2013-2017, Country Programme of Cooperation’, Chisinau, UNICEF Moldova Office, 2017. United Nations Children’s Fund, ‘National Social Pact to build a system of Social Care Services: Milestones, challenges and future opportunities’, Working Paper 25 September 2017. United Nations Children’s Fund, ‘Social Care Services Reform in Albania (2012-2016) to address vulnerability and marginalization of children, women and families (with focus on integration of Roma communities)’, Final Donor Report, 30 June 2017. United Nations Children’s Fund, The State of the World’s Children 2016: A fair chance for every child, UNICEF, New York, 2016. United Nations Children’s Fund, ‘Regional Knowledge and Leadership Area 8: A child’s right to social protection’, Regional Strategy 2016-2020, UNICEF CEE/CIS, 16 February 2016. United Nations Children’s Fund, ‘Country programme document Albania’, Report to the Economic and Social Council, E/ICEF/2016/P/L.12, United Nations, 18 July 2016. United Nations Children’s Fund, ‘UNICEF Publication Toolkit’, October 2015 United Nations Children’s Fund, ‘UNICEF Procedure for Ethical Standards in Research, Evaluation, Data Collection and Analysis’ (Internal document), 1 April 2015.
70 United Nations Children’s Fund, ‘UNICEF in a changing world: Programming in Middle Income Countries’, Draft Report, Middle-Income Cluster Group, UNICEF, New York, 2014. United Nations Children’s Fund - World Bank, ‘Common Ground: UNICEF and World Bank Approaches to Building Social Protection Systems’, Note, January 2013. United Nations Children’s Fund, UNICEF and World Bank ‘Common Ground’ Approaches to Building Social Protection Systems, Webinar Presentation by Laura Rawlings, Sheila Murthy and Natalia Winder, 19 February 2013. United Nations, ‘Development cooperation with middle-income countries: Report of the Secretary-Genera’l, A/68/265, United Nations, New York, 5 August 2013. United Nations Children’s Fund, ‘Social Protection Strategic Framework – Integrated Social Protection Systems: Enhancing Equity for Children’, UNICEF, New York, 2012. United Nations Children’s Fund, ’Social Protection Strategic Framework, 18 May 2012’, UNICEF, New York, 18 May 2012. United Nations Children’s Fund, ‘Integrated Social Protection Systems: Enhancing equity for children’, UNICEF Social Protection Strategic Framework, New York, March 2012. United Nations Children’s Fund, ‘Integrated Social Protection Systems: Enhancing equity for children’, UNICEF Social Protection Strategic Framework, Concept Note for Consultation, UNICEF, New York, 2011. United Nations Children’s Fund, ‘UNICEF’s Approach in Middle Income Countries – Six Core Strategic Roles’, Discussion Note 1, Policy and Practice, UNICEF, New York, May, 2010. United Nations Children’s Fund, ‘The Programme Policy and Procedure Manual (PPPM): programming approach in Middle Income Countries (MICs) and High-Income Countries (HICs)’, 2010. United Nations Children’s Fund, ‘Global Study on Child Poverty and Disparities 2007-2008’, Guide, UNICEF’s Economic and Social Policy Unit, Global Policy Section, Division of Policy and Planning’, UNICEF, New York, September 2007. United Nations Economic and Social Council, ‘UNICEF Child Protection Strategy’, E/ICEF/2008/5/ Rev.1, para. 12-13, 2008. UNHCR, ‘Protecting Children of Concern through a Systems Approach: Guidance for Field Offices’, UNHCR, 2010. URI, ‘Matrix of Competencies of Local Self-Government 2019’, Funded by the European Union, February 2019. USAID, ‘Early Childhood Development for Orphans and Vulnerable Children: Key Considerations’, Technical Brief, USAID, Arlington, Virginia, 2011. World Bank, ‘Inclusion Matters: The Foundation for Shared Prosperity’, New Frontiers of Social Policy, Washington DC, 2013. World Bank, ‘Social Protection Assessment of Results and Country Systems - SPARCS’, Brief Concept and Invitation to Partner, 8 February 2013. World Bank, ‘The World Bank’s Social Protection and Labor Strategy 2012–2022: Resilience, Equity, and Opportunity’, April 2012. World Bank, ‘World Bank’s analysis of its own Europe and Central Asia Social Protection database’, (Map of Social Protection Policy), 2009. Williamson, John and Aaron Greenberg, Families not orphanages, Better Care Network, New York, 2010. World Vision International, ‘A Systems Approach to Child Protection’, A discussion Paper, Children in Ministry on behalf of World Vision International, 2011. Wulczyn Fred., et al., ‘Adapting a Systems Approach to Child Protection: Key Concepts and Considerations’, Working Paper, UNICEF, New York, 2010.
UNICEF in Albania ANNEXES TO THE EVALUATION REPORT 71 LEGAL AND POLICY DOCUMENTS
STRATEGIES
Republic of Albania, Decision of Council of Ministers Nr. 87, date 03/02/2016, On the approval of the Social Inclusion Policy Paper 2016 -2020, Official Gazette Nr. 20, date 11/02/2016, Council of Ministers,
LEGAL ACTS Republic of Albania, Law Nr. 121-2016, date 24/11/2016, On social care services in the republic of Albania, Official Gazette Nr. 236, date 7/12/2016, Parliament,
72 DECISIONS Republic of Albania, Decision of Council of Ministers Nr. 508; date 13/09/2017, On the definition of the state responsibilities field of the Ministry of Health and Social Protection, Official Gazette Nr. 166; date 19/06/2017, Council of Ministers,
ORDERS Republic of Albania, Minister Instruction Nr. 394, date 9.5.2019, On the organization and functioning of the multidisciplinary commission and procedures for implementation of the standards of alternative care services for children in foster care families, Official Gazette Nr.81, date 05/06/2019, Ministry of Health and Social Protection,
UNICEF in Albania ANNEXES TO THE EVALUATION REPORT 73 Republic of Albania, Minister Instruction Nr.597; date 22.12.2017, On the approval of standards of social care services, home care for people with disabilities, Official Gazette Nr.238; date 31/12/2017, Ministry of Health and Social Protection,
URLS United Nations Children’s Fund, ‘The State of the World’s Children 2016: A fair chance for every child’, June 2016, UNICEF,
74 and Economic Policy Working Briefs, UNICEF Policy and Practice, August 2010,
UNICEF in Albania ANNEXES TO THE EVALUATION REPORT 75 ANNEX INFORMATION ON 12 THE EVALUATORS
Ms. Camille Massey, International Consultant/Team Leader, disposes of over seventeen years of experience with non-governmental, governmental and international organisations, out of which thirteen in evaluation. Her academic and professional career gives her solid background in international human rights law and standards, in particular children rights, economic and social rights and gender equality. She has advised and trained international civil servants and government officials on strategic planning and policy development, including rights-based and gender-sensitive policies, and has experienced comparable interventions, both as a programme implementer in the field (in Bosnia and Herzegovina, where, as Human Rights Officer and Economic and Social Rights Adviser working on social care services reform, and on the rights of children and Roma), and as an evaluator (in the Council of Europe, evaluating standard setting and monitoring, as well as regional programmes regarding children rights and economic and social rights, particularly in Albania). She worked in the Western Balkans for nine years (in permanent residence), and an additional five years on field missions and case studies, including repeatedly in Albania. She was therefore particularly well suited to lead the team through this sensitive and complex evaluation.
Ms. Merita Poni, National Consultant, Lecturer at the Department of Sociology of the University of Tirana, brings twenty years’ experience as Independent consultant on Human Rights and Social Inclusion. Her work as Social Policy Adviser at the Ministry of Labour Social Welfare and Equal Opportunities, Albania, and as Programme Manager for various international humanitarian NGOs made her ideal for her requirements as National Consultant in the evaluation.
Ms. Elayn Sammon, Social Care Services Expert/peer reviewer, offers more than thirty years’ experience in project design and development, implementation, and monitoring and evaluation. Her expertise in child and social protection systems development including cash transfers and social welfare service delivery, as well as health, education and broader social care sectors were instrumental in her role as Expert Consultant and Peer Reviewer of the evaluation.
76 UNICEF in Albania ANNEXES TO THE EVALUATION REPORT 77
80