Scaling up Parenting for Lifelong Health in Montenegro: Funded by the European Union

A feasibility assessment Author: Amalee McCoy

Editing and proofreading: Peter Stonelake

Cover photo: Duško Miljanić

Design: Pixella

Circulation: 70 copies Scaling up Parenting Acknowledgements for Lifelong Health in Montenegro: This feasibility assessment The researcher would like to thank Ida Ferdinandi at UNICEF of the institutionalization, Montenegro for her high level of support and engagement throughout this study. In addition, various UNICEF staff A feasibility assessment scale-up and sustainability at the Country and Regional Offices provided important of Parenting for Lifelong feedback which strengthened this report, namely: Jessica Health for Young Children Katherine Brown, Ivana Ceković, Nada Đurović Martinović, (PLH-YC) in Montenegro Guzal Kamalova, Maja Kovačević, Danilo Smolović and Stevan Stanišić. Special and sincere gratitude is due to Professor was commissioned by the Table of Contents Judy Hutchings (Bangor University), who provided vital UNICEF Montenegro Country insights concerning strategies for quality PLH-YC delivery, Office with support from the drawing from over 40 years of experience in the parenting ACKNOWLEDGEMENTS ��5 European Union. The research support field, during several online meetings, as well as ACRONYMS ��5 through her review of the draft report. Many thanks are was conducted by Dr Amalee due to Dr Yulia Shenderovich (University of Oxford), who EXECUTIVE SUMMARY ��6 McCoy, as an independent also provided many helpful comments and suggestions on 1. INTRODUCTION �� 10 researcher and consultant with the draft report. Furthermore, the researcher would like PARENTING FOR LIFELONG HEALTH FOR YOUNG CHILDREN �� 12 experience in the adaptation to thank Danilo Leković and Tamara Jurlina, who provided high-quality simultaneous translation support during primary PURPOSE AND OBJECTIVES �� 13 and empirical evaluation of data collection. Finally, the researcher is very grateful to 2. METHODOLOGY �� 14 PLH-YC. the individuals and institutions that participated in this STUDY MANAGEMENT �� 14 assessment, for taking the time to share their valuable experiences and candid views. STUDY APPROACH �� 14

STUDY PROCESS �� 15 Disclaimer: The contents of this report are the sole 3. IMPLEMENTATION STATUS OF PARENTING FOR responsibility of the researcher and can in no way be taken to reflect the views of UNICEF, the European Union and their LIFELONG HEALTH FOR YOUNG CHILDREN IN MONTENEGRO �� 18 partners. A) OVERVIEW OF IMPLEMENTATION TO DATE �� 18

B) PROGRAMME DELIVERY PROCESS ��22

C) PARTICIPANT PROFILES AND EXPERIENCES OF THE PROGRAMME ��25 Acronyms

D) EXPERIENCES OF PLH-YC FACILITATORS AND SUPERVISORS ��27

E) PROGRAMME IMPACT ��30 CRC Convention on the Rights of the Child F) PROGRAMME COSTS ��31

4. MOVING FORWARD: INSTITUTIONALIZATION AND SCALING-UP ��38 EU European Union A) SUPPORTIVE LAWS, POLICIES, AND INSTITUTIONS ��38

B) NATIONAL STRATEGIC PLANNING AND STANDARDIZED GUIDANCE ��41 FGD Focus group discussion

C) STAFF RECRUITMENT AND CAPACITY BUILDING ��46 HC Healthcare Centre D) PROGRAMME REACH ��49

E) PROGRAMME ADAPTATION AND INNOVATIVE APPROACHES ��50 PLH-YC Parenting for Lifelong Health F) PROGRAMME MONITORING, EVALUATION, AND FURTHER RESEARCH ��52 or Young Children 5. RECOMMENDATIONS ��54

APPENDIX A. LIST OF PLH-YC FACILITATORS AND SUPERVISORS ��57 RCT Randomized controlled trial APPENDIX B. RATES OF PARTICIPANT RECRUITMENT, ENROLMENT, UNICEF United Nations Children’s Fund COMPLETION, AND ATTENDANCE ACROSS SITES ��60

APPENDIX C. ESTIMATED COSTS OF PLH-YC PROGRAMME DELIVERY ��62 WHO World Health Organization APPENDIX D. EXAMPLES OF CASCADE TRAINING MODELS ��64

APPENDIX E. LIST OF RESEARCH PARTICIPANTS ��66

REFERENCES ��68

Scaling up Parenting for Lifelong Health in Montenegro: A feasibility assessment 5 Executive summary

Introduction

In response to the high growing body of scientific literature has shown that A social-learning-theory-based parenting programmes prevalence rates of can effectively diminish rates of violence against children violence against children, and improve a range of related outcomes, such as positive parenting skills, developmentally responsive care with at least one billion and affection, and parental mental health. One of these interventions is Parenting for Lifelong Health for Young children subjected to Children (PLH-YC), a social-learning-theory and group- such harm during the based programme for parents and primary caregivers of children aged 2–9 years which has been designed for past year alone, there has the particular needs of low- and middle-income country contexts. PLH-YC is intended for delivery as a secondary been a surge of global prevention intervention for those families at risk of child interest in evidence-based maltreatment. violence prevention and Purpose and objectives the promotion of nurturing This feasibility assessment was commissioned by relationships between UNICEF with support from the European Union to support the institutionalization, scale-up and sustainability parents/caregivers and of PLH-YC at the national level in Montenegro. The their children. specific objectives were to: 1. Document evidence of programme implementation progress to date and the effectiveness of PLH-YC both in Montenegro Photo: and internationally; Duško Miljanić / 2. Propose modalities for institutionalizing the UNICEF Montenegro programme in terms of coordination and planning, programme delivery, monitoring and evaluation, drawing on examples from other participants; and an online survey targeting 68 parents Healthcare Centres, kindergartens and NGOs. Six cycles countries; and caregivers who did not successfully complete the of programme delivery were undertaken between 2018 3. Cost the programme, utilizing a costing tool that programme, with seven people responding. The research and early 2021, with a total of 458 parent and caregiver can be customized in the future as variables utilized thematic analysis and a hybrid approach to the participants enrolled across five municipalities: Podgorica, change; and coding of qualitative data. Desk review findings and Nikšić, Bijelo Polje, Berane and Cetinje. 4. Share reflections on a broader strategy for primary research data were integrated to draw key parenting support in Montenegro with a conclusions and propose recommendations for scaling To recruit parents and caregivers, facilitators utilized continuum of services/programmes. up. organizational websites and social media platforms, existing caseloads and contacts, and cross-sector Methodology PLH-YC programme implementation in referrals. Many facilitators noted that parent group compositions with mixed demographic and risk This assessment adopted a mixed-method approach, Montenegro profiles were an ideal arrangement. Data on participant combining a desk review and primary qualitative and engagement revealed that there were high rates of quantitative research. Primary data collection comprised Since late 2017, UNICEF, the Parenting for Lifelong Health enrolment (96%), completion (89%) and attendance individual interviews with 16 directors of service initiative and the Government of Montenegro have made (86%) across the six delivery cycles. Demographic data providers, ministry officials and senior representatives considerable progress in implementing PLH-YC over showed that nearly nine out of 10 participants were of independent institutes; four focus group discussions four phases in Montenegro. A total of 48 facilitators and mothers, the average age was 35 years, and 20% had at Photo: six supervisors have been trained and supervised, who Duško Miljanić / with 19 PLH-YC facilitators, supervisors and UNICEF least one household difficulty. UNICEF Montenegro staff; written feedback via e-mail from seven professional delivered the programme within the context of their existing work as psychologists, pedagogues, nurses, Feedback from participants demonstrated high overall preschool teachers and social workers at Primary satisfaction ratings (an average score of 4.74 out of 5),

6 Scaling up Parenting for Lifelong Health in Montenegro: A feasibility assessment Scaling up Parenting for Lifelong Health in Montenegro: A feasibility assessment 7 while noted obstacles to engagement were: insufficient scheduling and logistics; heavy workloads; unsupportive The most significant costs for delivery of PLH-YC relate Additional areas for inclusion in the National Action Plan available time; busy work schedules; and the movement management; venue limitations; and impediments to facilitator wages, with research respondents revealing and Guidelines comprise activities for local programme to online group sessions during the COVID-19 pandemic. attributed to the pandemic. that health-sector and education-sector facilitators were promotion with communities and other institutions that Inputs from PLH-YC facilitators and supervisors compensated for overtime and the additional workload families regularly come into contact with. Guidance underscored a number of factors and opportunities that The 2018 pre-post evaluation of the programme which (€2.70 per hour and €100 per month, respectively), should indicate how service providers can undertake both enabled successful implementation, including: personal comprised 82 parents/caregivers found an overall 70% while NGO facilitators received no additional fees. universal and targeted recruitment approaches, as well motivation; professional development and application reduction in harsh and abusive parenting. There were Other costs included: venues and refreshments; mobile as how to reach and accommodate particularly vulnerable to facilitators’ regular work with families; supportive improvements in supporting positive behaviour (11%) phone service, internet access and any travel costs families through cross-sector procedures. Furthermore, management and an enabling work environment; high- and limit setting (14%), with reductions in dysfunctional for facilitators; video equipment for supervision; any the incorporation of advice as to how service providers quality supervision; and community collaboration. The parenting (25%), child behaviour problems (31%), and expenses to address barriers to participation, such as can incorporate minor adaptations to suit particular perceived implementation challenges and obstacles were: parental depression (45%). There were no significant childcare and transport; and costs related to monitoring, subgroups without affecting programme effectiveness is insecure or insufficient financial support; difficulties with effects on parental stress. evaluation and dissemination. critical. Innovative approaches for programme delivery, including through ParentChat, should be explored and Moving forward: institutionalization incorporated into national planning if the feasibility studies Photo: are successful. The National Action Plan and Guidelines Duško Miljanić / and scaling up should also incorporate guidance and standardized tools UNICEF Montenegro for the systematic collection and analysis of process Montenegro has a well-articulated legal and policy data (programme reach, fidelity and quality) and impact framework that provides a basis for concerted state data (outcomes, acceptability and case records). Finally, action to prevent violence against children and to provide a national research agenda is needed to evaluate assistance to parent and caregivers in their child-rearing effectiveness at scale, test any significant programme responsibilities. In order to build on this platform, national adaptations and inform future actions for implementation. strategic planning and the issuance of standardized guidance are needed, which take into account various scale-up dimensions, including: resources, outputs, reach Recommendations and outcomes, as well as vertical and horizontal aspects. A multi-sector and multi-disciplinary national committee This feasibility assessment proposes the following key or task force should drive this effort, with priorities placed actions for the institutionalization and scale-up of PLH-YC on the development of a National Parenting Strategy, as in Montenegro: well as a National Action Plan and Guidelines for PLH-YC. The National Action Plan should be informed by several Domain 1: National strategic planning and guidance critical inputs, including: a review of laws, policies and 1. Creation of a multi-sector National Committee or sector-level guidelines; a review of continuum-of-care and Task Force service gaps; community consultations and research with 2. Development of a National Action Plan on PLH- parents/caregivers; and research with practitioners and YC service providers. Macro-level considerations that should 3. Issuance of National Guidelines for PLH-YC feature in the National Action Plan include: the setting 4. Setting of a wider National Parenting Strategy of objectives, targets and indicators; mechanisms for intra- and inter-sector coordination, as well as coordination Domain 2: Staff recruitment and capacity building with and across municipalities; activities for national 1. Identification and training of new PLH-YC staff programme promotion; the establishment of a National 2. Coordination of training across sectors Registry or other system for routine data collection; and 3. Organization of cascade training the articulation of funding commitments and criteria for reimbursement of service providers’ expenses. Domain 3: Programme reach 1. Encouragement of local programme promotion For the purpose of increasing coverage to new 2. Provision of guidance on balancing both universal geographical areas, as well as expanding delivery by the and targeted recruitment of parents/caregivers existing service providers, the National Action Plan should 3. Development of cross-sector referral procedures include a strategy for recruiting, training, supervising and retaining PLH-YC staff, with an emphasis on facilitators Domain 4: Programme adaptation and innovative and supervisors. The strategy should aim to organize approaches centralized and multi-sector capacity building, with 1. Issuance of guidance on parameters for local recognition of training completion across the health, adaptation education and social welfare sectors, as well as additional 2. Pursuit of innovative approaches, such as accreditation within the health sector. A cascade training ParentChat plan should be instituted, articulating the multi-year strategy for reaching an annual target number of parents/ Domain 5: Programme monitoring, evaluation and caregivers through the training of new facilitators, further research supervisors and in-country trainers. 1. Establishment of a PLH-YC National Registry or other system for national data collection 2. Development of a national research agenda on PLH-YC

8 Scaling up Parenting for Lifelong Health in Montenegro: A feasibility assessment Scaling up Parenting for Lifelong Health in Montenegro: A feasibility assessment 9 1. Introduction

Violence against children hile children living across a range of socio- W economic conditions, cultures, ethnicities and comprises all forms of country income categories have been victimized, estimates of rates of violence tend to be higher in physical abuse, emotional low- and middle-income countries than in high-income abuse, neglect and contexts [4]. In Montenegro, according to population- based data from the 2018 Multiple Indicator Cluster exploitation, resulting in Survey, 66% of children aged 1–14 years experienced at least one form of physical punishment or emotional “actual or potential harm” aggression in the past month alone [5]. to the “health, survival, In comparison with their high-income counterparts, development, or dignity” low- and middle- income countries are relatively less equipped with the evidence-based policies, programmes of a person under 18 years and accompanying resources necessary to effectively of age [1, 2]. Such violence prevent and respond to such harm, with devastating consequences. A large body of research indicates that is widely prevalent, with violence against children leads to adverse outcomes during childhood, and extends over the life span well recent global estimates into adulthood. These negative impacts include poorer revealing that at least one physical and mental health, child behaviour problems, higher rates of self-harm and criminality, reductions billion children have been in learning achievement and employment prospects, engagement in higher-risk behaviours, as well as a greater subjected to such harm in likelihood of continuing the cycle of violence through child the past year alone [3]. maltreatment and intimate partner violence as an adult [6–10]. During early childhood, due to this unique period of both rapid brain development and high dependency parent–child relationships. Studies have shown that such emotional maltreatment in multiple country contexts on adult protection and care, children are particularly interventions can effectively diminish rates of violence [18, 22–24]. These programmes bolster parenting skills vulnerable to these negative effects. Given the greater against children, as well as improve a range of outcomes using collaborative methods and practical guidance on use of health, welfare and criminal justice services by relevant to increased risks of child maltreatment, such positive parent–child interactions, non-violent discipline victims and their families, as well as losses in educational as child behaviour problems, emotional difficulties, techniques, problem solving, communication and investments and work productivity, one study estimates parenting competence and stress, maternal depression coaching about emotions, and responsive supervision that the costs to societies and governments may be as and couple interaction quality [13–17]. Furthermore, [25, 26]. Triple P, Incredible Years and Parent Management high as 8% of global GDP [11]. these interventions can also improve related protective Training Oregon are examples of such programmes, factors, such as positive parenting behaviour, parent– which all share similar content and utilize modelling, In response to these trends and to mitigate such child interaction, parental feelings of competence and role-playing and group discussion techniques, while repercussions, violence prevention has taken an developmentally responsive care and affection [18, 19]. encouraging parents to view themselves as pivotal to the increasingly prominent position on the world stage. It has Programmes for parent and caregiver support can be change process [27–29]. been incorporated into two UN Sustainable Development delivered through individualized approaches, group- Goals (SDGs 5 and 16) and was emphasized by the based programmes, or a combination of both methods. Analyses of the cost-effectiveness of these social-learning- UN Committee on the Rights of the Child in General Individual-based interventions can include behavioural theory-based programmes have repeatedly demonstrated Comment No. 8 (the right of the child to protection parent training or psychotherapeutic programmes cost savings over the long term, despite the fact that the from corporal punishment and other cruel or degrading delivered by professionals in clinical settings or trained costs of delivery in high-income contexts are considerable. forms of punishment) and General Comment No. health workers or community volunteers through home An analysis of the findings from a population trial of Triple 13 (the right of the child to freedom from all forms visits [20]. On the other hand, group-based programmes P in one county in the U.S., in which the programme of violence). It was also the impetus for INSPIRE, a are often delivered by trained staff in centres or was made available to all parents regardless of risk global initiative to promote a set of evidence-based community-based settings through regular, collaborative status, determined that the cost of US$2.2 million to strategies to prevent violence against children [12]. group sessions [21]. train service providers was recovered in a single year by reducing instances of child maltreatment by just Photo: One core strategy emphasizes the provision of parent Duško Miljanić / and caregiver support, with the aim of reducing harsh In particular, a considerable body of literature supports 10% [30]. In Sweden, the Incredible Years programme UNICEF Montenegro and abusive parenting practices and bolstering positive social-learning-theory-based parenting programmes, was estimated to cost US$1,302 per child, with a net with systematic reviews demonstrating effectiveness monetary benefit of US$11,614 by the end of childhood in reducing harsh parenting and prevent physical and (18 years) in comparison to controls [31]. Implementation

10 Scaling up Parenting for Lifelong Health in Montenegro: A feasibility assessment Scaling up Parenting for Lifelong Health in Montenegro: A feasibility assessment 11 of the same programme in Ireland is yielding an internal illustrated stories), encouragement of peer support, and subsequently launched by UNICEF in collaboration with sustainability at the national level in Montenegro. Given rate of return – or the rate of return on an investment that individualized tailoring of content during consultations, the government and with the support of the European the considerable progress to date in adapting, training excludes external factors – of 11% over the long run, with phone calls and chat groups. Union in late 2017. The project encompassed programme and piloting the programme since early 2018, as well as this conservative estimate based on reductions in child adaptation, training of service providers, supervision, the embedding of delivery within the existing government behaviour problems, leading to improved educational The key parenting skills that are emphasized in the and evaluation of the programme effects. To date, 48 health, education and social welfare systems, outcomes, reduced unemployment and reduced criminal programme include: spending quality time together, professionals, including psychologists, social workers, Montenegro is well-positioned to undertake this study activity [32]. child-directed play, socio-emotional regulation, positive pedagogues, nurses and preschool teachers, have been at such a crucial juncture in programme implementation. reinforcement through praise and rewards for desirable trained as PLH-YC facilitators, while six of these have This activity is part of the EU-funded, 3-year initiative been trained and certified as PLH-YC supervisors. From “EU and UNICEF for Early Childhood Development in Parenting for Lifelong Health for Young behaviour, instruction giving, setting limits and household rules, non-violent discipline techniques and brief January 2018 to January 2021, six cycles of PLH-YC were Montenegro” (2020–2023) [47]. Children mindfulness-based activities for stress management. delivered to a total of 458 parents and caregivers1 across Through the application of these skills to interactions with five municipalities: Podgorica, Nikšić, Berane, Bijelo Polje The specific objectives of this assessment are to: For many low- and middle-income countries, the initial their children at home, the programme is hypothesized to and Cetinje. A description of the different phases and 1. Document evidence of implementation progress outlay for ‘brand name’ parenting programmes, including directly improve several parenting outcomes, including: results of PLH-YC implementation in Montenegro are to date and the effectiveness of PLH-YC, both in expensive fees for licensing, materials, training and reductions in abusive and harsh parenting, dysfunctional detailed in section 3 of this report. Montenegro and internationally; support, places such interventions out of reach [33]. parenting and support for corporal punishment; increased 2. Propose modalities for institutionalizing the To overcome this challenge, as well as to respond parental confidence and use of positive parenting skills; Purpose and objectives programme in terms of coordination and to the particular needs of low-resource contexts, the and improved child monitoring and supervision [15, planning, programme delivery, monitoring and Universities of Oxford, Cape Town, Stellenbosch and evaluation, drawing on examples from other 18, 23, 36, 37]. In turn, such changes contribute to UNICEF, with support from the European Union, has Bangor, together with UNICEF and WHO, developed the countries; additional positive effects for parents, such as reductions commissioned this feasibility assessment for the purpose Parenting for Lifelong Health initiative (https://www.who. 3. Cost the programme, utilizing a costing tool that in depression, anxiety and stress, as well as a decrease of facilitating PLH-YC institutionalization, scale-up and int/teams/social-determinants-of-health/parenting-for- in intimate partner violence [38–42]. Positive changes can be customized in the future as variables lifelong-health). Parenting for Lifelong Health comprises a in parenting behaviour are also hypothesized to impact change; and suite of four evidence-based programmes that are tailored various child outcomes, including reduced child behaviour 4. Share reflections on a broader strategy for parenting support in Montenegro with a to child development stages and are freely available to problems and improved child socio-emotional regulation 1 This refers to the number of parents and caregivers who continuum of services/programmes. low- and middle-income countries. Originally developed [15, 43]. As a direct consequence of reductions in abusive ‘enrolled’ in PLH-YC (i.e. attended at least one programme for low-income families in Cape Town, South Africa, and harsh parenting, as well as indirectly through indirect, session). these programmes are suitable for adaptation in various improved parental and child outcomes, rates of child cultural and contextual settings and have been tested maltreatment are reduced – although the evidence more or implemented in over 28 countries in Latin America, strongly favours reductions in physical and emotional Figure 1. Theory of Change model for Parenting for Lifelong Health for Young Children [44] Southeast Asia, Africa and Europe. abuse of children over reductions in child neglect.

One of these four programmes, Parenting for Lifelong This Theory of Change model has been supported by Direct Effect Health for Young Children (PLH-YC), is a group-based several randomized controlled trials (RCTs), which PLH Theory of Change programme targeting parents and other primary constitute one of the best methods for determining Indirect Effect caregivers of children aged 2–9 years. The PLH-YC programme effectiveness. To date, these trials have programme was developed in a staged manner, been used to evaluate PLH-YC programmes in South involving the identification of common core components Africa, the Philippines and Thailand. The study in South PLH-YC Parenting Direct Parenting Indirect Adult Core Components Skills Outcomes Outcomes associated with evidence-based parenting programmes, Africa comprised 296 parents and caregivers and such as the Incredible Years, Parent Management showed significant positive effects immediately after the Training Oregon, Triple P and Parent–Child Interaction intervention, including less physical and psychological • Practicing skills • Quality time • Reduces • Reduced mental Therapy. The developers also examined various delivery discipline (28% and 14%, respectively), as well as a & role-plays together abusive & harsh health problems methods and other implementation factors that would 39% increase in observed positive-parenting practices • Facilitator • Child-directed parenting • Reduced enhance parent engagement in the programme, such as [45]. In the Philippines, the trial included 120 parents and empathy play • Reduced intimate partner adequate supervision of programme facilitators, as well caregivers, and demonstrated a significant 23% reduction • Modelling • Socio-emotional dysfuntional violence as assuring accessibility for low-income families [34]. The in child maltreatment one year after intervention. parenting parenting regulation Reduced original version of the programme consists of 12 weekly Finally, in Thailand, there was a 58% reduction in child practices • Positive • Increased sessions, although a subsequent, eight-session version Violence maltreatment among 120 parents and caregivers, as • Collaborative reinforcement parental has been developed and was shown to be effective in well as 40% less parental mental health problems and Against faciliation • Giving confidence Thailand [35]. 60% less child behaviour problems at three months after Indirect Child Children • Group problem instructions • Increased programme completion [46]. Outcomes solving • Limit setting, positive As depicted in the Theory of Change model in Figure parenting skills 1, these common core components are designed to Drawing from such initial positive findings, and on the • Supportive rules, monitoring • Improved • Reduced child strengthen parenting skills through utilizing delivery heels of a national campaign on violence against children materials (e.g., • Non-violent monitoring & behaviour methods and programme content that is discussed by the Government of Montenegro, UNICEF and EU audio-visuals) discipline supervision problems in weekly group sessions. These methods include during 2016–2017, PLH-YC was presented in Montenegro • Peer support • Stress reduction • Reduced support • Improved practicing parenting skills in groups using role plays as in 2017 by two programme developers: Professor Frances • Individual for corporal socio-emotional well as at home, demonstrating facilitator empathy, Gardner (University of Oxford) at a UNICEF and EU- tailoring & punishment regulation modelling parenting practices, adopting collaborative supported conference and Professor Judy Hutchings support facilitation techniques, group problem solving, the use (Bangor University) at a WHO-supported conference. of supportive materials (such as parent handbooks and A multiphase initiative to implement PLH-YC was

12 Scaling up Parenting for Lifelong Health in Montenegro: A feasibility assessment Scaling up Parenting for Lifelong Health in Montenegro: A feasibility assessment 13 2. Methodology

Study management Study approach

The feasibility assessment he methodology for this assessment is underpinned T by a mixed methods approach, utilizing a combination was conducted by an of desk review and primary qualitative and quantitative research. A mixed-method approach capitalizes on the international consultant strengths of both quantitative and qualitative methods [48, 49]. This assessment adopted a convergent design [50], who was contracted by which allows for obtaining “different but complementary UNICEF Montenegro from data on the same topic” (p. 122) [51]. The convergent design provides opportunities for the researcher to obtain December 2020 to March a more holistic understanding of the research problem, enabling comparison of quantitative results (e.g. facilitator 2021. satisfaction surveys) with qualitative findings (e.g. facilitator feedback on the opportunities and challenges faced when delivering the programme), and illustrating quantitative results with qualitative findings (or vice versa).

Although a convergent design was adopted, the emphasis was on the collection of qualitative data through individual interviews and focus group discussions (FGDs). This made use of the existing abundance of quantitative data on the implementation of PLH-YC in Montenegro – particularly: facilitator feedback on training received in 2018; coach/supervisor feedback on training and supervision received in 2019; demographic data on programme participation; process outcomes their participation was voluntary, and that they could the appropriate staff at UNICEF Montenegro, so that a (e.g. recruitment, enrolment, attendance, completion); withdraw at any time without negative consequences. suitable course of action could be taken in line with child outcome evaluation data; and parental satisfaction. The invited participants were only thereafter asked if they protection laws in Montenegro and international child would like to provide verbal, informed consent. At the rights principles. No such disclosures were made during The researcher strove to adhere to the General Norms for beginning of each interview and FGD, the participants this assessment. Evaluation as indicated in the United Nations Evaluation were assured that the information they provided would Group Norms and Standards for Evaluation [52]. These be kept confidential, without associating the information norms underscore the importance of utility, credibility, provided with their identities. This approach was Study process independence, impartiality, ethics, transparency, human undertaken to reduce potential selective feedback and rights and gender equality, national evaluation capacities limit social desirability bias. Both translators involved As depicted in Figure 2, the assessment followed a and professionalism. in primary data collection signed a confidentiality series of steps. Following initial orientation with UNICEF agreement, so that any sensitive data that was shared Montenegro, the researcher undertook the following: Ethical considerations was not disclosed to others and would not be traced to its source. The audio/video files, transcripts, survey 1) Desk review: UNICEF provided the researcher with Given that this assessment was a low-risk study that did feedback and researchers’ notes were maintained by and a range of requisite materials for the desk review. not meet the threshold criteria for review by the UNICEF only accessible to the researcher. In several instances, The researcher reviewed: relevant laws, policies Ethical Review Board, prior ethical approval was not respondents providing written feedback also shared their and strategies; analytical reports related to national sought for this study. The researcher took steps to apply inputs with UNICEF, although it was specified in writing in efforts to promote early childhood development, the principle of “do no harm” by collecting data in an advance that this feedback should only be sent directly to as well as the status of health, education and appropriate and respectful manner, taking into account the researcher. social welfare service delivery systems; qualitative researcher–participant dynamics, the cultural context and data on PLH-YC implementation and facilitator/ potential protection risks. All those invited to participate Given that the research participants included those with caregiver satisfaction; and quantitative findings in individual interviews and FGDs were provided with direct contact with vulnerable families, an exception to a on the programme effects following a pre-post Photo: design evaluation. In addition, the researcher pulled Duško Miljanić / information sheets ahead of time, which stated the breach of confidentiality would occur if any information UNICEF Montenegro intention of and process for data collection, the fact that was disclosed that a child had been seriously harmed or together information from academic papers and was at risk of serious harm. If such a rare event were to PLH-YC implementation manuals on critical areas to occur, the researcher was prepared to discuss this with include in monitoring and evaluating PLH-YC process

14 Scaling up Parenting for Lifelong Health in Montenegro: A feasibility assessment Scaling up Parenting for Lifelong Health in Montenegro: A feasibility assessment 15 outcomes and effects, as well as potential modalities management, early childhood development, Figure 2. Feasibility assessment process for programme institutionalization and scale-up within adolescent development, child protection, and Desk review: current country coordination and planning systems, education. FGDs were conducted according to programme delivery capacities and processes, and a semi-structured format, following an interview 1. Relevant laws, policies and strategies mechanisms for regular data collection and analysis. guide grouped by theme. Each FGD took 2. Analytical reports of key sector and service delivery systems approximately 90 minutes. 2) Development of assessment methodology: 3. Qualitative data on PLH-YC implementation 1 The researcher developed the methodology for • Written feedback: Although not part of the 4. Quantitative data on PLH-YC facilitator satisfation the assessment, including participant information original research protocol, written feedback 5. Qualitative data on parent/caregiver satisfaction sheets and tailored guides for individual interviews was solicited via email from those invited and FGDs, dependent on the profession of the professionals who were unavailable to 6. Quantitave data on programme effect participant. Research participants were also identified participate in individual interviews and FGDs in 7. Adademic literature on parenting programme effects, costs and scaling-up based on discussions with UNICEF and lists of person. Key questions drawn from the interview efforts in other countries PLH-YC developers in the UK, and facilitators and guides and tailored to their experience with supervisors in Montenegro. Finally, for the purpose PLH-YC delivery were submitted via email. With Development of assessment methodology of obtaining insights regarding direct feedback from one exception, written responses were sent parents and primary caregivers, the researcher directly to the researcher, who then forwarded 1. Identification of research participants adapted the Obstacles to Engagement Scale, a these to a professional translator for translation. 2. Development of data collecting tools 15-item instrument that measures the influence One respondent chose to share her responses 2 of personal and intervention-based obstacles to directly with UNICEF via telephone, who then 3. Consultation with UNICEF Montenegro attending parenting programme sessions [53]. forwarded this feedback to the researcher. 4. Finalization of methodology report

3) Primary data collection: During late December • Online survey: The researcher used Primary data collection 2020 and early January 2021, the researcher utilized SurveyMonkey, an anonymous online survey four methods to collect primary data through: a) platform, to administer questions from the 1. Identification and recruitment of research participants individual interviews; b) FGDs; c) key questions sent Obstacles to Engagement Scale. The survey link 2. Conducting of individual interviews (N=16) via e-mail to solicit written feedback; and d) an online was shared with UNICEF, who then forwarded 3. Conducting of four focus group discussions (total N=19) 3 survey. Individual simultaneous Montenegrin–English it to PLH-YC facilitators. Facilitators were translation was provided via a professional interpreter requested to share this link with the 68 parents/ 4. Collection of written feedback (N=6) for most interviews, and through two interpreters caregivers who did not successfully complete 5. Administration of online survey with parents/caregivers (N=7) for two FGDs. Individual interviews and FGDs were the programme (which requires attendance of at conducted using the Zoom online meeting platform, least seven out of the 12 sessions). Only seven Data analasys and video and audio recordings were made. responses were submitted to the platform. 1. Data transcription and translation • Individual interviews: Individual interviews 4) Data analysis: Responses during the individual 2. Development of coding framework for qualitative data were conducted with 16 participants with the interviews and FGDs were transcribed into English 4 following professional profiles (see Appendix using auto-transcription software; transcripts were 3. Data coding A): two directors of primary healthcare centres then crosschecked for accuracy against the audio and 4. Integrate qualitative and quantitative data to draw key conclusions that had delivered PLH-YC; three directors of video recordings, as well as against the researcher’s 5. Examine implications for institutionalization and scaling-up kindergartens that had delivered PLH-YC; three handwritten notes. For one interview, due to a directors of local non-government organizations technical error, a human-generated transcription Report writing & finalization (NGOs) that had delivered PLH-YC; three service was used to transcribe responses in ministerial officials from the Ministry of Health, Montenegrin, which were then translated into 1. Report drafting Ministry of Education and Ministry of Finance English using Google Translate. 2. Dialogue with UNICEF on report findings and Social Welfare; two departmental directors 5 from the Institute for Public Health and the In order to analyse qualitative data from the 3. Report finalization and sharing with research participants Institute for Social and Child Protection; one interviews and FGDs, the research utilized thematic faculty dean from the University of Montenegro; analysis and the application of a hybrid approach. and two PLH-YC programme developers. A hybrid approach combines deductive methods Interviews were conducted according to a semi- that are driven by the theoretical underpinnings of structured format, following an interview guide parenting research [54] with an inductive approach grouped according to several themes. Each [55]. A coding framework was developed based develop the guidance and estimates on programme feedback, in order to clarify the assessment findings interview took approximately one hour. on the semi-structured guide questions, as well costs, the researcher drew from the desk review and and recommendations, and to ensure that the format as emergent patterns and themes from the data the detailed costing information provided by UNICEF, and language of the report was suitable for the target • Focus group discussions: Four FGDs were [56]. The researcher then applied these codes to as well as from feedback obtained through the audience. It was also shared with two academics in respectively facilitated with: 1) four facilitators/ the transcripts using the qualitative data analysis interviews and FGDs. the field of parenting, including a PLH-YC programme supervisors from the health sector; 2) five software NVivo 12 Pro. De-identified quotes by developer, for their inputs. In order to uphold the facilitators/supervisors from the education participants were then selected which represented 5) Report writing and finalization: The draft report independence of the assessment and to maintain sector; 3) three facilitators/supervisors the key themes, as well as areas of disagreement. was submitted to UNICEF Montenegro, the UNICEF study credibility, all decisions regarding the final from NGOs; and 4) seven UNICEF staff Quantitative data collected through the online survey Regional Office for Europe and Central Asia, and version of the report were made by the researcher with responsibilities in country programme was analysed by summing responses. In order to an external peer review facility (Universalia) for alone.

16 Scaling up Parenting for Lifelong Health in Montenegro: A feasibility assessment Scaling up Parenting for Lifelong Health in Montenegro: A feasibility assessment 17 3. Implementation status of Parenting for Lifelong Health for Young Children in Montenegro

A) Overview of Preparation implementation to date s depicted in Figure 3, this effort began with the Ainitial sharing of the programme aims, design Since late 2017, UNICEF and methods by PLH programme developers through conferences in Podgorica organized by UNICEF and and the Parenting for WHO in 2017. Following consultations between UNICEF Lifelong Health initiative, representatives and service managers from a range of statutory and voluntary service providers across the in partnership with health, education, and social welfare sectors, a multi- stage project was formalized between UNICEF and Early the Government of Intervention Wales Training Ltd., the training arm of the Montenegro and with registered charity Children’s Early Intervention Trust (CEIT) in Wales, to deliver training and consultation on PLH-YC support from the implementation.

European Union, have As part of the preparatory phase, the original 12-session made considerable version of PLH-YC, as initially designed in South Africa, was adapted for the Montenegrin context. Support for core lesson and group discussion centred on illustrated Hutchings (Bangor University) and Dr Jamie Lachman progress in implementation this endeavour was provided by Clowns Without Borders stories; f) practicing of new parenting skills through role (University of Oxford). A total of 24 participants attended, South Africa and Dr Mihela Erjavec, a Montenegrin plays; g) assignment of home practice activities based on spanning a range of relevant backgrounds, including of PLH-YC in Montenegro. senior lecturer at Bangor University. Given that the newly learned skills; h) a closing mindfulness activity; and 12 psychologists, five nurses, three pedagogues, original version was developed for Xhosa-speaking i) the sharing of emotions to ‘check out’ of the group. three preschool teachers and one social worker. The families in Cape Town, several important adaptations participants spanned five municipalities (Podgorica, were undertaken. These included: the creation of new Akin to the original programme, the programme in Berane, Kotor, Bijelo Polje and Nikšić), and included five illustrated story characters, attire and contexts so that Montenegro includes one-on-one consultations with Healthcare Centres, two kindergartens, three NGOs they were more relevant to a European audience; parents and caregivers prior to the first session, which and a centre for children with behavioural problems (see modifications to African storytelling so that culturally can be held in community settings or in participants’ Appendix B for the full listing of facilitators, supervisors relevant stories and activities could be incorporated; and homes. The purpose of these consultations is to provide and participating organizations). The training included: the translation of the programme into Montenegrin. The parents and caregivers with an opportunity to learn more content that modelled empathy, collaborative discussion, core content of the programme remained mainly intact, about the programme and ask any questions they may role plays and specific activities from the programme with minor modifications. This content included key have, while it also allows facilitators to learn more about (e.g. naming feelings); the use of the Accept–Explore– parenting skills that are covered over the course of 12 each participant, his or her child(ren), and the wider Connect–Practice (AECP) facilitation method and weekly sessions (see Figure 4), which are also depicted in family. In addition, facilitators conduct one-on-one ‘catch- activities; and the application of building blocks to the ‘House of Support’ model (see Figure 5) that is used up’ sessions with those parents and caregivers who facilitating home activity discussions, guiding group by programme facilitators to describe the progression miss any sessions or require additional support. Finally, practice and leading illustrated story discussions. During of participants through the programme as they build a facilitators are encouraged to conduct weekly phone calls the fifth day of the training, managers were invited to strong ‘house’ for their child. with each participant when possible, as well as to create attend with participants in order to discuss plans for PLH- and engage in online parent group chats (such as through YC implementation. The participants’ evaluations of the Each session has a duration of approximately 2–2.5 the Viber application), in order to foster peer sharing training were very positive, 85% of the responses to 13 hours, with about 12 parents and caregivers per group. and motivate home practice activities in-between group evaluation form items indicating that they were “very” or Each session also includes the following activities: a) sessions. brief mindfulness exercise for stress management (i.e. “completely” satisfied. ‘taking a pause’); b) sharing of emotions to ‘check-in’ to Photo: Phase 1 From January 2018 through early 2019, 17 newly trained Duško Miljanić / the group; c) physical exercise; d) group discussion on UNICEF Montenegro home practice activities from the previous session; e) facilitators worked in pairs at their respective work sites During Phase 1, a five-day training for PLH-YC facilitators to deliver the PLH-YC programme over three cycles. was conducted in October 2017 by Professor Judy During the first cycle, eight parenting groups were

18 Scaling up Parenting for Lifelong Health in Montenegro: A feasibility assessment Scaling up Parenting for Lifelong Health in Montenegro: A feasibility assessment 19 Figure 4. Summary of programme Figure 3. Overlapping phases of content for PLH-YC 12-session version PLH-YC implementation Walls: Positive Parenting

Session 1: One-on-One Time with Your Child Session 2: Say What You See 0 1 2 3 4 Session 3: Talking About Feelings Session 4: Praising & Rewarding Our Children Preparation: Phase 1: Phase 2: Phase 3: Phase 4: Session 5: Giving Positive, Specific & Realistic Instructions Partnership 1st facilitator 2nd facilitator Supervisor Planning for building & training, pro- training & training & institutionalization adaptation gramme delivery programme coaching & scale-up Roof: Limit Setting & Discipline

& evaluation delivery delivery Session 6: Houselhold Rules & Routines • Sharing of PLH-YC • Oct. 2017: 5-day • Sept. 2019: 5-day • Oct. 2019: 2-day • Dec. 2018: Parlia- Session 7: Redirecting Negative Behaviours programme aims, PLH-YC Training for PLH-YC Trainin for PLH-YC Training of mentary commit- design & evidence 24 professionals 24 additional pro- 6 supervisors tee recommends Session 8: Replacing Negative Attention Seeking & Demanding Behaviours with the Govern- • Early 2018 - early fessionals • Coaching of super- national expansion through Ignoring & Giving Attention to Other Behaviour ment of Montene- 2019: Programme • Late 2019 - late visors by Prof. Judy • 2018 & 2019: Session 9: Using Consequences to Support Compliance gro and UNICEF delivery to 188 par- 2020: Programme Hutchings Accreditation of • Adaptation of PLH- ents over 3 cycles delivery to 270 • Coaching of new facilitator and/or Session 10: Cool Down for Aggressive Behaviours YC materials with of parenting groups parents over 3 supervisor trainings facilitators by new Session 11: Resolving Conflicts in the Family support by Clowns • Coaching of cycles of pareting supervisors by ISCP and BOE Without Borders facilitators by Prof. groups (including • 2021: Training of South Africa and Judy Hutchings online session PLH-YC Trainers Closing: the University of during cycles 1-2 during COVID-19 • 2021: Beginning of Bangor lockdown) • Pre-post evaluation ParentChat pilot Session 12: Reflection & Moving on of programme • 2021: Feasibility effects during assessment for Community Celebration (optional) cycles 1-2 systems integration & scale-up

established, followed by five during the second, and six From late 2019 to late 2020, the PLH-YC programme was 2019), supervisor trainees provided coaching to all new realization of the Strategy on the Prevention and during the third. Eight supervision sessions of facilitators, delivered in another three cycles by both generations facilitators in pairs, with pairings made on the basis of Protection of Children from Violence (2017–2021) [57]. divided into two groups, were provided during the first of facilitators: nine from the first cohort and 20 from the sectors that they came from (e.g. two supervisors cycle by Professor Judy Hutchings, either in person or the second cohort. During the fourth cycle, 13 parent from two NGOs, two from two Healthcare Centres In addition, the training of PLH-YC facilitators and via Skype. Facilitators were asked to bring a completed groups were formed, followed by 12 groups during and two from one kindergarten). Subsequently (during supervisors was accredited by the National Institute self-reflection report and a five-minute video recording the fifth, and one during the sixth cycle. Given that the spring 2020), mixed-sector pairs were formed. Feedback for Social and Child Protection in 2018 and 2020, from their group to each supervision session, with such COVID-19 pandemic affected delivery during the fifth from trained supervisors was obtained at the end of the respectively, while the Bureau of Education accredited videos highlighting challenging interactions between the and sixth cycles, half of these sessions were delivered course, with all responses to eight items consisting of the training for facilitators in 2019. These accreditations facilitator and a parent from the previous group session. online due to the lockdown (spring 2020) or self-isolation “very” or “completely” satisfied. make participating in such training appealing to of facilitators (autumn 2020). Supervision was provided professionals who are interested in continuing their professional development, which is sometimes mandated At the end of the first cycle, the process and outcome by six experienced facilitators trained as supervisors by Phase 4 evaluation data was collected and analysed, along with Professor Judy Hutchings (see Phase 3). or encouraged by employers or professional organizations. feedback on parental satisfaction (see section 3.F below). Phase 4 of implementation, currently ongoing, focuses In addition to this feasibility assessment, other efforts Phase 3 on institutionalization of the programme within have been planned for 2021 that will further promote Phase 2 national systems, as well as planning for scale-up PLH-YC scale-up. The first is the organization of the Phase 3 provided an opportunity to develop a cohort of in- and sustainability. To date, this phase has included: a first training of trainers, which will draw from the pool Building on the successes of the previous phase, Phase country PLH-YC supervisors. Six experienced facilitators, conference in June 2018 to present the results from of existing supervisors in Montenegro. This cohort will 2 consisted of a second round of facilitator training who had previously delivered the programme on three the first cycle; advocacy by programme implementers then be in a position to train additional facilitators, and for a new cohort, delivered in September 2019 to 24 occasions, were recruited to attend two-day supervisor and UNICEF for programme sustainability; and a subsequently to accredit facilitators and identify potential participants, including six psychologists, five preschool training in October 2019, with content emphasizing recommendation issued on 3 December 2018 from supervisors. There are also plans for Montenegro to teachers, five pedagogues, three social workers and five practical skills. The training included modelling of live the Committee on Human Rights and Freedoms of the participate in a six-country pilot study of ParentChat, nurses. The new trainees hailed from seven municipalities coaching of a facilitator by Professor Judy Hutchings, Parliament of Montenegro that “most efforts should an online PLH parenting support group programme for (Podgorica, Nikšić, Kotor, Cetinje, Bijelo Polje, Berane and and then supervision of trainee supervisor coaching be invested in strengthening family support services” parents and caregivers of children aged 2–17. This effort Herceg Novi), and included four Healthcare Centres, four practice over four sessions, followed by an assessment and that PLH-YC should be expanded to all relevant will overlap with and reinforce the ongoing promotion and kindergartens and four NGOs. of a full supervision session. Initially (during autumn institutions in Montenegro, given that it enables the expansion of PLH-YC programme delivery.

20 Scaling up Parenting for Lifelong Health in Montenegro: A feasibility assessment Scaling up Parenting for Lifelong Health in Montenegro: A feasibility assessment 21 The following sections delve into more specific aspects caregivers of children aged 2–9 years into the programme target the ones who really had problems with house rules expressed that such collaboration had been or would be regarding the implementation of PLH-YC in Montenegro, (see Appendix B), with recruitment following the general and discipline with their children. Because we worked fruitful. One health facilitator explained: “I’m in a bit of drawing from both reports and primary research data. guidance that they were being invited to learn about with those families, so we know that, we know that.” favourable position, because I closely cooperate with the These include: insights into the programme delivery positive parenting. Some facilitators used existing However, another recounted the procedural challenges Centre for Social Work and some of the parents were process, including participant recruitment, enrolment, caseloads for recruitment, as they were already familiar in approaching those outside her caseload: “The problem actually already using the services of the centre for social completion and drop-out; participant profiles and feedback with families exhibiting harsh parenting practices or facing was that everything I do has to undergo certain formal work – we’re in everyday contact. And we exchange regarding their experiences of the programme; service challenges in managing difficult child behaviour. procedures. Parents of children with disabilities were my information on a daily basis.” Another health facilitator provider experiences during training and programme primary target population, [as] they were the only ones suggested: “In my opinion … it would be best to have delivery; the utilization of monitoring and evaluation Health sector. Facilitators in primary healthcare centres that had open case files and that was the only way to this kind of programme delivered in close communication systems; findings from the pre-post evaluation of the first used the websites of their centres to notify the public that evidence my work. In cases where I wasn’t working with and cooperation with the Centre for Social Work, like the delivery cycle; and the costs of training and delivery. the programme was open for recruitment. In addition, [these parents in my caseload] … it was hard for me to case in Berane.” they spoke to parents who were accessing services at prove that I was working on something.” B) Programme delivery process the counselling centre, relied on word of mouth, and Referrals from kindergartens to NGO providers. A also left programme leaflets in corridors and waiting Education sector. Facilitators who were based in facilitator from an NGOs described that their close rooms. One centre that utilized the centre website to kindergartens also used dual strategies to recruit potential cooperation with preschools in Podgorica was a critical Participant recruitment promote PLH-YC described needing to create a waiting parent and caregiver participants on a universal as well as and effective way of obtaining parent and caregiver list, due to high demand, while the facilitator who targeted basis. Social media channels such as Facebook referrals to the programme. The facilitator stated: “We Unlike the context in South Africa where PLH-YC was disseminated leaflets noted that demand was relatively and kindergarten websites were utilized to reach out communicate and cooperate [with kindergartens] on originally developed, the facilitators in Montenegro were low. One facilitator described how the recruitment of to interested parents in general, while psychologists a daily basis. The … pedagogues and psychologists … embedded within various service providers, with some vulnerable families was emphasized, in the following mentioned using their counselling centres to identify were the first to indicate the need for this programme already working with clinical populations. This provided way: “Depending on the age of their children and the families in need of more support, such as those parenting to some parents. … We continued our collaboration with ample opportunities for facilitators to invite parents and problems that parents were facing, we were trying to children with disabilities. Another noted: “We also had the local kindergartens and we based the referrals on that contact with parents through our Counselling Centre channel of communication, our primary way of recruiting for parents in case we recognized that the parent could parents and sending messages, sending the message of benefit from the programme. We would engage a bit prevention.” Figure 5. House of Support model more, then we would go into more detail explaining to them how this kind of parenting programme could help.” Composition of parent groups NGO sector. NGO-based facilitators were able to readily Several facilitators discussed the composition of their reach out and recruit vulnerable parents/caregivers parenting groups, and how they managed the challenges Sunshine of in need of support, given the nature of their existing of engaging with families with diverse risk backgrounds, Positive Attention work. According to one facilitator: “… We have over 200 while keeping the group coherent and on track. Many families that are socially vulnerable, that we support described group members with low-risk profiles who in many other ways. So, we reached out to them first. were simply eager to improve their parenting skills, We also reached out to our single-parent family groups, alongside those contending with complex or multiple groups [receiving] support, and we asked them first.” challenges, such as domestic violence or parenting Communication via social media platforms also expedited children with special needs. These facilitators expressed this process: “There are a couple of, let’s say, channels that facilitating groups with mixed compositions was of communication that we use with our targeting groups. actually ideal, as it fostered the exchange of peer support One of them is our community on Facebook. So, we have and sharing of good parenting practices with those a large community … and that’s what makes things easier who were struggling. One health facilitator noted: “We for us.” Roof: Limited have the best experience when we have very mixed groups of parents coming from various backgrounds, Setting and Cross-sector referrals with children with various problems and issues, children Roof Discipline with typical or atypical development. We did not have Problem-Solving Some facilitators shared that they utilized cross-sector any problem … having them together, because some of referrals in order to identify parents who could most these issues are universal to all.” Another health facilitator Nonviolent Discipline benefit from the PLH-YC programme; however, most shared: “Whenever you have a mixed group, those from Instruction Giving and Household Rules revealed that, while such coordination would have been vulnerable groups can see that other parents are using useful, they did not tap into such channels. … techniques or approaches in this kind of interactive

programme … they may learn from their peers. They can Walls Walls: Collaboration with Centres for Social Work. In the health change, based on the advice which can come from other Using Praise and Rewards Positive sector, some facilitators, such as those based at the parents.” However, one NGO facilitator cautioned that a Healthcare Centres in Berane and Bijelo Polje, utilized balance was necessary, in order to ensure that facilitators Parenting Naming Feelings and Actions existing referral systems with local Centres for Social could handle potentially competing demands: “So I think Quality Time with Your Child Work due to membership within multidisciplinary teams. that up to half of the group is OK … with having people Others, such as the Healthcare Centre in Nikšić, preferred from socially vulnerable groups. But more than that, I Esablishing Parent Goals to identify families that they were already in contact with, don't think it’s good.” and were concerned that liaising with local Centres for Social Work would culminate in an overload of referrals. Despite this, most of those who were interviewed

Scaling up Parenting for Lifelong Health in Montenegro: A feasibility assessment 23 Participant engagement C) Participant profiles and experiences of the programme Engagement in parenting interventions can be described During the FGDs and interviews for this assessment, as encompassing three aspects: attendance (enrolment, service providers shared some of the factors that they degree of attending sessions and completion of the perceived had encouraged participant engagement Demographic profiles programme), adherence (level of compliance with the and contributed to the overall success of programme programme, including in group discussions and home implementation thus far. One education facilitator Basic demographic data collected from 472 parents/ A total of 28 participants (29.8% of those who reported practice activities), and cognitions (e.g. participant underscored that facilitator–participant rapport and the caregivers who completed pre-programme surveys during any difficulties) disclosed that they faced at least two expectations, therapeutic alliance and participant emphasis on peer exchange and group problem solving cycles 1–6 indicated that nearly nine out of 10 participants or more of these challenges, underscoring the multiple satisfaction) [58]. The level of participant engagement is were all vital contributors: “… Parents understood that were mothers, while almost one in 10 were fathers (see adversities that are present in some households (Figure an important determinant of how beneficial a parenting they were not alone. … They trust the professionals Figure 6). The average participant age was 34.8 years, 8). programme will be to a parent or caregiver; moreover, assisting them and the best possible thing is that they are with most having a child aged 2–9 years who was a boy the degree of engagement can provide indications of surrounded by parents who make the very same mistakes (57.8%). Figure 6. Parent/caregiver relationship to a child aged 2–9 years the relevance of the programme to parental needs, as … It takes only one session for them to gain trust and well as the extent of support by other family members to feel reassured … by session 12 you end up having Almost one in five participants (94 persons) to date for the parent/caregiver’s attendance [59]. Low levels of a very strong network of people working on the same reported at least one of the following types of difficulties engagement may be a red flag that the programme is problem, supporting each other…” An NGO facilitator in their respective households (see Figure 7): 9.7% not sufficiently attuned to the cultural context of delivery, shared that being flexible and attuned to the needs of • Food insecurity (running out of money for food or that implementation quality is low, or that barriers to parents/caregivers was important to keeping participants essentials in the last month); programme accessibility exist (e.g. lack of transport, engaged: “There were some situations where we would • Adult illness (an adult who is very unwell – in childcare needs, programme scheduling during working see parents drop off … Some parents would start the hospital or in bed a lot of the time); hours) [59, 60]. programme and then just decide to leave it. We are trying • Child illness (a child in the household who is very to adapt to their needs. And the challenge was whether unwell); As is presented in Appendix B, there were relatively high we could keep the same group of parents throughout • Potential child disability (a child who has trouble rates of enrolment (participants who attended at least the cycle. We did manage to overcome that by setting hearing, seeing, talking or walking, or who one session), of completion (participants who attended the pace and the time that suited them.” Finally, an struggles at school); 89.7% at least seven out of the 12 sessions), and of attendance NGO working with Roma parenting groups noted that • Household conflict (problematic arguments with (the proportion of the total group sessions attended by delivery in the Albanian and Romani languages, as well as shouting or hitting); and enrolled participants) across all cycles of programme coordinating with local Roma mediators, were key factors • Alcohol (people with problem drinking or taking delivery and for almost all the delivery sites. The lowest to effective engagement. This underscores the fact that drugs). rate of attendance was 46.3% for one group delivered for this particular population, additional community-based at the Healthcare Centre Podgorica during cycle 3; staff may be essential and should be incorporated into Mothers Fathers Other caregivers however, the attendance rates during all the other cycles budgets and planning accordingly: “We would need to delivered at this site were 93.8% or higher. It should also have Roma mediators as individuals who would be liaison be underscored that such rates remained high during officers. They would visit the community, they would Figure 7. Types of difficulties faced by participants reporting at least one household difficulty cycles 5 and 6, when approximately half of the group check on the parents – whether they need additional sessions were delivered on-line due to social distancing assistance or help before the next workshop.” requirements. A total of seven groups out of a total of 46 across all cycles obtained an attendance rate of 100%. Although the levels of engagement were high overall, Table 1 describes the overall engagement rates for all six the Healthcare Centre Podgorica, Healthcare Centre Potential child disability cycles, with 96% of those who were recruited enrolling in Nikšić, and the NGO Centre for Child Rights respectively PLH-YC and 89.3% of those who enrolled completing the supported three parenting groups that had low rates programme, with an average attendance rate of 86.1% of attendance during the first and third cycles. While Child illness across all sessions. In comparison, the randomized barriers to attendance would have been best assessed by controlled trial of PLH-YC in South Africa reported an obtaining information directly from the parent participants in those three groups, in the view of one health attendance rate of 70% (for group sessions plus home Household conflict visits), while this rate was 86.6% (for group sessions) facilitator, one potential reason is that some of these for the eight-session version of PLH-YC during its trial in participants may have experienced multiple adversities or challenges with their children at home that affected their Thailand [35, 45]. Alcohol or drugs engagement. The facilitator indicated: “I believe that they had quite a lot of family issues to deal with at that time Table 1. Participant rates of enrolment, completion and attendance and those issues influenced their regularity of attendance. over six cycles These were usually families with very complex family Adult illness histories and background and rather problematic children No. % that they had to deal with on a daily basis. When I say Total participants recruited 477 - problematic and challenging, I’m referring to children Food insecurity Total participants enrolled 458 96.0% with certain disabilities. And in addition to those children, they also had other children who had certain behavioural Total participants completed 409 89.3% 0 7 14 21 28 35 issues.” Total sessions attended 4,730 86.1% Types of difficulties in household

24 Scaling up Parenting for Lifelong Health in Montenegro: A feasibility assessment Scaling up Parenting for Lifelong Health in Montenegro: A feasibility assessment 25 Feedback neutral response to this question. Most (three people) were either neutral or agreed (one person) that the belief Figure 8. Number of difficulties faced by participants reporting at least one household difficulty Although more extensive inputs from parents and that parenting programmes have little connection with caregivers are still needed, especially from those with the problems in their family prevented their attendance. relatively low levels of engagement, the feedback Specific questions about whether other reasons posed received from participants regarding PLH-YC has been obstacles to attendance (e.g. a lack of trust in the 80,00% very positive. Following the end of the first programme facilitators, transportation issues, fear or nervousness) did not elicit any responses in agreement. However, when . delivery cycle in April 2018, satisfaction questionnaires were administered to 68 participants by independent asked an open question about whether there were any researchers. Parents/caregivers were asked to rate seven other obstacles, three respondents pointed out that the items on a five-point scale, ranging from very dissatisfied movement to online group sessions due to the COVID-19 60,00% (1) to very satisfied (5), with the mid-point being a pandemic had prevented their attendance. One parent/ neutral score (3). As anticipated, given the high levels of caregiver cited family obligations due to the pandemic, programme attendance, overall satisfaction ratings yielded and another shared that a lack of time and child illness a high average rating of 4.74, with scores for specific were impediments. question items represented in Figure 9 [61]. 40,00% D) Experiences of PLH-YC facilitators and supervisors In addition, interviews conducted by UNICEF with several parent participants respondents reveal how the . programme has altered their parenting practices and Enablers and opportunities 20,00% affected their relationships with their children. According to Aleksandra Boričić, mother of a 6-year-old boy and of In recounting their experiencing in both delivering the an 11-year-old girl with disabilities, her decision to enrol in PLH-YC programme and supervising other facilitators, . PLH-YC was one of the best she has made in her life. She service providers underscored a number of enabling . recounted: “I am quieter, more collected, I have changed factors and opportunities that made successful implementation possible. Such aspects largely centred 0,00% the way I react in certain situations, which would earlier 1 2 3 4 make me lose my temper. I have more understanding on five key enablers and opportunities: 1) personal towards children and their behaviour. For example, in case motivation; 2) professional development and application he would break something and I would react impulsively. to existing work with families; 3) supportive management Now, I perceive the whole matter from the perspective of and an enabling work environment; 4) high-quality a six-year-old and I can see that it is not that terrible.” [62] supervision; and 5) community collaboration.

Figure 9. Parental satisfaction of the programme following the first PLH-YC delivery cycle In addition, Nusret Bešo, a 59-year-old father living and 1. Personal motivation and commitment. Service with seven children ranging from 6 to 19 years of age, providers repeatedly noted that personal motivation described that he had a positive experience participating and commitment, fuelled by their observations of the in a parenting group organized by the NGO Centre for programme’s impact on parents/caregivers, as well Roma Initiatives. He explained that he uses the learned as their own experiences as parents, inspired their Benefits for child enrolment in the facilitator training, as well as their behaviour parenting skills daily, and has shared these techniques with his wife and neighbours: “They see that I understand willingness to continue programme delivery. As noted by one Healthcare Centre director: “They’re all parents Achieved personal goal my children better now, as well as that my children understand me better. There are no more quarrels. When themselves, so they recognized the benefits of the I get angry at my children, I explain the situation to them programme, and they felt sorry that they had not had Parenting skills helpful so that they know why I am mad and then we solve the this programme when they had younger children.” A issue by talking about it. Children need our attention and kindergarten facilitator also explained: “We are really Confidence in the future it is our duty to show them the right path.” [63] proud as facilitators and we would feel sorry if this programme failed to become an integral part of our work, and something sustainable and scaled up. We see Facilitator quality Obstacles to engagement the benefits of this programme, and it can offer a lot to children and families. Not only locally, but also nationally – Supportive group Using a survey based on the Obstacles to Engagement this is a way to heal one’s society.” Scale and administered through an online survey Recomment programme platform, seven parent/caregiver participants – who did 2. Professional development and application to others not successfully complete the PLH-YC programme – to existing work with families. Service providers shared their views regarding the factors that prevented highlighted that their participation as a facilitator and/ Overall satisfaction their engagement. Almost all respondents (six out of or supervisor was also attributed to opportunities for seven) agreed that not having enough time to attend professional development, as well as recognition that 0 1 2 3 4 5 parent meetings for several weeks in a row stopped the programme had a diffusion effect – it enhanced their them from attending or fully attending PLH-YC. Three own and their colleagues’ existing work with families, agreed that their work schedule stopped them from with benefits accruing to families beyond those in the attending or fully attending, while three others had a PLH-YC parent groups. According to one kindergarten

26 Scaling up Parenting for Lifelong Health in Montenegro: A feasibility assessment Scaling up Parenting for Lifelong Health in Montenegro: A feasibility assessment 27 director, who referred to a facilitator who is a preschool their assignments.” The director of a kindergarten also teacher: “She makes use of the skills and knowledge pointed out: I believe that PLH as a programme can be that she gained … to help her peers to work with parents sustainable in our community because we have sufficient and children. She’s also more supportive and offers more spatial resources, … sufficient professional resources, professional knowledge to the parents she communicates and we have brilliant long-standing cooperation with the with on a daily basis. So I would say the benefits of local community parents … Our good work with parents this programme were great, both institutionally and can only help to improve the quality of the pedagogical personally.” A Healthcare Centre director also shared the work.” following: “They [the facilitators] informed me that they were really satisfied, because this programme helped Challenges and obstacles them to perform their daily tasks in a more routine fashion.” PLH-YC service providers also faced several challenges and obstacles to programme implementation which 3. Supportive management and an enabling affected the timing, quality, and coverage of delivery. work environment. Many service providers shared In some cases, these difficulties delayed the delivery that managers were supportive of their involvement of PLH-YC, while in others, such obstacles were so in programme delivery; that flexible scheduling, considerable that parent groups could not be organized, workload, and workplace facilities were conducive to and delivery was brought to a standstill. These prohibitive implementation; and that the initial small-scale funding factors largely focused on the following themes: 1) provided by UNICEF were all important contributors insecure or insufficient financial support; 2) difficulties to success. An education sector facilitator remarked: with scheduling and logistics; 3) heavy workloads; 4) “Thanks to the conditions that we have at the local unsupportive management; 5) venue limitations; and 6) kindergarten and thanks to the work of our management, impediments due to the COVID-19 pandemic. we have great conditions for implementing the Photo: programme, and I’m particularly satisfied with the 1. Insecure or insufficient financial support. Service Duško Miljanić / UNICEF Montenegro response.” One health-sector facilitator also emphasized: providers often raised the issue of insecure or insufficient “We did run smoothly concerning organizational issues. financial support, and how this either prevented or We agreed quite easily about the schedule and we tried threatened to halt programme delivery. Facilitators to adapt to the needs of parents, not to us and our needs, mentioned that such funds were needed to cover and we would reshuffle our schedules so that we could facilitator wages, including for overtime, as well as for our work, territorially speaking, having in mind that we 4. Unsupportive management. Service providers who deliver the programme at a time that suited the parents.” refreshments during parenting group sessions. This was Finally, the director of one NGO provider remarked: “They particularly a problem for NGO facilitators, who were cover a number of municipalities.” One director of an were unable to continue to deliver the programme also [the facilitators] could absorb it. We would reduce their not compensated for programme delivery, as well as NGO shared: “… Two of our colleagues who implemented underscored that their managers were unsupportive of burden in terms of working with families in the field.” for some education facilitators, who worked with their the programme would have to travel once a week … and unresponsive to their involvement as facilitators. managers to formally request additional wages from the they live in Podgorica and they have to travel to deliver According to one facilitator: “The management was not 4. High-quality supervision. Several service providers Ministry of Education. According to one NGO facilitator: the programme. So this was one of the key obstacles to there for us to promote and advertise the programme, underscored the importance of the high-quality “The management and the institution provides full continuous, sustainable delivery.” In the health sector, one so we lacked support from them in that regard … the supervision that they received from Prof. Judy Hutchings, support … However, when it comes to finances, the facilitator who also could no longer deliver PLH-YC, noted: organizational problems were many. … I tried hard as well as the six supervisor trainees in Montenegro. One organization is not able to provide additional funding “Well, the obstacles were many, they were huge. … I to reach the management and to talk with them. But health sector facilitator, a psychologist, noted that this that could enumerate the facilitators. So that was one of had a hard time selecting my colleague to work with me … management has a very bad attitude toward … supervision was instrumental in guiding her to adopt the the obstacles and challenges that we have that affects as a pair. None of my colleagues who are psychologists my work…” Another facilitator from a different sector collaborative delivery methods that are essential to PLH- sustainability…” Another NGO facilitator, who could not … were willing to participate. Only one medical nurse simply noted that there was a “lack of support from the YC: “Having in mind that I have been doing this job for continue with programme delivery, stated: “You normally accepted. … So it was really hard for us to adjust our institution’s management.” many years … I get a bit involved in solving the problem do many things in parallel, and we wanted to do this as working hours to the programme because her shifts as a psychologist – a bit more than I should. The work well. … But we do not have additional funding. We are would change weekly and mine would change daily.” 5. Venue limitations. Limitations on access to of the supervisor helped me overcome this challenge project-based. We receive project financing and that’s appropriate venues for hosting parenting group sessions, of mine. I think it is necessary to provide supervision it.” Finally, one education facilitator indicated: “… the last 3. Heavy workloads. Heavy existing workloads were which require space for physical exercise and working in services from the very beginning, and I have to praise our cycle led to certain questions of whether the Ministry also mentioned as a major obstacle to service delivery pairs, were also considered an obstacle for some service supervisors because they were excellent.” of Education would pay for our engagement … We did in some workplaces. This was cited as the main reason providers. One official noted that Centres for Social Work not want the programme to come to a halt. The ministry why Centres for Social Work were unable to commit lacked “spatial resources” and that their “venues were 5. Community collaboration. Several service providers received our formal request. We explained the number of staff to facilitator training and delivery. According to one not suitable” for delivery of a programme like PLH-YC. shared that close collaboration with local beneficiary hours, and we were financially supported by the ministry.” government official: “It’s very hard for [Centres for Social Two facilitators from two different healthcare centres communities was a strong enabling factor in quality Work] to organize their everyday duties and to conduct noted that space was an issue, with one sharing: “I had programme delivery. For facilitators working with Roma 2. Difficulties with scheduling and logistics. Several their everyday work, having in mind that they have these to deliver it in a separate building and there were certain parents and caregivers, such cooperation was essential, service providers discussed the challenges in coordinating really reduced capacities…” One NGO facilitator also problems with asking the caretaker to open the facility as emphasized by one NGO director: “We went on travel and schedules between pairs of facilitators, as emphasized that such demands were prohibitive: “As an and to unlock it.” One NGO director explained that they home visits because of the lack of the use of mobile well as arranging weekly sessions outside of normal outreach worker, you work with 10 families – [but] we could only deliver PLH-YC to very small groups of parents/ phones by the community. In addition to us, we also working hours. An NGO facilitator who could not continue currently work with 100 families, 330 children, and there caregivers, and that when members dropped out, they had a Roma mediator who lives within the community. to deliver the programme stated: “… We have two are only two of us who are coordinating the work. So decided not to encourage them to rejoin due to limited So all of us would go to their homes and we would facilitators. There’s me and another representative from many details to cover. You supervise, you coordinate, you available space: “So, the main issue and challenge that visit those families and we would remind them … of this organization. It was a bit of a problem to organize keep records. So, we lack the human capacity to do that.” we had was related to the size of the room available to

28 Scaling up Parenting for Lifelong Health in Montenegro: A feasibility assessment Scaling up Parenting for Lifelong Health in Montenegro: A feasibility assessment 29 us. For this particular reason we did not push for greater problems and parental mental health (secondary Chained Equations. Multivariate Linear Mixed Models F) Programme costs involvement of those who were hesitating and eventually outcomes of interest). were utilized to account for random effects [70], with dropped out … Meaning that we need to have sufficient the level of statistical significance (p < 0.05) assessed to Similar to other evidence- and group-based parenting funding, for example, to rent a proper room and to deliver Procedures. The assessment utilized a pre-post design, determine the likelihood that programme effects were not programmes, quality programme delivery and the programme…” with a questionnaire administered prior to the first due to chance. maintenance over time require careful consideration of parenting group session in January 2018 and again costs. Cost considerations in low- and middle-income 6. Impediments due to the COVID-19 pandemic. after the end of the 12th and final session in April, Results. Findings demonstrated a large reduction countries are more considerable impediments to the Service providers recounted the challenges they faced approximately 4-5 months post-baseline. Data was in overall harsh and abusive parenting, with a 70% scalability and sustainability of parenting interventions in in delivering the programme due to the pandemic, collected at pre-test by PLH-YC facilitators, and by an reduction in the average rates (Figure 10). The average low- and middle-income countries than in high-income including: the necessity of switching to online delivery for independent research not associated with programme rates of physical discipline fell by 86%, while emotional countries, given the substantial challenges in building some sessions, preventing those families without online delivery at post-test, in order to limit social desirability discipline was reduced by 60%. As depicted in Figure 11, human resource capacity, generating sufficient and long- access from taking part; social distancing requirements reporting bias. Ethical approval for the study was granted there were also medium and large improvements in the term financial resources, as well as linking scale-up with leading to limitations in group size; and the removal of by UNICEF Montenegro and the Universities of Cape average rates of supporting positive behaviour (11%) and macro-level funding mechanisms [73, 74]. Although PLH- available space for infirmary needs in Healthcare Centres. Town and Oxford. limit setting (14%). There was a large reduction in the YC has a Creative Commons licence and is freely available One NGO facilitator shared: “We lost a few parents, overall average scores related to dysfunctional parenting to LMICs, thus eliminating expensive licensing fees I think two or three parents, in our last group in the Participants and service providers. A total of 82 parents (25%), as well as a medium reduction in the average and material costs, several major costs will need to be middle of the programme, because they were unable and caregivers were recruited into the study, with 79 rates of overall child behaviour problems (31%). Finally, covered through a combination of government and locally to attend an online version of it. They didn’t have an completing pre-test questionnaires, 71 enrolling in the there was a large reduction in average scores on parental generated funds on an ongoing basis. While a detailed internet connection or other things that were needed.” programme and 68 completing post-test questionnaires. depression (45%); however, there were no significant table of the various budget items is provided in Appendix A kindergarten facilitator pointed out: “So one of the Seventeen PLH-YC facilitators delivered the programme, effects on parental stress. C, the major budget considerations are outlined below. challenges that comes to my mind is the organization including nine psychologists, four nurses, two preschool Main limitations. Given the lack of a comparison group, of workshops during COVID-19 … the programme was teachers, one pedagogue and one social worker. The the results should be treated with caution. Findings Programme delivery planned for 12 participants … we had to reduce the facilitators delivered the programme in the context of from a pre-post, non-randomized trial cannot reliably number … [to] seven parents.” regular service delivery at their respective workplaces, be attributed to the intervention and can only suggest Given that the PLH-YC materials are relatively minimal, which comprised eight different service providers: four potential programme effects [71]. Furthermore, post-test the most significant costs for programme delivery lie Healthcare Centres, three NGOs and one kindergarten E) Programme impact assessments were conducted shortly after programme in fees for facilitator wages. Fees will need to take into (see Appendices A and B). completion, making it uncertain whether the effects will account the following: At the end of the first delivery cycle, during Phase 1 of be sustained, diminished or delayed over the medium to • Whether delivery is part of the usual workload of PLH-YC implementation (see Figure 3), an evaluation was Measures. Measures included in the evaluation are listed long term [72]. a facilitator (no additional pay); funded by UNICEF Montenegro and conducted by the in Table 2. • Whether no additional pay will be given, but the Universities of Bangor, Cape Town and Oxford to assess Figure 10. Programme effects on primary outcomes usual workload of the facilitator will be shifted the programme’s effects on harsh and abusive parenting Analysis. Following an intention-to-treat design, data onto another staff member; (the primary outcome of interest), as well as positive from all 82 participants was included in the analysis, with • Whether delivery is in addition to the usual parenting, dysfunctional parenting, child behaviour missing data imputed using Multivariate Imputation by workload of a facilitator, and whether he/she will be paid a flat honorarium or percent wage Table 2. Evaluation measures increase; and/or • Whether delivery is in addition to the usual Outcomes Instruments Measures workload of a facilitator, and he/she will be paid overtime (by the hour). Harsh and abusive parenting International Society for the Preven- Frequency of emotional and physical Harsh & abusive Physical Emotional tion of Child Abuse and Neglect Child disciplinary practices parenting discipline discipline Fees paid to facilitators may also be sector-specific. Abuse Screening Tool (ICAST) [64] According to feedback received via the FGDs, the following wages have been given to date: Positive parenting Parenting Young Children (PARYC) Frequency of supporting positive • Health-sector facilitators: €2.70 per hour scale [65] behaviour and limit setting • Education-sector facilitators: €100 per month Figure 11. Programme effects on secondary outcomes • Dysfunctional parenting Parenting Scale (PS) [66] Parental attitudes and beliefs regard- NGO-sector facilitators: No compensation ing permissive discipline, lengthy reprimands that rely on talking, and authoritarian discipline

Child behaviour problems Strengths and Difficulties Frequency of child behaviour prob- Questionnaire (SDQ) [67] lems, including emotional symptoms, conduct problems, hyperactivity, peer problems and prosocial behaviour

Parental stress Parent Stress Scale (PSS) [68] Parental feelings associated with parenting stress Supporting Setting Dysfunctional Child behaviour Parental positive limits parenting problems depression Parental depression Center for Epidemiological Studies Frequency of parental feelings behaviour Depression scale (CES-D) [69] associated with depression

30 Scaling up Parenting for Lifelong Health in Montenegro: A feasibility assessment Scaling up Parenting for Lifelong Health in Montenegro: A feasibility assessment 31 Table 3. Sector-specific fees for programme facilitators *To be determined Table 4. Programme delivery cost considerations

Sector Rate/hour Fee per month Fee for one parent group (12 sessions) Item Item specification Potential variations

Health €2.70 €137.00 €410.40 Parent Handbook • Printing or photocopying of one book • Variations in parent group size per participant Education €100.00 €300.00 Flip chart stand • One stand per parent group • None NGO * * * Misc. supplies • House of Support poster, LCD projec- • Variations in parent group size Sector Rate/hour Fee per month Fee for one parent group (8 sessions) tor, screen, name tags, markers, flip chart paper, toys (for role plays), cray- Health €2.70 €140.40 €280.80 ons, adhesive, pens, stickers, etc.

Education €100.00 €200.00 Facilitator time & wages • About 1.5 days per facilitator • Fees have varied by sector (see Table 3) per week during programme delivery Facilitation role is part of or an addition NGO * * * • Two facilitators required to existing workload per parent group • Variations in parent group size (about • About one hour per family for 10–15) Table 3 provides the estimated total fees for facilitation staff, as well as travel, subsistence and accommodation pre-programme consultations • Pre-programme and catch-up consulta- of one parenting group by two facilitators, on the basis for facilitators who may need to travel to another site for tions may be conducted through home that the facilitators are paid for the full estimated number delivery. visits or at Healthcare Centres/kinder- of working hours to deliver the 12-session version of PLH- gartens/ NGO premises YC in order to ensure quality (19 working days; see Table • Number of weekly working hours varies 4). The estimated fees for the eight-session version of the Facilitator supervision & staff training depending on whether parents/care- programme (13 working days), should this be delivered in givers have intensive support needs or the future, are also shown for comparative purposes. Another set of budget considerations includes fees for facilitator supervision and the provision of staff training for require weekly catch-up sessions A total of 19 days per facilitator is estimated to cover facilitators, supervisors and trainers (see Table 5). Venue & food • Venue rental, electricity costs, • Variations in parent group size the time needed for delivery of the 12-session version, cleaning fees • Lunch may be provided if appropriate for including time for parenting group meetings, session Supervision for facilitators involves viewing recorded • Refreshments for session breaks the community setting preparation, completion of process monitoring and video clips of facilitators leading parent group sessions; Lunch (if appropriate) evaluation forms (e.g. pre-assessment surveys, for this purpose, video camera equipment for each pair • Some service providers may not have participant case profiles, attendance, phone call/online of facilitators is needed. Online supervision sessions can adequate facilities (e.g. NGOs), so rental chat group logs, fidelity check-lists, self-reflection forms be organized, which helps to save costs. The degree of may be necessary for supervision sessions), and participation in supervision supervision also varies depending on the level of facilitator sessions. Additional time may be needed depending on experience. Facilitator certification is normally based Phone calls & moder- • Weekly phone calls to • Variations in parent group size whether parents/caregivers require additional one-on- on an assessment of facilitator performance following ation of online group parents/caregivers • Variations in family needs (some partici- one consultations due to missing a session (i.e. catch-up two cycles of parent group delivery. Once this has been discussions Weekly online group communications pants may need more support) consultations) or intensive support needs. achieved, supervision is not required for the third cycle (e.g. through Viber) • Depending on parent/caregiver access of delivery onward; however, ongoing supervision is to mobile phones/internet Other costs which may need to be covered for generally beneficial and should be costed if possible. The programme delivery include venue and refreshments/ current rate set for supervisors is €100 per day. Accessibility provisions Item sapecification Potential variations food, depending on the facilities available to the service provider and whether it would be appropriate to provide Training of facilitators is conducted by two certified Childcare services • On-site childcare services for partic- • Number of children to be supervised meals. Mobile phone service and internet access may trainers, who delivers the training in pairs to a maximum ipants/communities where lack of need to be covered for facilitators, for the purpose of of 24 participants. Training of supervisors is conducted provision would be a barrier making weekly phone calls and facilitating online regular by a certified trainer, who delivers the training to 6–8 group chats. participants. Finally, training of trainers is conducted by a certified master trainer, who delivers the training to 2–4 Transport for participants • Transport or transport costs for partici- • Mode of transport and distance Additional costs that may need to be covered include participants or more, depending on local needs. pants in communities where distance/ accessibility provisions to address barriers to participation, transport fees are a barrier such as childcare services and transport or transport Other costs which may need to be covered for cost reimbursement for those parents/caregivers who supervisions and trainings include venue and would otherwise be unable to attend sessions. On-line refreshments/food, depending on the facilities available to Online delivery Item specification Potential variations delivery of some or all of the parent group sessions the service provider. Additional costs that may be incurred and associated costs may also need to be covered if will depend on service provider staff arrangements Online meeting platform Facilitation of online parent groups for Depending on facilitator and parent/care- necessary, due to social distancing needs. and locations include wages for any coordination or fees and internet access partial or all of group sessions giver access to laptops/smart phones/ Costs not included in Table 4, and which will depend administrative staff, as well as travel, subsistence, and internet on service provider staff arrangements and locations, accommodation for trainers and participants who may include: wages for any coordination or administrative need to travel to supervision or training venues.

32 Scaling up Parenting for Lifelong Health in Montenegro: A feasibility assessment Scaling up Parenting for Lifelong Health in Montenegro: A feasibility assessment 33 Table 5. Cost considerations for facilitator supervision and staff training

Supervision for Item specification Potential variations Trainer training Item specification Potential variations facilitators Materials • Markers, flip chart paper, adhe- • Number of trainer trainees Materials & equip- • Video cameras, tripods, memory • Number of facilitators per supervision group sive, pens, etc. ment cards (one set per facilitator pair) • Misc. supplies (e.g. paper, folders) Trainer times & • One trainer for 3–4 trainers • None wages • Wages for preparation & delivery Supervisor time & • Wages of €100 per day • Supervision will vary depending on facilitator (one day preparation, two days wages • One supervisor coaches eight experience: training) facilitators • For 1st-time facilitators: eight sessions (four working days) per 12-week programme Trainer trainees • Travel to training venue • Number of trainer trainees • For 2nd-time facilitators: three sessions (1.5 • Daily subsistence • Distance between trainer residence and training working days) per 12-week programme • Accommodation venue • For 3rd-time facilitators: two sessions (one working day) per 12-week programme Venue & food • Venue & equipment rental for • Number of trainer trainees / Distance between supervision meetings trainer residence and training venue Venue & food • Venue & equipment rental for • Distance between facilitator residence and • Refreshments/lunch supervision meetings supervision venue • Refreshmentsa • Online supervision will not incur these costs

Facilitator training Item specification Potential variations

Materials • Facilitator Manuals • Number of facilitator trainees • Name tags, markers, flip chart paper, toys (for role plays), crayons, adhesive, pens, stickers, etc.

Trainer time & wages • Two trainers per 24 facilitators • None • Wages for preparation & delivery (one day preparation, five days training)

Facilitator trainees • Travel to training venue • Number of facilitator trainees • Daily subsistence • Distance between facilitator residence and • Accommodation training venue

Venue & food • Venue & equipment rental for • Number of facilitator trainees supervision meetings • Distance between facilitator residence and • Refreshments/lunch training venue

Supervisor training Item specification Potential variations

Materials • Markers, flip chart paper, adhe- • Number of supervisor trainees sive, pens, etc.

Trainer time & wages • One trainer for 6–8 supervisors • None • Wages for preparation & delivery (one day preparation, two days training)

Supervisor trainees • Travel to training venue • Number of supervisor trainees • Daily subsistence • Distance between supervisor residence and • Accommodation training venue

Venue & food • Venue & equipment rental for • Number of supervisor trainees Photo: supervision meetings • Distance between supervisor residence and Duško Miljanić / • Refreshments/lunch training venue UNICEF Montenegro

34 Scaling up Parenting for Lifelong Health in Montenegro: A feasibility assessment Scaling up Parenting for Lifelong Health in Montenegro: A feasibility assessment 35 Table 6. Cost considerations for monitoring, evaluation and dissemination of their working hours allotted to delivery. However, data collectors and analysts are €90 and €150 per day, additional staff will ideally be required to administer respectively. Data collection & Item specification Potential variations post-assessment and satisfaction surveys to participants, analysis in order to reduce the likelihood of biased responses Following data analysis, staff will be required to produce from participants, as well as to consolidate this data at regular reports (e.g. quarterly, biannual, annual) on the Data collection • Group monitoring forms • Variations in parent group size central levels. Furthermore, data analysts will be needed implementation progress and outcomes. Dissemination materials (e.g. attendance, phone calls, • Electronic or paper-based collection to clean and analyse both the process and outcome meetings should be organized involving policymakers Viber group facilitation) (e.g. e-tablets) data. At times, this may involve following up with and relevant professionals to review the status of • Facilitator self-reflection forms • Subscription to data collection software service providers to verify inaccurate or missing data implementation, and to make adjustments to national- • Facilitator fidelity checklists (e.g. ODK) from parents/caregivers. Those analysing the outcome and municipal-level plans. This information should also assessments will need to merge data, check scores and be shared with communities to both inform them of • Coaching report forms reverse codes, run basic descriptive statistics, address progress and promote the programme, ideally utilizing • Participant case profiles missing data, as well as run analyses using Excel, SPSS, accessible print and electronic media. • Pre-post assessment surveys R, or other statistical software. Estimated fee rates for

Data collectors • Estimated at €90 per day • Pre- and post-test assessments can occur • Responsible for collecting post-test data online or on paper by the parent/caregiver • Responsible for consolidating data from • Data collectors can collect data on paper all service providers or using e-tablets

Data analysts • Estimated at €150 per day • Complexity of statistical models to be • Responsible for using statistical soft- applied ware to assess implementation progress • Number of participants in dataset and outcomes

Evaluation dissem- Item specification Potential variations ination

Report writers • 1–2 writers to consolidate analysed data • Length and frequency of report and draft reports summarizing implementation progress and outcomes

Report layout • Layout of annual report • Number of pages and designs • Graphic design

Report printing • Production of annual reports • Number of printed reports needed • Electronic reports will not incur these costs

Dissemination • Dissemination meetings for • Online meetings will incur lower costs meetings policymakers and professional staff • Promotional materials can include • Dissemination meetings for brochures, leaflets, videos for internet or communities conference viewing, PSAs for TV or radio • Promotional materials

Monitoring, evaluation and uploaded, and/or through web-based applications with dissemination links sent to participants for self-completion. Paper- based administration is a low-cost method, but requires good systems for paper management and time for data The final set of budget considerations relates to the checking and entry. Electronic administration, with proper collection, analysis and dissemination of monitoring and safeguards, generally leads to less missing data and evaluation data (see Table 6). As discussed further on data entry error. However, participant self-completion in section 4.F, such data includes process information (e.g. of pre-post assessment surveys) requires e-literacy on programme reach, engagement, fidelity, quality and as well as access to a smart phone/computer device acceptability, as well as outcome measures to assess and the internet, which may not be possible for some effectiveness. marginalized populations or individuals. Photo: Data collection materials include various process and Duško Miljanić / impact forms, which can be administered on paper, Process data relating to programme reach and fidelity UNICEF Montenegro using e-tablets issued to service providers with forms can be maintained by programme facilitators as part

36 Scaling up Parenting for Lifelong Health in Montenegro: A feasibility assessment Scaling up Parenting for Lifelong Health in Montenegro: A feasibility assessment 37 4. Moving forward: institutionalization and scaling up

The progress achieved iven that PLH-YC is a relatively young programme Gand has only been piloted as part of routine during phases 1–3 of service delivery in three countries (Montenegro, the Philippines and Thailand), there are limited cross-country the preliminary testing comparisons to be made and few exemplary good and implementation of practices and scaling-up strategies on which to draw. However, Montenegro has already demonstrated that it PLH-YC in Montenegro is well-positioned to become an international model of country readiness for PLH-YC implementation on a large has demonstrated scale, given the existence of: supportive laws, policies considerable success in and institutions; skilled and knowledgeable human resources; complementarity of the existing family support terms of capacity building, programmes; as well as expressed commitment by key stakeholders, including those in the health, education and service delivery through social welfare sectors [75]. multiple sectors and for varied target groups, and A) Supportive laws, policies and institutions demonstration of initial “secure conditions for free and responsible parenthood” scientific research (Art. 148). Finally, financing for social positive effects on parents/ The existence of supportive laws, policies and institutional through various social, health and legal measures (Art. 7), and child protection services shall by funded by the state mandates is an important dimension of national echoing the state’s obligations under CRC Article 18. and municipal budgets, noting that the state will cover caregivers and their implementation capacity for scaling-up success, as well relevant services if municipal funds are insufficient (Art. as the country’s readiness for effective prevention of child The 2017 Law on Social and Child Protection appears 154). children. maltreatment [74, 75]. Montenegro has a well-articulated to position parenting support as a service in the “area of legal and policy framework that clearly delineates the social and child protection” (Art. 60), with “counselling- There are several key overarching and sector-focused rights of children to prevention of and protection from therapy and social-educational services” including strategies also related to prevention of violence against violence, as well as the responsibility of state institutions services that help overcome situations of crisis and children and parenting support services. The 2017–2021 across sectors to actively provide assistance to parents in improve family relations (Art. 63). This law also outlines Strategy on the Prevention and Protection of Children their child-rearing duties. the role and duties of Centres for Social Work, including: from Violence underscores the overall complexity of conducting assessments of beneficiaries; creating this area as well as the necessity of a multi-disciplinary Articles 5 and 9 of the 2016 Law on Amendment of the and monitoring individual service plans; and deciding approach to violence, with the Ministries of Social Family Law, in line with Article 19 of the Convention on applications for the exercising of social and child Welfare, Health, Education, Justice, Human and Minority on the Rights of the Child (CRC), commit the state to protection rights (Art. 114). The responsibilities of the Rights, Finance, and the Interior, as well as NGO undertaking all necessary measures to protect children Institute for Social and Child Protection also encompass representatives, all involved in the drafting process and from violence, and prohibit the corporal punishment various activities relevant to the implementation of tasked with activities under the National Action Plan. of children by all persons coming into contact with the parenting support, including: monitoring service quality; The strategy points out the strengths that each sector child, including parents and guardians. The law affirms organization of training; programme accreditation; brings to the table, with primary healthcare centres as that children are entitled to the “best possible living preparation of analyses and reports in the field of social the key institutions where prevention starts, schools circumstances favouring his proper and full physical, and child protection; coordination of the development as places where violence is often identified, and social mental and emotional development” (Art. 64), with of service standards; development, monitoring and service providers as uniquely positioned to prevent parents being responsible for exercising parental care evaluation of strategy implementation; and informing the violence through family support. Strategic Objective 2 in the child’s best interests (Art. 69) and with a duty professional and general public about the implementation (related to quality and integrated interventions to support to develop a parent–child relationship that is based on of relevant services (Art. 121). Furthermore, the law prevention), Strategic Objective 4 (on changing harmful Photo: affection, trust and mutual respect (Art. 71). In order to Duško Miljanić / stipulates that the competent state administration social norms and attitudes), and Strategic Objective 6 (on fulfil these roles, the law confirms that the state will UNICEF Montenegro authority should manage, maintain and supervise an improving national systems for monitoring, evaluation, overall information system for social and child protection, and research) are especially relevant to parenting support with data collection informing planning, monitoring and interventions.

38 Scaling up Parenting for Lifelong Health in Montenegro: A feasibility assessment Scaling up Parenting for Lifelong Health in Montenegro: A feasibility assessment 39 The 2019–2023 Strategy for Exercising the Rights of chronic non-communicable diseases are the improvement B) National strategic planning and also need to consider vertical scale-up (institutionalization the Child is another policy document that emphasizes of mental health and the reduction of injury, both standardized guidance through policy, political, legal, budgetary or other system inter-sector cooperation, and prioritizes the prevention strongly related to violence prevention and parenting changes) and horizontal scale-up (opportunities for service of all types of violence (Strategic Goal 3). The strategy support. In particular, the prevention of injury is noted expansion and replication) [74]. As a complex social Although there are relatively strong legal, policy, and also specifically highlights that there is no “continuous as requiring a multi-sector approach in order to achieve intervention which tackles deep-rooted social problems, institutional frameworks for implementing PLH-YC over programme for strengthening positive parenting skills” safer physical and social environments. The health care the scale-up of PLH-YC will need to address multiple the long term, the literature on implementation science in Montenegro, but specifically mentions PLH-YC as an of especially vulnerable and threatened populations is levels of the social system (e.g. individual, organizational, on scaling up pilot interventions, as well as feedback intervention currently in progress. Moreover, Activity 8.5 also underscored, with an emphasis on the health of community, economic and cultural environments), with obtained from interviews and FGDs for this assessment, under Operational Objective 8 states the government’s schoolchildren due to the significant impacts that risk a deliberate strategy incorporating timepoints for review, indicates that more needs to be in place at the national commitment to implementing positive parenting factors in this period of growth and development may reflection, and the documentation and promotion of level. Tested parenting interventions often fail after programmes during 2020, specifying funding that will have on their health across the lifespan. The ‘building success [78]. Finally, such planning must be realistic scale-up when developers and implementers have not support this effort. Other noteworthy commitments of public health’ is also promoted, with a role for the about available resources and system capacities, being undertaken the crucial formative work to ensure that include specification that the implementation of the Institute for Public Health in monitoring and evaluating careful not to impose too heavy a burden on inflexible or these are designed to fit the delivery system, and have Strategy will be coordinated by the Council for the the population’s health status and health care, as well fragile delivery systems [74]. not considered that the ‘scale-up’ context of wider Rights of the Child, that joint training programmes for as being tasked with prioritizing the implementation service systems are likely to have less supportive professionals will be organized, and that line ministries of preventive programmes in all health institutions. management, reduced capacities and additional pressing National committee leadership and will cooperate with institutions such as the Statistical Finally, the plan places importance on the continuous priorities in comparison to pilot conditions [74, 76]. Office (MONSTAT) and the Institute of Public Health to development of the skills of health staff through membership collect and process various data on children. education and training, as well as the importance of inter- It is crucial for strategic planning to carefully take into sector cooperation to secure “health in all policies.” In order to develop a realistic and well-informed National account all the scale-up dimensions, including resources The 2018–2022 Strategy for the Development of Action Plan, PLH-YC stakeholders in Montenegro should (workforce, supply, government expenditure), outputs the Social and Child Protection System, based on Furthermore, the 2019–2023 Strategy on the Protection consider forming a multi-sector national committee (access, scope, quality, efficiency), reach (coverage and the 2017 Law on Social and Child Protection, provides and Improvement of Mental Health in Montenegro or task force. This committee would be responsible utilization) and outcomes (reductions or improvements further elaboration on the important roles of various promotes the involvement of parents in preventative for mobilizing necessary inputs to the plan, exploring in behaviours and attitudes) [77]. Strategic planning will bodies and services in violence prevention, including programmes, allowing for the identification and early and addressing ‘macro-level’ factors that will enable or Centres for Social Work, the Council for the Rights intervention in child and youth behavioural disorders and of the Child, the Council for the Care of Persons with mental health problems. Objective 1, which emphasizes Disabilities, the Institute for Social and Child Protection, activities to promote mental health and prevent mental the family outreach worker service (provided through health disorders amongst the most vulnerable groups, Inputs National Committee / the NGO Family Centre in Kotor), and a national SOS recommends programmes that strengthen parenting 24/7 hotline for victims of family violence. This strategy skills and encourage parents to engage in more activities Task Force also draws attention to the necessity of a higher level of with their children (e.g. playing, reading, family meals and Review of law, Review of participation by both local self-governments and NGOs outings). Figure 12. Proposed national policies, sectoral continuum-of-care in strengthening the provision and quality of community- committee and strategic guidelines & service gaps based social and child protection services. Target 1 Finally, the 2019 Guidelines for Health Service planning actions (improvement of the normative framework in social and Providers Acting with a View to Protecting Children National Parenting child protection), Target 2 (improvement of the system and Adolescents from Violence, Abuse and Neglect Strategy (optional) of quality in social and child protection) and Target 3 sets forth the goals of promoting non-violence against Community (improvement of social and child protection services) all children through: changing attitudes, values and Research with consultations & provide a foundation for the implementation of parenting behaviours; improving the preventive and systemic practitioners & research with service providers interventions. work of service providers with children and families; parents and improving professional capacities in the fields of National Action Plan & Guidelines The 2021–2024 draft Strategy for Early and identification, recording, referring, reporting and treating for Parenting for Lifelong Health Preschool Education puts forward a specific objective children subjected to violence. The guidelines indicate for Young Children of strengthening support to children and families that the promotion of non-violence will be implemented Macro level consideration (Operational Objective 2.6), with Target 2.6.1 stating the through specific programmes initiated, designed, role of the Ministry of Education in developing, piloting monitored and evaluated by the Ministry of Health and the and adopting a programme for supporting the parents of Institute of Public Health; for this purpose, the Ministry Setting of Mechanisms for Capacity building Activities for National Registry / Funding preschool-age children. In addition, Operational Objective will amend the programmes of work of the centres for objectives, intra-/inter-sector plan: Facilitators, programme System for routine commitments 3.3 delineates the commitment to improve support prevention in Healthcare Centres. The guidelines further targets & and municipality superviros & promotion with data collection & criteria for to parents from vulnerable groups by providing more highlights several universal interventions for prevention indicators coordination trainers general public reimbursement stimulating conditions for early development in the family that are relevant to this effort, including universal visits environment. Target 3.3.2 specifies that an integrated by home visiting nurses, the organization of parenting inter-sector programme will be developed to provide such schools, and advising and training parents on child targeted support, while Targets 3.3.3 and 3.3.4 indicate maltreatment, including the damaging effects of physical Guidance for programme delivery & data collection that guidelines to support parents from vulnerable groups punishment and the importance of positive disciplinary will be piloted, finalized and adopted. methods. Preparation: Programme During follow-up: Local promotion, delivery: Post-assessments, In the health sector, the 2015–2020 Master Plan for recruitment, Standards, identification & preassessments forms & comunication of the Development of the Health System indicates that checklists further needs two primary objectives for the prevention and control of

40 Scaling up Parenting for Lifelong Health in Montenegro: A feasibility assessment Scaling up Parenting for Lifelong Health in Montenegro: A feasibility assessment 41 obstruct progress, and issuing practical guidance for PLH- specify sector-specific policy actions and priority areas YC service providers across sectors, dividing into smaller, for further development and investment. Various existing Photo: focused working groups when necessary (see Figure 12). services could be incorporated, including nurse home- Duško Miljanić / UNICEF Montenegro This should be a ‘driving team’, consisting of a group of visiting services, Family Outreach Workers and different highly committed individuals, such as influential policy counselling centres operating within Healthcare Centres, champions, subject-matter experts, key service provider as well as potential interventions to be piloted (e.g. PLH directors and managers, several active PLH-YC facilitators/ Teens and PLH ParentChat). One assessment respondent supervisors, NGO and faith-based representatives, noted the need for the creation of such a strategy by a researchers, and monitoring and evaluation experts. The central committee, stating: “This very commission … membership should represent a variety of skills, such maybe could have a bigger mandate, a mandate which as abilities to forge effective coalitions, competencies could cover more than this programme only.” It is within in technical areas, and expertise in management and the context of this wider parenting strategy that a specific training [77]. Research shows that successfully scaled- National Action Plan and Guidelines for PLH-YC could be up projects are often led by persistent, well-connected issued. Examples of countries that have issued similar, and convincing leaders with clarity of vision and the broad national parenting strategies include Ireland, Malta ability to mobilize resources [79, 80]. The need for and Scotland [82-84]. such a collaborative team was suggested by several interview and FGD respondents in this assessment. One respondent emphasized: “The only way to scale National Action Plan and up a programme like this is to have some kind of inter- Guidelines for PLH-YC ministry, inter-sector entity. I believe that the ministries themselves should be leading this process.” Further, one A National Action Plan and Guidelines for PLH-YC would ministry official noted: “I believe it would be only normal aim to specify relevant supportive policies and financial and possible to create a multi-sector body, an entity resources, delineate a time-bound approach for building which would be comprised of representatives from the in-country capacity for programme roll-out, as well as Ministries of Health, Education, and Social Welfare … [it] cover details regarding: PLH-YC delivery; expectations for could then be in charge of coordinating activities related supervision; accreditation and certification; managerial to data collection, quality assurance, etc.” support by service providers; programme administration; and monitoring and evaluation. The plan should be fully duties; to specify certain things that could lead to a very include the identification of obstacles to cross-sector The creation of such a committee or task force inevitably costed and should clarify the country-wide approach specific monitoring and evaluation.” collaboration, which were frequently mentioned by raises questions of where its ‘institutional home’ should for PLH-YC implementation to policymakers and other interviewees. These barriers include multi-sector training reside. For interventions related to early childhood national-level stakeholders, while the Guidelines should Inputs. As portrayed in Figure 12, the National Action and accreditation, central budget funding to cover development in particular, which by nature involve a be intended for use by service provider managers, Plan and Guidelines may require a range of inputs in additional working hours by facilitators and supervisors, variety of stakeholders, it is often difficult to find the administrators who support delivery, as well as PLH-YC order to effectively respond to gaps and impediments and electronic data collection and data sharing. As ‘right’ home, as all relevant institutions will present facilitators, supervisors and the trainers themselves. to implementation, as well as to remain relevant to the noted by one expert: “The law itself stipulates that we advantages and disadvantages [81]. Indeed, each of the The National Action Plan and Guidelines would assist needs of particular geographical locales and marginalized provide training … [and] issue certifications to those who three ministries were proposed by various assessment in formally recognizing PLH-YC as an evidence-based and minority language populations (e.g. Roma and work in the social and child protection sector only. This respondents as being ideal for this institutional home programme that is endorsed at senior levels of the Egyptian communities). These inputs may require regular is a legal obstacle. We can issue a certificate to those and the accompanying leadership responsibilities this government for scaling up, help to maintain quality and feedback, using an intentional strategy of renewal and working in other sectors, but it would be of no value would entail. However, in terms of committee or task standardize the various stages of PLH-YC preparation, regeneration [78], such as through brief surveys or FGDs because their system does not recognize certificates force membership beyond the three relevant ministries, delivery and follow-up at the local level, and address with PLH-YC service providers, in-depth interviews with proving professional training in other fields.” One several respondents repeatedly pointed out that it would the specific challenges and obstacles identified in this parents/caregivers on their experiences with PLH-YC, ministry official stated in relation to wage compensation be important to ensure the inclusion of NGOs, as well as feasibility assessment report (section 3.D). In support and mechanisms for community-level consultations. through the central budget: “It’s very hard to expect the Institute for Social and Child Protection, the Institute of the National Action Plan and Guidelines, various As suggested by one assessment respondent: “It’s that. That simply is not an option because of the current for Public Health, and the Bureau of Education, given assessment respondents referred to the need for very important to have local focal points able to share legal arrangements.” Regarding the consolidation and their roles in capacity building, quality assurance, and official recognition of the programme, formalization of the information locally and to communicate with the analysis of data collected by service providers across monitoring and evaluation. cooperation between service providers, clarification of centralized body. And this work should be formalized.” sectors, another ministry official claimed: “…You have responsibilities by all actors, and formal documentation Another noted: “[There could be] some small bodies in to understand that the Institute [of Social and Child National Parenting Strategy and monitoring of programme activities. local communities with a constant flow of information Protection] is in charge of institutions working in the and communication between the local and central levels.” social welfare sector only. … The data comes from their Prior to the development of a specific action plan and The need to establish official plans and guidance was Such regular feedback from practitioners, in the form own respective sectors only.” guidelines on PLH-YC implementation, the committee referred to by several assessment interviewees, who also of reports and the sharing of anecdotal success stories, or task force could consider the creation of a wider mentioned the relevance of memoranda of understanding may also help to sustain positive attitudes toward the The second review, conducted in close cooperation National Parenting Strategy, which has scope beyond between sectors and the creation of a formal paper programme, as well as bolster programme quality and with case managers and other practitioners who work PLH-YC. This strategy, in line with the 2018 Decree trail to document programme processes. According to maintenance [85]. closely with vulnerable families, should examine how on Methodology and Procedure for Strategy Paper one ministry official: “There should be some kind of PLH-YC fits into existing service menus for different Development, Alignment and Monitoring, would set forth formal guideline or instruction related to how to refer Other inputs that appear necessary, based on feedback target populations and identify points of referral both the government’s vision and commitment to parenting families and parents to this programme.” Another expert from research participants, include two reviews: a into and (if needed) in follow-up to the programme. For support across sectors and child development stages, and respondent indicated: “…essentially it is very important to review of laws, policies and sector-level guidelines example, existing guidelines for case management, would aim to identify existing parenting support services, have some kind of formalized form of cooperation related that may impede PLH-YC scale-up, and a review of multi-disciplinary teams, and preventing and responding pinpoint gaps in provision, identify beneficiaries, and to certain activities with clearly listed obligations and potential continuum-of-care gaps. The first review should to violence will need to be reviewed, as well as practices

42 Scaling up Parenting for Lifelong Health in Montenegro: A feasibility assessment Scaling up Parenting for Lifelong Health in Montenegro: A feasibility assessment 43 for supporting parents of children with disabilities, can enrol in the programme, promote help-seeking director underscored the necessity of central budget acknowledged that local self-government units and the parents of children with neurodevelopmental and conduct behaviour for parenting challenges, as well as more funding for the programme. She stated: “We need to be service providers themselves can readily contribute to disorders, and remote and marginalized populations broadly raise awareness on the importance of positive sure that the Ministry of Education supports this project, other recurring costs, such as for: programme materials, such as those in rural areas or living in Roma or Egyptian parenting and non-violent discipline. An example of such not declaratively, not in a tokenistic approach, but by refreshments, any relevant facilitator per diem and communities. The risk that a parent/caregiver could an initiative was undertaken in the Netherlands through paying those compensations to facilitators. It is necessary transport costs, as well as mobile phone and internet still be in need of further, more intensive support after the ‘Stay Positive’ campaign for the Triple P parenting to have (their) full understanding and support … This is expenses. One ministry official suggested: “Local self- completing PLH-YC, yet fall through the gaps by failing programme, which aimed to increase awareness of the a must. By paying these compensations to facilitators, governments have their own locally generated funds and to receive an appropriate referral, was highlighted by one intervention in Amsterdam, as well as promote positive they make sure that we sustain this programme.” In budgets and essentially they may use these funds to fund education sector facilitator: “Certain families are indeed parenting messages through billboards, bus shelter addition, although it may not be possible to compensate the services they perceive as necessary or beneficial for troubled with domestic violence … It’s more than obvious displays, TV commercials, a parenting newspaper, facilitators for their time at a standard rate across sectors, the local communities.” Several kindergarten directors … But we do not communicate on these issues with shopping bags, and a campaign website with parenting some level of parity should be pursued. The importance noted that funding for materials and refreshments was Centres for Social Work. If we … cannot help the child tips and practitioner contact information. Over a 10-month of ensuring all facilitators are compensated for additional possible through locally raised resources, while one in our kindergarten, if we refer the child to a particular period, 72% of parents in Amsterdam had heard about workloads or for working overtime was emphasized Healthcare Centre director shared that facilitators based physician, it means that the child will end up going to a the programme [86]. The importance of public promotion by one expert, who noted: “If you have one preschool at their centre utilized local funds through the issuance professional who is already burdened with quite a lot of of PLH-YC was also underscored by a NGO-sector paying their psychologists to do the programme, and of medical certificates that are self-paid by citizens. work.” Addressing these gaps and ensuring that there are facilitator interviewed for this assessment, indicating: “It another one is not doing that, but you know, is finding However, NGOs with limited infrastructure and that clear referral pathways and case management procedures would be important for the society at large to recognize some other ways – I mean, people know each other and work with particularly vulnerable populations may need that support PLH-YC implementation would be important the importance of this programme – maybe some media get demotivated.” Directors and facilitators in the NGO central funding assistance, given venue requirements outcomes for this review. coverage would be helpful to promote this further and sector, who unlike other sectors were not compensated and the additional costs needed for recruiting (e.g. Roma increase coverage of parents.” for overtime or the additional workload, were especially mediators) and addressing obstacles to engagement Macro-level considerations. The National Action Plan concerned about the lack of financial support that they (e.g. childcare needs, transport). One NGO director should encompass a range of aspects. At the outset, An additional, critical area for consideration is the have received to date. One facilitator stated: “I think that shared that such additional funds were vital: “I know objectives, targets, and indicators will need to be development of a national register of participants or other the solution might be to just engage the ministry and that my [facilitator] colleagues from other municipalities established, which represent the overall approach to electronic data collection system for the gathering of both other governmental organizations more to understand have parents who completed university education, and scaling up, such as its scope (e.g. geographical expansion programme process and impact data. This register would how important it is to finance this type of programme you can’t compare the group of such parents with the and levels within service systems), pace (rapid or provide relevant ministries and the national committee/ … Although they promised that they would help us and group of parents who are illiterate. It’s hard to expect that gradual), the number of agencies involved, as well as the task force with the necessary information to evaluate support us, they didn’t do that in the end.” the financing of those two programmes would be the degree of centralization and decentralization. A qualitative and report on implementation progress and programme same…” study with implementers of early childhood development effects. The register should be regularly updated by The existing literature on financing scaling-up efforts projects in low- and middle-income countries found PLH-YC facilitators and service provider administrators highlights that such expansion efforts are not yet routine, In addition to clarification on funding commitments, that scaling up is often more effective when taken at a who are assigned unique identifiers and who can fill out and for this reason, dedicated resources in the medium the National Action Plan and Guidelines should include slow and steady pace, allowing implementers to better electronic forms on: when and where PLH-YC parenting term are necessary until programme delivery becomes information for service providers on how to budget for understand the context for expanding project delivery, groups were organized; participant names and basic standard practice and costs are allocated through regular programme delivery, what expenses are eligible for build relationships with new stakeholders, and make family demographic and contact information; enrolment, budgets [77]. Over the medium to long term, the scaling- reimbursement from central funds, as well as how to adjustments in the scale-up strategy, while ensuring that attendance, completion, and drop-out information; when up of interventions is generally financed through taxation, make expense claims. If vouchers for parents/caregivers programme fidelity and quality is upheld [81]. In addition, pre-consultations and catch-up consultations were held; social health insurance, public–private arrangements, or are necessary to assist some families in accessing mechanisms for intra- and inter-sector coordination, as logs of phone calls and whether Viber group chats were by international donors. Funding through taxation and childcare services or transport, sample voucher receipts well as cooperation with self-government units, will need created; pre- and post-programme assessment data; and insurance schemes is preferable in that such allocations and registries should also be included [87]. to be reviewed or created, in order to ensure consistent results from participant satisfaction surveys (see section can become institutionalized and sustainable over time; data sharing and the exchange of implementation 4.F for more details). Data protection standards for such however, external financing raises concerns regarding Guidance for programme delivery and data collection. experiences. According to one ministry interviewee: a registry should also be in place, in line with the EU both sustainability concerns and local ownership [68]. In addition to the National Action Plan, it is critical that “In my opinion, it would be very important to engage General Data Protection Regulations (GDPR). An example One Healthcare Centre director stated that utilization of all PLH-YC service providers are provided with clear and local self-government units in the discussion around this of this type of national register is described in ‘Guidelines the National Health Insurance Fund was an appropriate standardized guidance for programme delivery, which programme in terms of supporting it, in terms of helping for the Incredible Years Parent Programme,’ issued by funding pathway: “… We need to be proactive in terms of will help to uphold programme fidelity during roll-out. to deliver it and expand it.” the Ministry of Education in New Zealand. This registry is finding opportunities to sustain this programme. (PLH- Maintaining fidelity is essential, as local modification linked to the Ministry’s case management system, which YC) could be financed by the National Health Insurance of the programme’s ‘essential ingredients’ – those A fundamental macro-level consideration is the allows it to be integrated into an existing e-database and Fund. In that way, this programme could become part components that support the intervention’s internal development of a country-wide capacity-building plan to link with other relevant services for parents/caregivers of the regular activities undertaken by Primary Health logic and associated effectiveness – may result in a less (refer to section 4.C below for details), which identifies: and children [87]. Care Centres.” Another expert echoed this sentiment effective or even ineffective programme [88]. In fact, suitable service providers for programme delivery; by suggesting that a proactive approach to long-term ‘intervention drift’ is a common problem often faced by appropriate professions and qualifications for facilitators, Finally, a major aspect to be addressed in the National funding by this fund should be undertaken: “I believe implementers during scale-up, and which can arise when supervisors and trainers; which organizations(s) will Action Plan is the financing of PLH-YC institutionalization that those persons who are communicating with UNICEF there is insufficient provider training and supervision, be responsible for conducting staff training and the and expansion. As noted in earlier sections 2.C and and who should know about this programme should poor integration into the usual care setting, and lack of frequency of such training; and how newly trained staff 2.E, insecure and insufficient funding for programme engage in advocating for the allocation of such funds guidance on what types and degrees of local adaptations will be sufficiently supervised and supported to deliver delivery – especially for facilitator costs – is the most before the National Health Insurance Fund. In that way, are permissible [89]. While the standardized PLH-YC PLH-YC with fidelity and quality. significant obstacle to scale-up. Many assessment the National Health Insurance Fund would be familiar with Facilitator Manual and Parent Handbook will assist in respondents expressed concern about the future of the importance of these programmes and they would be promoting uniform delivery, a centralized training and Another important consideration is the development of PLH-YC in Montenegro, given the constraints that they influenced to allocate funds for it.” technical assistance process will also be needed for country-wide activities for programme promotion. These faced in mobilizing local resources within their respective facilitators and administrators across local delivery activities are needed in order to: raise the public visibility institutions; this appeared to be particularly pertinent While central budget funding for facilitator wages systems [90, 91]. of PLH-YC, share information on how parents/caregivers to the education and NGO sectors. One kindergarten was recommended by several interviewees, it was

44 Scaling up Parenting for Lifelong Health in Montenegro: A feasibility assessment Scaling up Parenting for Lifelong Health in Montenegro: A feasibility assessment 45 In addition to information on the actual delivery of sustained supervision, clear communications with service consideration should be given to the recruitment of staff Supervision programme content, and in conformity with the National provider managers, provision of professional development who will deliver the programme to particular parent Registry or other data collection system that has been opportunities through accreditation, and regular contact groups in clinical settings, such as the parents/caregivers The provision of skilled and regular supervision is created, data collection instructions and form templates and feedback can all assist in keeping the workforce of children with Attention Deficit Hyperactivity Disorder, repeatedly highlighted in the literature as critical to should also be included in the guidance. Promoting engaged. In Wales, the maintenance of a register of autism-spectrum problems, or severe conduct disorders. the scaling-up of parenting interventions and retaining consistency across data collection methods and forms all professionals who have been trained to deliver the Such facilitators will need additional background skills and effective outcomes [95, 96]. As an increasing number of will facilitate the analysis of data across sites, as well Incredible Years programme allows the training centre qualifications in order to anticipate and better support the PLH-YC facilitators are being trained across municipalities as enable national-level monitoring and reporting of the at Bangor University to remain in contact with them, likely challenges that such families are facing. and service providers, ensuring that adequate and skilled implementation progress. updating them on training opportunities and research staff supervision is available on an ongoing basis will findings, as well as inviting them to annual conferences. Many assessment respondents shared that there are require careful strategic planning. Some or all supervision This should be organized in a practical and easy-to-follow In addition, annual workshops with managers were other professionals within their respective organizations sessions for newly trained facilitators and supervisors format, such as with sections on programme preparation, organized to help them to learn about local research who would be well-qualified to serve as facilitators and may need to be conducted online, to save on time and programme delivery and programme follow-up. Parenting findings and the requirements for effective programme supervisors. One Healthcare Centre director expressed costs, while service providers should ensure that venues for Lifelong Health has developed two manuals on delivery, while assisting them to set goals linked to their that a number of staff across their counselling centres – for parenting groups are equipped with the necessary implementation and monitoring and evaluation which can own organizational service plans [94]. which include those that focus on reproductive health, video cameras and equipment, so that facilitators can be freely adapted for the Montenegrin context [92, 93]; mental health and children with disabilities – would be bring video recordings of their delivery to supervision moreover, the New Zealand Incredible Years guidelines Staff recruitment well-placed to deliver PLH-YC. He noted that a specialized sessions. In Wales, where sustained supervision for provide a good example of how such country-specific centre focused on parenting support could even be the Incredible Years was cited as a key challenge, 22 Expanding PLH-YC delivery to more parents/caregivers guidance might be presented [87]. Table 7 contains an created: “My suggestion is to make this programme a local authorities were encouraged to establish in-house via the existing service providers, as well as increasing overview of what content should be included. Additional mandatory part of the work, an integrated part of the supervision, so that supervisors were always available to coverage to include broader geographical areas through information on capacity building, reach, adaptation, work of counselling centres in Primary Healthcare Centres recently trained staff [94]. research, and monitoring and evaluation is provided new providers, will entail a concerted, national effort … The core of those working in these Primary Healthcare below, and should be incorporated into these guidelines. to identify and train new staff. These efforts should Centres who should be facilitators should not be part- concentrate on increasing the numbers of facilitators time. They should commit all their working hours to this Training coordination and supervisors, while only a few select trainers are C) Staff recruitment and capacity programme. We aren’t talking about a big number, but six needed who are able to regularly conduct training for both of them could serve in this potential counselling centre Training of facilitators and supervisors across the health, building new facilitators and new supervisors. Staff should be for lifelong parenting.” In the education sector, one expert education and social welfare/NGO sectors will require a recruited who possess not only the appropriate general noted the large nursing workforce in kindergartens could long-term strategy that ensures a standardized level of Recruiting, capacity building and retaining skilled and qualifications (see Table 8), but who also encompass be recruited: “To see who we can involve … you have quality. Such training can either be centralized and multi- motivated staff are fundamental to the scaling-up a mix of genders, ethnicities, abilities and language around 500 nurses working in kindergartens. So, they are sector, or provided within each sector; however, the latter efforts. Mirroring findings on enabling factors from this competencies, in order to better reflect the diverse target maybe more willing or have more time to also be trained may be a relatively inefficient use of human and financial assessment, research with implementers who have communities where the programme will be implemented. and to deliver the programme.” Finally, there may be resources, it may present greater risks in terms of scaled up early childhood development projects has This is especially relevant to the encouragement of male opportunities to increase staffing within the current NGO fluctuating training quality, and could be more challenging shown that staff must be personally motivated and parent/caregiver engagement, as well as the participation service providers. The NGO Parents Association has 460 to coordinate and monitor. dedicated to the intervention [81]. Robust training, of Roma and Egyptian families. In addition, careful members countrywide, 40 of whom are active; moreover, the NGO Family Centre has a total of nine family outreach Assessment respondents frequently mentioned the Table 7. Proposed key content for national PLH-YC guidelines workers operating in eight municipalities. potential difficulties in cross-sector capacity building. Programme preparation Programme delivery Programme follow-up Table 8. Staff requirements for the 12-session version of PLH-YC

Community-based promotion Pre-programme consultations Administering post-programme with participants assessments Role General Training required Supervision required Assessment for qualifications certification Parent/caregiver recruitment Catch-up consultations Conducting satisfaction surveys Facilitator Preferred: Prior Five-day Facilitator • For 1st-time facilita- Assessment after Managing participant referrals from Weekly phone calls and group Referrals for families in need of knowledge of and Training tors: eight sessions delivery of two other providers chats additional support experience related • For 2nd-time facilita- PLH-YC parent to early childhood tors: three sessions group cycles Completing participant service Managing participant case Claiming for reimbursement of development & child • For 3rd-time facilita- agreements (if required) profile data expenses protection; experience tors: two sessions facilitating youth or Administering pre-programme Recording process data family groups assessments Supervisor Certified PLH-YC Two-day Supervisor Supervision of their su- Assessment of a Addressing obstacles to engagement Issuing certificates to Facilitator Training pervision practice over full supervision ses- participants four sessions sion of facilitators

Budget planning and management Making referrals to child Trainer Certified PLH-YC Involvement in Supervision during Assessment of in- protection & welfare services Supervisor co-training five-day initial independent deliv- dependent delivery Facilitator Training ery of five-day Facilita- of five-day Facilita- Preparing the venue and materials Video recording sessions & and/or two-day Su- tor Training tor Training attending supervision pervisor Training

46 Scaling up Parenting for Lifelong Health in Montenegro: A feasibility assessment Scaling up Parenting for Lifelong Health in Montenegro: A feasibility assessment 47 One expert stated: “Our sectors are siloed, fragmented events.” However, given the present lack of training and 11–21 supervisors in total may need to be identified and other partners could help.” In addition to their in such a way that they are not networked and linked accreditation within the health sector, this initiative should and trained, with a time commitment per facilitator of institutional social media presence, some respondents sufficiently, so that we could, for example, deliver be explored and pursued within the National Action Plan. 24–48 weeks per year. Delivery of the eight-session noted that disseminating fliers and brochures at their training for those outside the social welfare sector.” As stated by one health sector facilitator: “We need PLH-YC version would considerably reduce the amount Healthcare Centres, as well as promoting PLH-YC However, such training does exist. This expert continued: accreditation by the Ministry of Health. It’s good for us to of work per facilitator, with a time commitment of 16–32 during parent–teacher meetings at kindergartens, would “… We do have some training courses which are have that accreditation.” weeks per year. be worthwhile endeavours. The language used when intended for multidisciplinary teams, in case there is promoting the programme should remain positive and a need to integrate certain services.” The same expert Cascade training Examples of potential cascade training models for focus on what PLH-YC can offer to parents, rather than recommended that one institution should be the sole trainers, supervisors and facilitators are provided in shaming parents or suggesting that only ‘problem’ training provider, and that arrangements should be made In scale-up contexts, the organization of cascade training Appendix D. In addition to this initial training and the families should join. One health facilitator suggested: to foster inter-sector recognition of training completion: can allow for capacity building to occur more quickly, corresponding certifications for staff, refresher training “When it comes to marketing this programme, I believe “… One institution could offer [training] to all who need it using increasingly expanded resources [81]. If necessary, may need to be organized for trainers, supervisors, and/ the keywords should be on improving parental capacities and deliver it in various respective sectors … Some kind such as during the COVID-19 pandemic, training can or facilitators in order to promote programme fidelity and or competencies. In that way, parents would recognize of effort between the ministries could be a way to lead to be conducted fully online or through a mix of online quality. These refresher workshops can be held when a this as a potential programme that could improve their some kind of automatic recognition of training received and in-person sessions, with sessions restructured significant amount of time has lapsed between receipt own behaviour and strategies.” by other sectors … Representatives from various to accommodate online role play and small-group of trainings and the provision of training, supervision or ministries could monitor and evaluate those programmes exercises. As suggested by Prof. Judy Hutchings, a programme delivery, or if process data collected through Parent/caregiver recruitment and themselves so that they could rest assured about its PLH-YC developer, one approach to initiating a training regular monitoring and evaluation (see section 4.F) follow-up quality.” plan is to determine the total number of parents/ indicates that the levels of programme fidelity and quality caregivers of children aged 2–9 years to be targeted over are poor or insufficient. Given the feedback from the current cohort of PLH- The institutions most frequently mentioned by a particular period, and then calculate the number that YC facilitator and supervisors that parent groups with respondents as potential central sites for PLH-training could participate in the programme per year. Working mixed demographic profiles worked well, the National included the Institute for Social and Child Protection backwards, the number of parenting groups to be held D) Programme reach Action Plan and Guidelines should clearly indicate how and the Bureau of Education, given that both have per year, organized by facilitators and supported by both universal and targeted recruitment approaches already accredited the facilitator training programme supervisors, could then be determined. For example, Local programme promotion should be balanced by service providers. One option is and the Institute has accredited the supervisor training using estimates from UNICEF documented in the 2019 for service providers to reserve a proportion of places in programme. One ministry official noted: “I believe that conference paper ‘Investments in Early Childhood In synergy with national-level programme promotion an upcoming parenting group for families that are facing the Institute for Social and Child Protection could be the Development: The Case of Montenegro’, 2,000 parents/ strategies, community-based promotion of the particular challenges, such as managing difficult child central institution, primarily because they are in charge caregivers may need to be trained annually [97]. As programme should be pursued. Communication behaviour, parenting children with special needs, or who of providing training, in charge of accrediting certain presented in Table 9, based on a size of about 12 parents/ strategies supported by the various service providers have been referred to the provider by a Centre for Social programmes.” One expert also shared: “I believe that the caregivers per parenting group, 167 groups would need to should aim to stimulate local demand for PLH-YC and Work. The remaining places can be open to those who Bureau of Education should introduce this programme be organized per year. Based on respondent feedback that galvanize community support for scale-up [81]. Several have learned about the programme by word of mouth, or as a mandatory one, that this programme should be on the current 12-session programme may be delivered by evidence-based parenting programmes encourage local who have self-referred after being exposed to promotional the list of mandatory programmes … I believe that there facilitator pairs over 2–4 cycles per year, and if facilitator promotional initiatives. The Triple P Positive Parenting activities or materials. Another option is to concentrate shouldn’t be any obstacle or challenge to have people pairs and supervisors each delivered the programme a Programme in Australia is often promoted in schools and the expansion and coverage of PLH-YC in particular areas, from social welfare and health participating in these consistent number of times per year, 84–168 facilitators community healthcare centres, given that both settings to ensure that there are plenty of places available in are regularly in contact with families and are generally parent groups to accommodate both vulnerable families Table 9. PLH-YC staffing required to deliver the programme to 2,000 parents/caregivers per year perceived as free of stigma and judgment [98]. The and those who are at lower risk of using violent discipline. Strengthening Families Programme, which has been The strategy undertaken by government officials in New PLH-YC version Two cycles of delivery Three cycles of delivery Four cycles of delivery implemented in over 35 countries, has an implementation Zealand for the Incredible Years programme is to target per facilitator pair and per facilitator pair and per facilitator pair and manual containing examples of fliers and church geographical areas with a high population of ethnic supervisor per year supervisor per year supervisor per year bulletins for local distribution, as well as scripts that minority families, a high concentration of families in low professionals can use to talk about the programme during socio-economic groups, and a high number of referrals 12-session ver- • About 84 parent groups • About 56 parent groups • About 42 parent groups per parent–teacher meetings [99]. These initiatives may be for behavioural services [87]. Given regional disparities in sion per cycle per cycle cycle particularly important for and should be tailored to ethnic Montenegro, such as higher poverty and unemployment • Requires a total of 168 • Requires a total of 112 • Requires a total of 84 minorities, immigrants, refugees and those from culturally rates in the northern municipalities, as well as evidence facilitators facilitators facilitators diverse backgrounds, with research showing that such of inter-generational poverty and lower school enrolment • Requires a total of 21 • Requires a total of 14 • Requires a total of 11 parents/caregivers tend to be less likely than Caucasian rates amongst Roma and Egyptian populations, the supervisors (for 1st-time supervisors (for 1st-time supervisors (for 1st-time families to access parenting programmes and mental National Action Plan may need to incorporate different facilitators) facilitators) facilitators) health services [86]. regional and municipal targeting strategies [100, 101]. • Time commitment per • Time commitment per • Time commitment per staff Finally, increasing the recruitment and retention of male staff member: 24 weeks staff member: 36 weeks member: 48 weeks per year Many assessment respondents have the view that there parents/caregivers may also require a targeted approach. per year per year has been little national and local promotion of PLH-YC, Qualitative studies on father engagement in parenting and that increasing such publicity – both with the general programmes have identified that men may require more Eight-session • Same number of parent • Same number of parent • Same number of parent public as well as with other institutions – is essential. intensive community outreach and ‘non-traditional’ version groups, facilitators and groups, facilitators and groups, facilitators and One health expert shared: “The programme has not recruitment and retention methods, such as peer-to-peer supervisors as above supervisors as above supervisors as above been promoted sufficiently. The majority of us did not recruiting and the use of tailored promotional materials • Time commitment per • Time commitment per • Time commitment per staff hear about its delivery in Montenegro.” A ministry official emphasizing the importance of fatherhood [102, 103]. staff member: 16 weeks staff member: 24 weeks member: 32 weeks per year also remarked: “A lot of people are not familiar with this In order to address potential gaps identified by the per year per year programme in their ranks. So, I believe that a bit more aforementioned review of continuum-of-care systems, advocation and a bit more support on the part of UNICEF as well as to ensure inclusion of particularly vulnerable

48 Scaling up Parenting for Lifelong Health in Montenegro: A feasibility assessment Scaling up Parenting for Lifelong Health in Montenegro: A feasibility assessment 49 families, the guidelines may need to delineate cross- better meet special parental needs [81]. This may be Table 10. Overview of PLH-YC monitoring and evaluation data and tools sector procedures for referring families into and out of especially relevant to Roma and Egyptian families, as the programme. Programme promotion activities should well as parents/caregivers of children with disabilities, Data to be Description Data collection tool(s) Time points for collection target staff at all Centres for Social Work, Healthcare neurodevelopmental disorders or conduct disorders. collected Centres, kindergartens, primary schools, and nurseries In the view of one NGO director interviewed for this so that they are fully aware of PLH-YC and are able assessment, the current structure of PLH-YC should Process to identify and refer eligible families to local service be modified to suit the perceived needs of Roma and providers. In return, facilitators should ensure that Egyptian families: “These are parents with very little time Reach Programme Persons who have signed-up Sign-up register Pre-programme the person or organization that referred the parent/ to spend outside their homes … [and] 90% of parents recruitment to participate caregiver remains involved with the family both during we worked with were completely illiterate. And we and after the programme. In addition, the Guidelines notice that, as soon as 60 minutes is up, it proves to be Delivery of Participants receiving Participant case Pre-programme should include information for facilitators regarding how really hard for them to keep up … They start fidgeting, pre-programme pre-programme consultations profile or consul- to determine whether families require further support they start looking at their watch. So, having shorter consultation prior to the first session tation register after programme completion, and how to make such a sessions, such as sessions lasting 45–60 minutes, would referral. For example, such a determination may be based be more beneficial…” Other evidence-based parenting Programme Participants attending one or Attendance Weekly during on whether a child has met the clinical cut-off score on interventions, such as Triple P and the Incredible Years, enrolment more sessions register programme the Eyberg Child Behaviour Inventory (ECBI) during the offer adapted versions of their core programmes for post-programme assessment [104]. Several respondents the parents/caregivers of children with disabilities [105, Session attendance Log of participant weekly Attendance Weekly during shared that such initiatives to clarify and expedite 106]. These types of adaptations for Montenegro should session attendance register programme referrals to and from other service providers would be involve national supervisory bodies and the PLH-YC useful, especially with local Centres for Social Work. programme developers, who can provide guidance on Session completion Participants who have Attendance Weekly during According to one NGO director: “I think this kind of formal how to modify the programme without diluting the core completed at least seven out register programme communication and recruitment (to PLH-YC) that would components [107, 108]. of 12 sessions come from other institutions would be beneficial. Schools and Centres for Social Work, for example, have the option Programme Participants who have not Attendance Weekly during to refer the family to the services of, for example, this or In addition, the COVID-19 pandemic, budgetary drop-out attended three or more register programme other NGOs. Simply said, they can refer in, and I believe constraints in low- and middle-income countries, and sessions in a row with no this kind of networking between education, health and the imperative of increasing the population reach of catch-up consultations social welfare is essential for having them on-board…” PLH-YC have led to the development of innovative delivery approaches. One of these is ParentChat, an Delivery of catch-up Participants receiving Participant case Weekly during consultations catch-up consultations profile or consul- programme E) Programme adaptation and online parenting support group programme intended for universal populations, which is delivered through online tation register innovative approaches text messaging platforms, such as WhatsApp or Viber. Facilitators who have been trained on the in-person Individual phone Participants receiving weekly Phone call Weekly during It is crucial for the National Action Plan and Guidelines version of PLH-YC can utilize the ParentChat application calls phone calls register programme for PLH-YC scale-up to incorporate clear programme to facilitate parenting groups over eight short, online delivery guidance and monitoring and evaluation tools to interactive group sessions (see Figure 13). Programme Online group chat Facilitation of regular online Group chat Weekly during promote fidelity and prevent potential programme ‘drift.’ content focuses on spending quality time with your group chats register programme However, service providers may want some flexibility to child, using positive reinforcement, COVID-19 health slightly tailor the programme to particular sub-groups, and safety, and coping with difficult child behaviour, Fidelity and Session fidelity Number of core activities Fidelity checklist Weekly during in order to address local obstacles to engagement, parental stress and family conflict. Facilitators moderate quality conducted by facilitators programme bolster cultural relevance to programme content, or to discussions around these parenting themes, as well during each session

Figure 13. Overview of ParentChat Facilitator Record of highlights and Facilitator Prior to each self-reflection challenges of participants and self-reflection supervision session reports facilitators report

Online parent support groups Supervision reports Facilitator attendance and Supervisor report Following each overview of highlights and supervision session based on PLH (ages 2-17) challenges of participants, facilitators, and supervisors 8 weekly modules; 2 chat sessions per module (8-15 parents) Facilitator Level of competency of PLH-Facilitator Following each certification reports facilitators regarding quality of Assessment Tool facilitator certifica- Weekly assignments and feedback programme delivery tion assessment sessions Supervisor Level of competency of PLH-Coach Following each certification reports supervisors regarding Assessment Tool supervisor Emerging evidence of effectiveness quality of supervision certification assessment

50 Scaling up Parenting for Lifelong Health in Montenegro: A feasibility assessment Scaling up Parenting for Lifelong Health in Montenegro: A feasibility assessment 51 Finally, a National Action Plan for PLH-YC scale-up should satisfaction questionnaire data, in-depth research on Impact consider incorporating a research agenda. There are parent/caregiver experiences of the programme has not several potential studies that would be worthwhile to yet been conducted. Qualitative studies can provide Outcomes Pre-programme Participant questionnaire PLH Short-, Medi- Pre-programme pursue. First, a well-conducted randomized controlled trial important insights into how beneficiaries perceive that assessment using validated instruments um- or Long-Form of the programme during or after scale-up would provide their relationships with their children have changed, Questionnaire more reliable data on whether the programme is indeed as well as how reductions in violent discipline may or effective, given that the pre-post evaluation following the may not have occurred. Furthermore, interviews with Post-programme Participant questionnaire PLH Short-, Post-programme first cycle could not provide causal evidence. It would also dropped-out or low-attendance parents/caregivers can assessment using validated instruments Medium-, indicate whether a scaled-up model of implementation, reveal important obstacles to engagement that need to or Long-Form with national supervisory body involvement and without be addressed. To date, such qualitative studies of PLH Questionnaire regular, direct support by programme developers, is programmes have been undertaken in both South Africa both feasible and effective in real-world settings [90]. and Thailand [113, 114]. Fourth, adapted versions of PLH- Acceptability Participant Participant acceptability of Participant Post-programme Second, testing of the eight-session version of PLH-YC YC that have been significantly modified to suit particular satisfaction content and extent to which satisfaction is worth exploring, given the potential savings in terms populations (e.g. Roma and Egyptian families) should be expectations and goals were questionnaire of both expenses and demands on staff time. This may empirically evaluated. Such studies would help to gauge met come in the form of a feasibility pilot, or a three-armed whether programme effectiveness and acceptability randomized controlled trial, allowing for comparisons is maintained, improved or reduced. Finally, a cost- Case record Participant case Case notes on participant Participant case Weekly during between the effects of the 12-session version, the eight- effectiveness analysis of PLH-YC in Montenegro would profile goals and progress profile form programme session version and a control group [112]. Depending on assist in assessing the programme’s value for money by these results, as well as the target populations prioritized estimating the ratio of costs per unit of health outcome. by the national committee/task force, both versions of This would not only help to drive policy advocacy with as provide individualized support to parents. The outcome data during the six cycles of PLH-YC was the programme could be offered, with the eight-session key government stakeholders, but also assist the national programme also incorporates text/audio messages mainly maintained on local computer systems and on version delivered as a universal programme and the committee or task force in long-term budgeting and and feedback, illustrated stories, videos and activity paper, with data shared with UNICEF and independent 12-session version reserved for families most at risk further planning for scale-up [115, 116]. assignments. Montenegro has been included in a recent researchers following the first cycle for the purpose of of child maltreatment. Third, aside from participant seven-country pilot study of ParentChat managed by external evaluation. Health experts and facilitators noted the University of Oxford, which will test programme that Healthcare Centres use a centralized e-database feasibility and the initial effects on the primary outcomes for recording patient medical data; however, this is of child maltreatment, positive parenting and parenting not accessible by counselling centres. As one health stress [109]. Depending on whether ParentChat shows facilitator shared: “We have a huge responsibility and promising results in the Montenegrin context, the a lack of formal documents [in the e-database], which I National Action Plan and Guidelines could incorporate this really hope will be changed.” A health expert suggested development in the future, or a separate Action Plan and both integrating PLH-YC data into the health e-database Guidelines should be created. and introducing software to integrate cross-sector data: “So when it comes to the database, I believe it would F) Programme monitoring, evaluation be necessary for UNICEF to advocate for introducing this particular information and data in the existing electronic and further research database system … It would be good to have data and information on the programme in the respective sector The regular collection and evaluation of data is databases, and the next step would be linking those essential to drive and monitor the scale-up of parenting databases which could reflect the multi-sector approach interventions. It is critical to both state and service to this problem.” provider accountability, and also provides practical benefits in that it enables course correction if the As summarized in Table 10, there is a range of process implementation is not adhering to the strategic plans [81]. and impact data to be collected before, during and after In particular, maintaining programme fidelity at a larger programme delivery. Data pertaining to ‘reach’ concerns scale and over time is a common challenge to country the extent of target population coverage and dosage, efforts to institutionalize and expand evidence-based including delivery of programme components and rates parenting interventions. Establishing a National Registry of parent group recruitment, enrolment, attendance, or other system for consistent monitoring of programme completion and drop-out. ‘Fidelity’, as noted previously, is delivery, as well as regular evaluations of implementation related to the extent to which the intervention is delivered progress and outcome effects by supervisory bodies, as planned, while ‘quality’ is associated with facilitator can assist in maintaining fidelity, measuring impact, and supervisor skills in delivery and parent involvement. as well as tracking national roll-out and expenditures. ‘Impact’ describes the effect of the programme on The importance of this effort cannot be overstated: one participants, including ‘outcomes’ (effectiveness on review of almost 500 quantitative studies of child and pre-determined measures, including child maltreatment, adolescent programmes found that well-implemented positive parenting and child behaviour), and ‘acceptability’ programmes produce average effects that are at least (participant perceptions of relevance) [60, 110, 111]. Photo: 2–3 times as great as those which faced serious Participant case records are maintained by facilitators to Duško Miljanić / implementation problems [88]. capture qualitative information on individual goals and UNICEF Montenegro Assessment respondents shared that the process and progress.

52 Scaling up Parenting for Lifelong Health in Montenegro: A feasibility assessment Scaling up Parenting for Lifelong Health in Montenegro: A feasibility assessment 53 5. Recommendations

Based on the analyses Domain 1: National strategic planning resulting from the desk and guidance review and primary 1. Creation of a National Committee or Task Force: A multi-sector and multi-disciplinary data collection, and in ‘driving team’ should be established, with responsibility for developing a wider National consultation with UNICEF, Parenting Strategy and a specific National the following provides Action Plan and Guidelines tailored to PLH- YC implementation and scale-up. This team an overview of the key would be instrumental to the mobilization of inputs to the plan, exploring the ‘macro-level’ recommendations from factors affecting progress, ensuring that service this feasibility assessment providers are familiar with and equipped to implement PLH-YC with fidelity and quality, for the institutionalization and establishing systems for oversight. Ideally, members of the National Committee or Task and scaling up of PLH-YC in Force would include ministry-level officials Montenegro. While these of departments that are committed to PLH- YC roll-out, as well as active PLH-YC service recommendations are provider managers, supervisors, and/or facilitators who are very familiar with programme interconnected and should implementation. be considered in their 2. Development of a National Action Plan on PLH-YC: In order to ensure a consistent, the National Registry or other data collection centres, nurses in kindergartens, and a range of entirety, they are grouped country-wide approach to PLH-YC scale-up, a system and claim for expenses. Guidance on NGO network members and outreach workers. National Action Plan should be developed that working with marginalized, minority-language 2. Coordination of training across sectors: into five domains. establishes the objectives, targets and indicators and particularly vulnerable populations should be Efforts to centralize the multi-sector training of for institutionalization and expansion, and also incorporated. facilitators, supervisors and trainers should be delineates the scope, pace, mechanisms for 4. Setting of a National Parenting Strategy: explored, potentially through the Institute for intra- and inter-sector coordination, degree The development of a wider National Parenting Social and Child Protection and/or the Bureau of centralization or decentralization, activities Strategy, which sets forth the government’s of Education. The health, education and social for national PLH-YC promotion, and requisite vision and commitment to parenting support welfare sectors should aim to recognize such costs for implementation. The modalities for across sectors and child development stages, is cross-sector training certificates; moreover, the financing PLH-YC implementation through recommended but not essential to the scaling health sector should aim to accredit the PLH-YC central budgets, such as the National Health up of PLH-YC. This strategy would be valuable in facilitator and supervisor training. Insurance Fund, and strategies for mobilizing that it would map out the full range of parenting 3. Organization of cascade training: A plan for local resources should also be articulated. The support services, articulate how PLH-YC fits cascade training should be created, based on the coverage of PLH-YC staff fees for those working within these interventions and highlight priority total number of parents/caregivers of children overtime or delivering the programme on top of aged 2–9 years to be targeted, the number of areas for future development and investment. existing workloads requires particular attention. these parents that should be reached each year, 3. Issue National Guidelines for PLH-YC: Clear and the number of programme delivery cycles by and standardized guidance for programme Domain 2: Staff recruitment and each facilitator pair per year. delivery should be issued to uphold programme capacity building fidelity and quality during roll-out. Such guidance should encompass: programme preparation, Domain 3: Programme reach 1. Identification and training of new PLH-YC delivery and follow-up, with details on staff staff: In order to increase the service coverage 1. Encourage local programme promotion: recruitment, training and supervision; local by existing service providers, as well as to National guidelines should support local efforts programme promotion; parent/caregiver reach new geographical areas through new to promote the programme in order to stimulate Photo: recruitment and any necessary referrals into Duško Miljanić / providers, a concerted strategy should be local demand and galvanize community support PLH-YC or on to other service providers; and UNICEF Montenegro developed to identify and train new facilitators practical instruction on how to log progress in for the scaling up of PLH-YC. Such initiatives and supervisors. These efforts could include should aim to publicize PLH-YC with the general professionals in Healthcare Centre counselling public as well as with other institutions.

54 Scaling up Parenting for Lifelong Health in Montenegro: A feasibility assessment Scaling up Parenting for Lifelong Health in Montenegro: A feasibility assessment 55 2. Provide guidance on balancing both universal Pursue innovative approaches, such as Appendix A. List of PLH-YC and targeted recruitment of parents/ ParentChat: Depending on the findings from facilitators and supervisors caregivers: National guidelines should clarify the pilot study of ParentChat in Montenegro, Facilitator Programme delivery cycles *Dual role as facilitator and supervisor how service providers can recruit participants ParentChat should be incorporated into the generations with mixed demographic and risk profiles. This National Parenting Strategy, along with guidance may include reserving places in parent groups for service providers on how to discern whether for families who are facing particular challenges, e-parenting or in-person support would better Institution / Title / while permitting others at low risk of using serve different family profiles. Name M/F I II organization background violent discipline to occupy remaining seats. 1 (2018) 2 (2018) 3 (2019) 4 (2019) 5 (2020) 6 (2020) Concentrating PLH-YC expansion and coverage Domain 5: Programme monitoring, in targeted geographical areas should also be 1 NGO Miloš M psychologist x x x x x x considered. evaluation, and further research Pedagogical Bulatović* 3. Develop cross-sector procedures for referrals: 1. Establish a National Registry or other system Centre, Cross-sector procedures for referring families 2 Nataša F psychologist x x x x x for data collection: A National Registry or other Podgorica / both into and out of the programme should be Nikšić Durutović developed, reflecting and/or building on existing multi-sector system for data collection should be created to monitor programme delivery, relevant referral mechanisms. Such procedures 3 Marija F psychologist x x evaluate implementation progress and outcome would help Centres for Social Work, Healthcare Pašić Centres, kindergartens, primary schools, day- effects, and track expenditures during roll-out. Process and impact data should be collected care centres and other relevant institutions to 4 Jelena F preschool x x before, during and after programme delivery, identify and refer eligible families to PLH-YC Krivokapić teacher with data pertaining to reach, fidelity and quality, service providers, while also clarifying how these outcomes, acceptability and individual case service providers can refer families onward for 5 NGO Darija F psychother- x x x x x x records. Opportunities should be explored for additional support if needed. Association Petović apist integrating PLH-YC-related data into the existing Parents, Bambur* service provider databases, and consolidating Podgorica / Domain 4: Programme adaptation and this data across sectors. 6 Nikšić Marija F psychologist x x x x x x 2. Develop a national research agenda on PLH- Boljević innovative approaches YC: The National Action Plan for PLH-YC should incorporate several studies to identify service 7 Milica F psychologist x x x 1. Set guidance on parameters for local gaps and inform future directions for scaling Pušonjić adaptation: Guidelines should include up. Such studies may include a randomized explanations of how PLH-YC service providers controlled trial of PLH-YC during or after scale- 8 Snežana F social worker x x x can tailor the programme to particular subgroups up, the testing of the eight-session version of Milačić without diluting the core components. These PLH-YC for comparative purposes, qualitative target populations may include Roma and studies with parents/caregivers, the testing 9 NGO Child Ana F psychologist x x Egyptian families, as well as the parents/ of adapted programme versions for different Rights Centre, Jovanović caregivers of children with disabilities. populations, and a cost-effectiveness analysis. Podgorica 10 Ana Kulić F psychologist x x

11 NGO Centre Maja F psychologist x x for Roma Šaćiri Initiatives, 12 Nikšić Fana F pedagogue x x Delija

13 Slavko M pedagogue x x Milić

14 NGO Family Kristina F social worker x Centre, Kotor Vasiljević

15 Vesna F social worker x Antović

16 Ljubović Slavica F pedagogue x Centre for Merdović Children with Behaviour Photo: 17 Stanislava F pedagogue x Duško Miljanić / Problems, Radević UNICEF Montenegro Podgorica

56 Scaling up Parenting for Lifelong Health in Montenegro: A feasibility assessment Scaling up Parenting for Lifelong Health in Montenegro: A feasibility assessment 57 Facilitator Programme delivery cycles Facilitator Programme delivery cycles generations generations

Institution / Title / Institution / Title / Name M/F I II Name M/F I II organization background organization background 1 (2018) 2 (2018) 3 (2019) 4 (2019) 5 (2020) 6 (2020) 1 (2018) 2 (2018) 3 (2019) 4 (2019) 5 (2020) 6 (2020)

18 Đina Vrbica Dijana F psychologist x x x x x 36 Healthcare Ana F psychologist x x x x x x Kindergarten, Jović* Centre Berane Golubović Podgorica Popović* 19 Ljiljana F pedagogue x x Kovačević 37 Marijana F medical x x x x Bogavac nurse 20 Irena F preschool x x x x x x Badnjar* teacher 38 Tanja F medical x x x Bulatović nurse 21 Vesna F preschool x x x x x Marković teacher 39 Healthcare Tina F medical x x x x Centre Bijelo Novović nurse 22 Jasmina F preschool x x Polje Pejović teacher 40 Danijela F psychologist x x x x x x Vuković Femić*

23 Sandra F preschool x x x 41 Healthcare Branka F psychologist x Markuš teacher Centre Kotor Mitrić

24 Ljubica Popović Ilhana Ećo F psychologist x 42 Nada F medical x Kindergarten, Vukotić nurse 25 Podgorica Mirela F preschool x Šćepanović teacher 43 Healthcare Marija F psychologist x x Centre Nikšić Petranić 26 Zagorka Ana F pedagogue x x x Ivanović Rudović 44 Dijana F medical x x Kindergarten, Andrić nurse 27 Cetinje Ljiljana F preschool x x Vušurović teacher 45 Healthcare Jelena F psychologist x Centre Herceg Pejović 28 Dušo Basekić Dragana F pedagogue x x x x Novi Kindergarten, Drašković 46 Bogoljubka F medical x Bijelo Polje Dabanović nurse 29 Marija F preschool x x x x Bulatović teacher 47 Healthcare Branislavka F psychologist x x Centre Cetinje Vujović 30 Dragan Radovan M psychologist x x x 48 Vesna F medical x x Kovačević Cicmil Vulaš nurse Kindergarten, 31 Nikšić Jelena F pedagogue x x x Marković

32 Healthcare Milena F social worker x x Centre Lazović Podgorica 33 Bojana F medical x x x Stanišić nurse

34 Aida F paediatric x x x Piranić nurse

35 Bojana F paediatric x x x Grahovac nurse

58 Scaling up Parenting for Lifelong Health in Montenegro: A feasibility assessment Scaling up Parenting for Lifelong Health in Montenegro: A feasibility assessment 59 Appendix B. Rates of participant recruitment, enrolment, completion and attendance across sites

Cycle 1 Cycle 2 Cycle 3 Cycle 4 Cycle 5 Cycle 6

R E C A R E C A R E C A R E C A R E C A R E C A

Healthcare 5 4 4 97.9% 9 9 3 46.3% 4 4 4 93.8% 5 5 5 100% Centre Podgorica

Healthcare 15 14 13 83.9% 14 14 12 86.3% 14 13 12 82.7% 12 12 11 93.8% 13 13 13 100% Centre Berane

Healthcare 8 5 2 56.7% Centre Nikšić

Healthcare Centre 11 11 11 95.5% 10 10 10 85.8% 10 8 8 74.2% 13 13 13 89.7% 14 14 14 94.0% Bijelo Polje

Healthcare 10 10 7 75.0% Centre Cetinje

NGO Association 13 13 13 90.4% 12 11 11 100% 11 11 11 100% 31 31 26 86.0% 25 24* 21* 86.5%* Parents

NGO Pedagogical 8 8 8 100% 10 9 9 100% 9 9 8 89.8% 17 17 17 95.6% 6 6 6 100% Centre

NGO Centre for 11 11 5 53.0% Child Rights

NGO Centre for 16 13 8 70.5% Roma Initiatives

Kindergarten 11 9 9 79.6% 9 8 7 83.3% 11 11 11 93.2% 20 20 17 76.7% 20 19 16 77.6% Podgorica

Kindergarten 12 12 10 82.6% 8 8 7 93.8% Cetinje

Kindergarten 12 12 12 87.5% 12 12 12 84.0% 7 7 6 73.8% Bijelo Polje

Kindergarten 8 8 7 77.1% 11 10 10 88.6% Nikšić

TOTAL / RATE 82 75 65 82.1% 55 52 49 91.1% 64 61 53 81.0% 145 142 125 85.3% 124 121 111 90.0% 7 7 6 73.8% ACROSS SITES

R = numbers recruited; E = numbers enrolled (attended at least one session); C = numbers completed (attended at least seven sessions) A = average attendance rate of enrolled participants *Data is missing for one participant

60 Scaling up Parenting for Lifelong Health in Montenegro: A feasibility assessment Scaling up Parenting for Lifelong Health in Montenegro: A feasibility assessment 61 Appendix C. Estimated costs of PLH-YC programme delivery Online parent group Funding Unit Unit cost (€) No. Total cost (€) sessions (if in lieu of source* description units PLH Young Children Programme Delivery (12-session version) for one parenting group in-person sessions) (12 parents/caregivers) Fees for on-line meeting Local funding Annual licence 200.00 1 200.00 platform hosting (if applicable) Weekly parent group sessions Internet access for Local funding 2 facilitators × Variable 6 Variable Materials Funding Unit Unit cost (€) No. Total cost (€) facilitators (if applicable) 3 months source* description units

Internet access for Local funding 12 participants 8.00 36 288.00 participants (if applicable) × 3 months Printing of Parent Handbooks Local funding Handbooks 1.00 12 12.00

Pre-programme consultations

Miscellaneous materials Local funding Participants 2.00 12 24.00 Transport Funding Unit Unit cost (€) No. Total cost (€) source* description units

Wages during Central budget Participants 2.34 12 28.08 Facilitators Funding Unit Unit cost (€) No. Total cost (€) pre-programme consultations source* description units (1 hour per family)

Catch-up consultations Wages (1.5 working days per Central budget 2 facilitators × 18.72 36 673.92 Transport Funding Unit Unit cost (€) No. Total cost (€) person per week) 18 days source* description units

Facilitator transport to Local funding Participants Variable 15 Variable Daily subsistence, round-trip Local funding 2 facilitators × Variable 24 Variable participants’ houses travel & accommodation 12 days (if applicable) (if applicable) Weekly communications Venue & Food Funding Unit Unit cost (€) No. Total cost (€) source* description units Mobile Phone use Funding Unit Unit cost (€) No. Total cost (€) source* description units

Phone calls to parent / Local funding 2 facilitators × Variable 6 Variable Venue rental (for NGOs with Central budget Days Variable 12 Variable caregiver participants 3 months spatial needs)

Refreshments Local funding 14 persons × 1.00 168 168.00 Internet access Local funding 2 facilitators × Variable 6 Variable 12 days (for Viber parent group 3 months communications)

Accessibility provisions Funding Unit Unit cost (€) No. Total cost (€) source* description units (for communities with TOTAL ESTIMATED COSTS obstacles to participation) 906.00 (for in-person delivery, excluding potential applicable costs and excluding variable budget items)

Child care services Central budget Sessions 15.00 12 180.00 TOTAL ESTIMATED COSTS (if applicable) 1,316.40 (for in-person delivery, including potential applicable costs and excluding variable budget items)

TOTAL ESTIMATED COSTS Transport for participants to Central budget 12 persons × 1.60 144 230.40 1,202.00 venue (if applicable) 12 days (for online delivery, including potential applicable costs and excluding variable budget items)

*suggested source based on interviews and focus group discussions

62 Scaling up Parenting for Lifelong Health in Montenegro: A feasibility assessment Scaling up Parenting for Lifelong Health in Montenegro: A feasibility assessment 63 Appendix D. Examples of cascade training models

From a top-down perspective, the organization of training (for example), with each facilitator pair delivering just two Figure C depicts how one training course of trainers 1 can generate a pool of at least four trainers. These for facilitators may be organized as depicted in Figure A, cycles annually, a total of seven training courses for 168 PLH-YC Master Trainer with one facilitator training course by two PLH-YC trainers facilitators would first need to be organized (not taking trainers, in turn, can organize training in pairs to generating 24 facilitators. If a target of reaching about into account the existing cohort of facilitators). conduct four training courses for a total of 96 2,000 parents/caregivers per year is established facilitators (24 per training), as well as one supervisor training course by each trainer (six facilitators per supervisor).

Figure A. Sample facilitator training cascade model for reaching 2,016 parents/caregivers in one year, 1 Training: Pool of 4 Trainers with two PLH-YC cycles per year Figure C. Sample training of trainers cascade model for one trainer training 2 course, with four trainers training 96 PLH-YC Trainers facilitators and 24 supervisors

4 Trainings: 96 4 Trainings: 96 Facilitators Facilitators 24 24 24 24 24 24 24 Facilitators Facilitators Facilitators Facilitators Facilitators Facilitators Facilitators (12 pairs) (12 pairs) (12 pairs) (12 pairs) (12 pairs) (12 pairs) (12 pairs)

24 groups 24 groups 24 groups 24 groups 24 groups 24 groups 24 groups Photo: (288 parents) (288 parents) (288 parents) (288 parents) (288 parents) (288 parents) (288 parents) Duško Miljanić / UNICEF Montenegro

1 Figure B shows how three PLH-YC Trainer training courses of supervisors would generate a total of 18 supervisors, who in turn can supervise a total of 144 facilitators (eight facilitators per supervisor).

6 Supervisors 6 Supervisors 6 Supervisors

Figure B. Sample supervisor training cascade model for three training courses, with 18 supervisors supervising 144 facilitators

48 facilitators 48 facilitators 48 facilitators (8 per supervision (8 per supervision (8 per supervision group) group) group)

64 Scaling up Parenting for Lifelong Health in Montenegro: A feasibility assessment Scaling up Parenting for Lifelong Health in Montenegro: A feasibility assessment 65 Appendix E. List of research participants

Group Name Position, Organization Data type Group Name Position, Organization Data type

Directors, Primary Dr Nebojša Kavarić HC Podgorica Interview Facilitators & supervi- Ms Dijana Jović Psychologist (facilitator & supervisor), Đina FGD Healthcare Centres sors, education sector Vrbica Kindergarten, Podgorica Dr Zuhra Hadrović HC Berane Interview Ms Ana Rudović Pedagogue (facilitator), Zagorka Ivanović Dr Majda HC Bijelo Polje Written Kindergarten, Cetinje Dobardžić response Ms Dragana Drašković Pedagogue (facilitator), Dušo Basekić Kinder- Directors, NGOs Ms Tanja Sabolski NGO Family Centre Interview garten, Bijelo Polje

Ms Kristina NGO Association Parents Interview Mr Radovan Cicmil Psychologist (facilitator), Kindergarten Dra- Mihailović gan Kovačević, Nikšić

Ms Fana Delija NGO Centre for Roma Initiatives Interview Ms Ilhana Ećo Psychologist (facilitator), Ljubica Popović Kindergarten, Podgorica Directors, kindergar- Mr Vuk Stanišić Kindergarten Podgorica Interview tens Facilitators & supervi- Mr Miloš Bulatović Psychologist (facilitator & supervisor), Peda- FGD Ms Gordana Knežević Kindergarten Bijelo Polje Interview sors, NGO sector gogical Centre, Podgorica

Ms Senka Kindergarten Cetinje Interview Ms Darija Petović Psychotherapist (facilitator & supervisor), Vuksanović Bambur Parents Association, Podgorica

Ministries Ms Tamara Milić Head of the Unit for Preschool & Inclusive Interview Ms Vesna Antović Social worker (facilitator), NGO Family Education, Ministry of Education Centre

Ms Svetlana Sovilj Head of the Unit for Child Protection, Interview Ms Stanislava Šće- Pedagogue (facilitator), Ljubović Centre for Written Ministry of Finance and Social Welfare panović-Radević Children with Behaviour Problems response

Ms Milica Dukić Advisor, Ministry of Health Interview Ms Ana Jovanovich Psychologist (facilitator), Centre for Child Written Hales Rights response National institutions Professor Dr Agima Director, Centre for Health Promotion, Interview (jointly Ljaljević Institute for Public Health compiled)

Professor Dr Tatjana Dean, Faculty of Philosophy, University of Interview Ms Ana Kulić Psychologist (facilitator), Centre for Child Novović Montenegro Rights

Dr Bojana Miletić Head of the Unit, Department for Interview Programme Professor Professor, Bangor University Interview Supervision and Professional Support, developers Judy Hutchings Institute for Social & Child Protection Dr Jamie Research Officer, University of Oxford Interview Facilitators & Ms Ana Golubović Psychologist (facilitator & supervisor), HC FGD Lachman supervisors, Popović Berane health sector UNICEF Ms Michael Bauer Deputy Representative FGD Ms Tanja Bulatović Medical nurse (facilitator), HC Berane

Ms Danijela Femić Psychologist (facilitator & supervisor), HC Ms Ida Ferdinandi Early Childhood Development Officer Bijelo Polje Ms Nela Krnić Child Protection Officer Ms Marija Petranić Psychologist (facilitator), HC Nikšić Ms Nada Đurović Child Protection Officer Martinović Ms Branislavka Vujović Psychologist (facilitator), HC Cetinje Written response Ms Maja Kovačević Education Officer

Ms Ivana Ceković Education Officer Ms Aida Piranić Pediatric nurse (facilitator), HC Podgorica Written response Mr Nikola Vulić Adolescent Development Officer

66 Scaling up Parenting for Lifelong Health in Montenegro: A feasibility assessment Scaling up Parenting for Lifelong Health in Montenegro: A feasibility assessment 67 components to reduce physical child abuse 34. Lachman JM, Sherr LT, Cluver L, Ward CL, Hutchings References recurrence? Reanalysis of a systematic review using J, Gardner F. Integrating evidence and context to qualitative comparative analysis. Child Abuse Review. develop a parenting program for low-income families 2019;28(3):181-97. in South Africa. Journal of Child and Family Studies. 1. United Nations General Assembly. The Rights of the 14. Cowan CP, Cowan PA, Barry J. Couples' groups 27. Patterson GR. The next generation of PMTO models. 2016;25(7):2337-52. Child: Resolution adopted by the General Assembly, 15 for parents of preschoolers: ten-year outcomes of Behavior Therapist. 2005;28(25-32). 35. Gardner F, McCoy A, Lachman J, Ward C, Tapanya S. February 2002, A/RES/56/138. A/RES/56/138. a randomized trial. Journal of Family Psychology. 28. Sanders MR. Triple P-Positive Parenting Program: Manuscript in development. 2021. 2. World Health Organization, United Nations Office 2011;25(2):240-50. Towards an empirically validated multilevel parenting 36. Leung C, Sanders MR, Leung S, Mak R, Lau J. An on Drugs and Crime, United Nations Development 15. Furlong M, McGilloway S, Bywater T, Hutchings J, and family support strategy for the prevention of outcome evaluation of the implementation of the Triple Program. Global status report on violence prevention. Smith SM, Donnelly M. Cochrane review: Behavioural behavior and emotional problems in children. Clinical P-Positive Parenting Program in Hong Kong. Family 2014. Available from: https://www.who.int/violence_ and cognitive-behavioural group-based parenting Child and Family Psychology Review. 1999;2:71-90. Process. 2003;42(4):531-44. injury_prevention/violence/status_report/2014/en/. programmes for early-onset conduct problems in 29. Webster-Stratton C, Reid MJ. The Incredible Years 37. Puffer ES, Annan J, Sim AL, Salhi C, Betancourt TS. The 3. Hillis S, Mercy J, Amobi A, Kress H. Global prevalence children aged 3 to 12 years (Review). Evidence-Based parents, teachers and child training series: A impact of a family skills training intervention among of past-year violence against children: A systematic Child Health, . 2012;8(2):318-692. multifacted treatment approach for young children with Burmese migrant families in Thailand: A randomized review and minimum estimates. Pediatrics. 16. Gross D, Fogg L, Webster-Stratton C, Garvey C, Julion conduct problems. In: Kazdin AE, Weisz JR, editors. controlled trial. PloS one. 2017;12(3):e0172611. 2016;137(3). W, Grady J. Parent training of toddlers in day care in Evidence-based psychotherapies for children and 38. Alampay LP, Lachman JM, Landoy BV, Madrid BJ, Ward 4. Stoltenborgh M, Bakermans-Kranenburg MJ, van low-income urban communities. Journal of Consulting adolescents. New York: Guilford Press; 2003. p. 224-40. CL, Hutchings J, et al. Preventing child maltreatment Ijzendoorn MH, Alink LRA. Cultural–geographical and Clinical Psychology. 2003;71(2):261-78. 30. Foster EM, Prinz RJ, Sanders MR, Shapiro CJ. The in low and middle-income countries: Parenting differences in the occurrence of child physical abuse? 17. Sanders M, McFarland M. Treatment of depressed costs of a public health infrastructure for delivering for Lifelong Health in the Philippines. In: Verma S, A meta-analysis of global prevalence. International mothers with disruptive children: A controlled parenting and family support. Children and Youth Petersen C, editors. Developmental science and Journal of Psychology. 2013;48(2):81-94. evaluation of cognitive behavioural family intervention. Services Review. 2008;30(5):493-501. sustainable development goals for children and youth, 5. MONSTAT & UNICEF. Montenegro Multiple Indicator Behavior Therapy. 2000;31:89-112. 31. Nystrand C, Feldman I, Enebrink P, Sampaio F. Cost- Social Indicators Research Series 742018. p. 277-93. Cluster Surveys 2018: Child Discipline 2018. Available 18. Chen M, Chan KL. Effects of parenting programs effectiveness analysis of parenting interventions for 39. Armstrong MI, Birnie-Lefcovitch S, Ungar MT. from: https://www.unicef.org/montenegro/en/ on child maltreatment prevention: A meta-analysis. the prevention of behaviour problems in children. PloS Pathways Between Social Support, Family Well statistical-snapshot-general-population. Trauma, violence & abuse. 2016;17(1):88-104. one. 2019;14(12):e0225503. Being, Quality of Parenting, and Child Resilience: 6. Arnow BA. Relationships between childhood 19. Kaminski JW, Valle LA, Filene JH, Boyle CL. A meta- 32. O'Neill D, McGilloway S, Donnelly M, Bywater T, Kelly What We Know. Journal of Child and Family Studies. maltreatment, adult health and psychiatric outcomes, analytic review of components associated with parent P. A cost-effectiveness analysis of the Incredible Years 2005;14(2):269-81. and medical utilization. Journal of Clinical Psychiatry. training program effectiveness. Journal of Abnormal parenting programme in reducing childhood health 40. Barlow J, Coren E, Stewart-Brown S. Meta-analysis 2004;65(12):10-5. Child Psychology. 2008;36(4):567-89. inequalities. Eur J Health Econ. 2013;14(1):85-94. of the effectiveness of parenting programmes in 7. Fulu E, Miedema S, Roselli T, McCook S, Chan KL, 20. MacMillan HL, Wathen CN, Barlow J, Fergusson DM, 33. Mikton C. Two challenges to importing evidence-based improving maternal psychosocial health. British Journal Haardörfer R, et al. Pathways between childhood Leventhal JM, &, Taussig HN. Interventions to prevent child maltreatment prevention programs developed of General Practice. 2002;52:223-33. trauma, intimate partner violence, and harsh parenting: child maltreatment and associated impairment. The in high-income countries to low- and middle-income 41. Bennett C, Barlow J, Huband N, Smailagic N, Roloff findings from the UN Multi-country Study on Men and Lancet. 2009;373(9659):250-66. countries: Generalizability and affordability. In: V. Group-based parenting programs for improving Violence in Asia and the Pacific. The Lancet Global 21. Barlow J, Johnston I, Kendrick D, Polnay L, Stewart- Dubowitz H, editor. World perspectives on child abuse. parenting and psychosocial functioning: A systematic Health. 2017;5(5):e512-e22. Brown S. Individual and group-based parenting 10. Aurora, CO: International Society for the Prevention review. Journal of the Society for Social Work and 8. Maas C, Herrenkohl TI, Sousa C. Review of research programmes for the treatment of physical child of Child Abuse and Neglect; 2012. p. 97. Research. 2013;4(4):300-32. on child maltreatment and violence in youth. Trauma, abuse and neglect. Cochrane Database of Systematic violence & abuse. 2008;9(1):56-67. Reviews. 2006(3). 9. Maniglio R. The impact of child sexual abuse on health: 22. Gardner F, Knerr W, Lachman J, Melendez-Torres GJ, a systematic review of reviews. Clinical Psychology Riddell J, Wight D. Parenting for preventing risk of child Review. 2009;29(7):647-57. maltreatment in low- and middle-income countries: 10. Norman RE, Byambaa M, De R, Butchart A, Scott J, Updated systematic review and meta-analysis. Vos T. The long-term health consequences of child Unpublished manuscript. 2020. physical abuse, emotional abuse, and neglect: A 23. Knerr W, Gardner F, Cluver L. Improving positive systematic review and meta-analysis. PLoS medicine. parenting skills and reducing harsh and abusive 2012;9(11):e1001349. parenting in low- and middle-income countries: A 11. Pereznieto P, Montes A, Langston L, Routier S. systematic review. Prevention Science. 2013;14(4):352- The costs and economic impact of violence against 63. children. London; 2014. Available from: https:// 24. McCoy A, Melendez-Torres GJ, Gardner F. Parenting childfundalliance.org/wp-content/uploads/2014/10/ODI- interventions to prevent violence against children Policy-Brief.-The-cost-and-economic-impact-of-violence- in low- and middle-income countries in East and against-children.pdf. Southeast Asia: A systematic review and multi- 12. World Health Organization. INSPIRE: Seven strategies level meta-analysis. Child abuse & neglect. for ending violence against children. 2016. Available 2020;103:104444. from: https://www.who.int/violence_injury_prevention/ 25. Dishion TJ, Patterson GR. The development and violence/inspire/en/. ecology of antisocial behavior in children and 13. Cartwright-Hatton S, McNally D, Field AP, Rust S, adolescents. In: Cicchetti D, Cohen DJ, editors. Laskey B, Dixon C, et al. A new parenting-based Developmental psychopathology: Risk, disorder, and group intervention for young anxious children: Results adaptation. 3. 2nd ed. Hoboken, NJ: John Wiley & Sons of a randomized controlled trial. Journal of the Inc.; 2006. p. 503-41. Photo: Duško Miljanić / American Academy of Child & Adolescent Psychiatry. 26. Melendez-Torres GJ, Leijten P, Gardner F. What are UNICEF Montenegro 2011;50(3):242-51.e6. the optimal combinations of parenting intervention

68 Scaling up Parenting for Lifelong Health in Montenegro: A feasibility assessment Scaling up Parenting for Lifelong Health in Montenegro: A feasibility assessment 69 42. Gardner F, Burton J, Klimes I. Randomised controlled Children from Violence from 2017-2021.” Hearing 71. Moore CG, Carter RE, Nietert PJ, Stewart PW. 77. WHO. Scaling up health services: trial of a parenting intervention in the voluntary sector before the Committee on Human Rights and Recommendations for planning pilot studies in clinical Challenges and Choices. Technical Brief No. 3. 2008. for reducing child conduct problems: outcomes and Freedoms, Parliament of Montenegro, 27th Sess. and translational research. Clinical and Translational Available from: https://www.who.int/healthsystems/ mechanisms of change. Journal of Child Psychology (2018). Science. 2011;4(5):332-7. topics/delivery/technical_brief_scale-up_june12. and Psychiatry. 2006;47(11):1123-32. 58. Chacko A, Jensen SA, Lowry LS, Cornwell M, Chimklis 72. van Aar J, Leijten P, Orobio de Castro B, Overbeek G. pdf?ua=1. 43. Barlow J, Smailagic N, Ferriter M, Bennett C, Jones H. A, Chan E, et al. Engagement in Behavioral Parent Sustained, fade-out or sleeper effects? A systematic 78. Willis CD, Riley BL, Stockton L, Abramowicz A, Group-based parent-training programmes for improving Training: Review of the Literature and Implications for review and meta-analysis of parenting interventions Zummach D, Wong G, et al. Scaling up complex emotional and behavioural adjustment in children from Practice. Clin Child Fam Psychol Rev. 2016;19(3):204- for disruptive child behavior. Clin Psychol Rev. interventions: insights from a realist synthesis. birth to three years old (Review). Cochrane Database of 15. 2017;51:153-63. Health Res Policy Syst. 2016;14(1):88. Systematic Reviews, Issue 3 Art No: CD003680. 2010. 59. Wessels I, Lester S, Ward CL. Engagement in 73. Mangham LJ, Hanson K. Scaling up in international 79. Hartmann A, Linn JF. Scaling up: A framework and 44. Lachman J. Global Parenting Research Group: Studies parenting programmes: Exploring facilitators of and health: What are the key issues? Health Policy and lessons for development effectiveness from literature on parenting interventions in low- and middle-income barriers to participation, Policy Brief 82, Institute for Planning, . 2010;25(2):85-96. and practice. SSRN Electronic Journal. 2008. countries at the Department of Social Policy and Security Studies. 2016. Available from: https://issafrica. 74. World Health Organization, ExpandNet. Nine steps 80. Perez-Escamilla R, Cavallera V, Tomlinson M, Dua T. Intervention. [Presentation]. In press 2021. org/research/policy-brief/engagement-in-parenting- for developing a scaling-up strategy. 2010. Available Scaling up integrated early childhood development 45. Ward CL, Wessels IM, Lachman JM, Hutchings J, programmes-exploring-facilitators-of-and-barriers-to- from: https://www.who.int/immunization/hpv/deliver/ programs: Lessons from four countries. Child: Care, Cluver LD, Kassanjee R, et al. Parenting for Lifelong participation. nine_steps_for_developing_a_scalingup_strategy_ Health and Development. 2018;44:50-61. Health for young children: a randomized controlled trial 60. Kazdin AE. Perceived barriers to treatment participation who_2010.pdf. 81. Cavallera V, Tomlinson M, Radner J, Coetzee B, of a parenting program in South Africa to prevent harsh and treatment acceptability among antisocil children 75. Mikton C, Power M, Raleva M, Makoae M, Al Eissa M, Daelmans B, Hughes R, et al. Scaling early child parenting and child conduct problems. Journal of Child and their families. Journal of Child and Family Studies. Cheah I, et al. The assessment of the readiness of five development: what are the barriers and enablers? Psychology and Psychiatry. 2019. 2000;9:157-74. countries to implement child maltreatment prevention Archives of disease in childhood. 2019;104(Suppl 46. Lachman JM, Alampay LP, Alinea M, Madrid B, Ward 61. Hutchings J. Final report: UNICEF Montenegro programs on a large scale. Child abuse & neglect. 1):S43-S50. C, Hutchings J, et al. Integrating parent management Parenting for Lifelong Health Caring Families 2013;37(12):1237-51. 82. Child and Family Agency GoI. Investing training within a conditional cash transfer system programme for parents of 2–9 year olds project. 76. Banerjee A, Banerji R, Berry J, Duflo E, Kannan H, in Families: Supporting parents to improve for low-income families: Results from a small- Bangor University; 1998. Mukerji S, et al. From proof of concept to scalable outcomes for children. 2013. Available from: scale randomized controlled trial in the Philippines. 62. Dimić Raičević T. Better parenting programmes in policies: Challenges and solutions, with an application. https://www.tusla.ie/uploads/content/Family_Support_ Unpublished manuscript. 2021. Montenegro. 2018. Available from: https://www. Journal of Economic Perspectives. 2017;31(4):73-102. CFA_Parenting_Support_Strategy.pdf. 47. UNICEF Montenegro. EU and UNICEF: Early childhood unicef.org/montenegro/en/stories/better-parenting- development to be a national priority 2020. Available programmes-montenegro. from: https://www.unicef.org/montenegro/en/stories/ 63. Dimić Raičević T. Parenting schools expand to include Photo: eu-and-unicef-early-childhood-development-be-national- Nikšić. 2020. Available from: https://wcmsprod.unicef. Duško Miljanić / priority. org/montenegro/en/stories/parenting-schools-expand- UNICEF Montenegro 48. Creswell JW. A concise introduction to mixed methods include-niksic. research. Thousand Oaks, CA: Sage; 2014. 64. Meinck F, Boyes ME, Cluver L, Ward CL, Schmidt 49. Greene JC. Mixed methods in social inquiry. San P, DeStone S, et al. Adaptation and psychometric Francisco, CA: Jossey-Bass; 2007. properties of the ISPCAN Child Abuse Screening 50. Creswell JW, Plano Clark VL. Designing and conducting Tool for use in trials (ICAST-Trial) among South African mixed methods research. 3rd ed. Thousand Oaks: Sage adolescents and their primary caregivers. Child Abuse Publications; 2017. and Neglect. 2018;82:45-58. 51. Morse JM. Approaches to qualitative-quantitative 65. McEachern AD, Dishion TJ, Weaver CM, Shaw DS, methodological triangulation. Nursing Research. Wilson MN, Gardner F. Parenting Young Children 1991;40(2):120-3. (PARYC): Validation of a self-report parenting measure. 52. Group UNE. Norms and Standards for Evaluation. Journal of Child and Family Studies. 2012;21(3):498-511. New York; 2017. 66. Arnold DS, O'Leary SG, Wolff LS, Acker MM. The 53. Wilson SM, Wildman BG, Ciesla J, Smith AW, Parenting Scale: A measure of dysfunctional parenting Dempster R. Factor structure of the Obstacles in discipline situations. Psychological Assessment. of Engagement Scale: Problems, solutions, and 1993;5(2):137-44. hypotheses. Journal of Child & Family Studies. 67. Goodman R. The Strengths and Difficulties 2015;24:891-8. Questionnaire: A research note. Journal of Child 54. Crabtree B, Miller W. A template approach to text Psychology and Psychiatry. 1997;38:581-6. analysis: Developing and using codebooks. In: Crabtree 68. Berry JO, Jones WH. The parental stress scale: Initial B, Miller W, editors. Doing qualitative research. psychometric evidence. Journal of Social and Personal Newbury Park, CA: Sage; 1999. Relationships. 1995;12:463-72. 55. Boyatzis RE. Transforming qualitative information: 69. Irwin M, Artin KH, Oxman MN. Screening for Thematic analysis and code development. Thousand depression in the older adult: Criterion validity of the Oaks: Sage Publications; 1998. 10-item Center for Epidemiological Studies Depression 56. Patton MQ. Qualitative research & evaluation methods: Scale (CES-D). Archives of Internal Medicine. Integrating theory and practice. 4th ed. Los Angeles: 1999;159:1701-4. Sage; 2015. 70. Finch WH, Bolin JH, Kelley K. Group membership 57. Report on the Consultative Hearing on the Topic: predication when known groups consists of unknown “Implementation of the Action Plan for Implmentation subgroups: A Monte Carlo comparison of methods. of the Strategy for Prevention and Protection of Frontiers in Psychology. 2014;5:337.

70 Scaling up Parenting for Lifelong Health in Montenegro: A feasibility assessment Scaling up Parenting for Lifelong Health in Montenegro: A feasibility assessment 71 83. The Scottish Government. National Parenting Strategy: and health outcomes: a cluster-randomised factorial official journal of the Society for Prevention Research. Making a positive difference ot children and young effectiveness trial. The Lancet. 2014;384(9950):1282- 2011;12(1):23-33. people through parenting. 2012. 93. 111. Bernal G. Intervention development and cultural Available from: https://www.gov.scot/publications/ 97. Kaludjerovic J, Mirkovic M, Golubovic V. Investments adaptation research with diverse families. Family national-parenting-strategy-making-positive-difference- in Early Childhood Development: Case of Montenegro. Process. 2006;45:143-51. children-young-people-through/. 2019. 112. Sundell K, Ferrer-Wreder L, Fraser MW. Going global: 84. Ministry for the Family and Social Solidarity GoM. 98. Sanders MR. Using positive parenting programs in A model for evaluating empirically supported family- Positive Parenting: National Strategic Policy 2016- different delivery systems: An introduction. In: Sanders based interventions in new contexts. Eval Health Prof. 2024. 2016. Available from: https://family.gov.mt/en/ M, Mazzucchelli G, editors. The Power of Positive 2014;37(2):203-30. Documents/National%20Parenting%20Policy%20 Parenting: Transforming the Lives of Children, Parents, 113. Doubt J, Bray R, Loening-Voysey H, Cluver L, Byrne J, English%208.02.17.pdf. and Communities Using the Triple P System. Oxford: Nzima D, et al. "It has changed": Understanding change 85. Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Oxford University Press; 2017. p. 227-30. in a parenting program in South Africa. Ann Glob Alexander JA, Lowery JC. Fostering implementation 99. Development PSUCoHaH. Strengthening Families 10- Health. 2017;83(5-6):767-76. of health services research findings into practice: a 14 Implementation Manual 2015. Available from: http:// 114. McCoy A, Lachman JM, Tapanya S, Ward C, Gardner consolidated framework for advancing implementation www.episcenter.psu.edu/ebp/strengthening/im. F. Assessing the potential for reducing violence science. Implement Sci. 2009;4:50. 100. IDEAS. Analysis of the Work of Centres for Social Work against young children through an adapted parenting 86. Sanders MR, Kirby JN. Consumer engagement and in Montenegro. Podgorica: UNICEF; 2019. programme in Thailand: Findings from a pre-post trial the development, evaluation, and dissemination of 101. United Nations. Integrated UN Programme for (unpublished manuscript). 2019. evidence-based parenting programs. Behav Ther. Montenegro 2017-2021: UN Development Assistance 115. Charles JM, Bywater T, Edwards RT. Parenting 2012;43(2):236-50. Framework for Montenegro. 2016. Available from: interventions: a systematic review of the economic 87. Ministry of Education GoNZ. Guidelines for Incredible https://unece.org/DAM/operact/Technical_Cooperation/ evidence. Child Care Health Dev. 2011;37(4):462-74. Years Parent Programme. 2014. Available from: Delivering_as_One/UNDAF_country_files/UNDAF_ 116. Jamison DT. Cost effectiveness analysis: Concepts and https://pb4l.tki.org.nz/Incredible-Years-Parent. files_2015-2020/Montenegro-UNDAF-2017-2021-Final. applications. Oxford Testbook of Public Health: Volume 88. Durlak JA, DuPre EP. Implementation matters: a pdf. 2, The Methods of Public Health. 5th ed. Oxford: review of research on the influence of implementation 102. Mogro-Wilson C, Hayes C, Loomis AM, Drake A, Oxford University Press; 2009. p. 767-82. on program outcomes and the factors affecting Martin-Peele M, Fifield J. Supporting recruitment and implementation. American Journal of Community retention of young African-American and Hispanic Psychology. 2008;41(3-4):327-50. fathers in community-based parenting interventions 89. Aarons GA, Green AE, Palinkas LA, Self-Brown S, research. Advances in Social Work. 2019;18(4):1068-84. Whitaker DJ, Lutzker JR, et al. Dynamic adaptation 103. Stahlschmidt MJ, Threlfall J, Seay KD, Lewis EM, Kohl process to implement an evidence-based child PL. Recruiting Fathers to Parenting Programs: Advice maltreatment intervention. Implementation Science. from Dads and Fatherhood Program Providers. Child 2012;7(32). Youth Serv Rev. 2013;35(10):1734-41. 90. Flay BR, Biglan A, Boruch RF, Gonzalez Castro F, 104. Eyberg S, Pincus D. Eyberg Child Behavior Inventory Gottfredson D, Kellam S, et al. Standards of evidence: and Sutter-Eyberg Student Behavior Inventory - revised: Criteria for efficacy, effectiveness and dissemination. Professional manual. Odessa, FL: Pyshcological Prevention Science. 2005;6(3):151-75. Assessment Resources; 1999. 91. Marchand E, Stice E, Rohde P, Becker CB. Moving from 105. Triple P Positive Parenting Program. Standard Stepping efficacy to effectiveness trials in prevention research. Stones Triple P 2017. Available from: https://www. Behaviour Research and Therapy. 2011;49(1):32-41. triplep.net/files/5315/2886/8005/ENG_Standard_ 92. Parenting for Lifelong Health. Parenting for Lifelong Stepping_Stones_Triple_P_LTR.pdf. Health for Young Children: Implementation Guide for 106. The Incredible Years. Autism spectrum and Service Providers. Cape Town: Clows Without Borders language delays 2013. Available from: https://www. South Africa; 2021. incredibleyears.com/programs/parent/autism- 93. Parenting for Lifelong Health. Parenting for Lifelong spectrum-language-delays/. Health for Young Children: Monitoring & Evaluation 107. Cabassa LJ, Baumann AA. A two-way street: Bridging Manual. Cape Town: Clowns Without Borders South implementation science and cultural adaptations of Africa; 2021. mental health treatments. Implementation Science. 94. Hutchings J. Introducing, researching, and 2013;8(90). disseminating the Incredible Years Programmes in 108. Furlong M, McGilloway S. Barriers and facilitators to Wales. International Journal of Conflict and Violence. implementing evidence-based parenting programs in 2012;6(2):225-33. disadvantaged settings: A qualitative study. Journal of 95. Walker SP, Baker-Henningham H, Chang SM, Child and Family Studies. 2015;24(6):1809-18. Powell CA, Lopez-Boo F, Grantham-Mcgregor S. 109. University of Oxford. Study Protocol: COVID-19 Implementation of parenting interventions through e-Parenting for Lifelong Health (ePLH) Pilot Study - health services in Jamaica. Vulnerable Children and Promoting positive parenting and preventing violence Youth Studies. 2018;13(2):127-41. through online parent support groups - ParentChat. 96. Yousafzai AK, Rasheed MA, Rizvi A, Armstrong R, Oxford: University of Oxford; 2020. Bhutta ZA. Effect of integrated responsive stimulation 110. Berkel C, Mauricio AM, Schoenfelder E, Sandler IN. and nutrition interventions in the Lady Health Worker Putting the pieces together: an integrated model of programme in Pakistan on child development, growth, program implementation. Prevention science : the

72 Scaling up Parenting for Lifelong Health in Montenegro: A feasibility assessment