Evaluation of a Multimodal School-Based Depression and Suicide Prevention Program Among Dutch Adolescents: Design of a Cluster-Randomized Controlled Trial Mandy W

Gijzen et al. BMC Psychiatry (2018) 18:124 https://doi.org/10.1186/s12888-018-1710-2

STUDYPROTOCOL Open Access Evaluation of a multimodal school-based depression and suicide prevention program among Dutch adolescents: design of a cluster-randomized controlled trial Mandy W. M. Gijzen1,2,3*, Daan H. M. Creemers3,4, Sanne P. A. Rasing3,5, Filip Smit1,6 and Rutger C. M. E. Engels2

Abstract Background: Since 2010, suicide has been the most important cause of mortality in youth aged 15 to 29 years in the Netherlands. Depression is an important risk factor for suicidal behaviors (i.e., suicide ideation, deliberate self- harm, planning, and suicide attempts) in adolescents. Adolescents who develop depressive symptoms, are also at risk for adult depression. This developmental continuity is especially noticeable in adolescents compared to other age groups; therefore, it is necessary to develop preventive strategies for teens. This study will test a multimodal school-based approach to suicide and depression prevention, which integrates universal and targeted approaches and includes various stakeholders (schools, adolescents, parents, and mental health professionals) simultaneously. Methods: We will perform a cluster randomized controlled trial (RCT) with an intervention and control condition to test the effectiveness of a school-based multimodal stepped-prevention program for depression and suicidal behaviors in adolescents. Adolescents in their second year of secondary education will participate in the study. The participants in the intervention condition will receive the entire multimodal stepped-preventive program comprising early screening and detection of suicidal behaviors and depressive symptoms, a safety net consisting of gatekeepers at school, followed by universal and indicated prevention. The participants in the control condition will undergo only the screening and the safety net of gatekeepers at schools. They will complete assessments at baseline, post-intervention, and 6, 12, and 24-month follow-up. Primary outcome will be suicidal behaviors measured at 12-months follow-up. Additionally, the present study will identify mechanisms that mediate and moderate the program effects and test the effect of the program on various secondary outcomes. Discussion: If the school-based multimodal stepped-prevention program proves to be effective, it could be implemented in schools on a large scale. Trial registration: The study is registered in the Dutch Trial Register (NTR6622). Keywords: Prevention, Suicide, Depression, Adolescents, School-based, Multimodal

* Correspondence: [email protected] 1Trimbos Institute (Netherlands Institute of Mental Health and Addiction), P.O. Box 725, 3500 AS Utrecht, The Netherlands 2Erasmus School of Social and Behavioural Sciences, Erasmus University, P.O. Box 1738, 3000 DR Rotterdam, The Netherlands Full list of author information is available at the end of the article

© The Author(s). 2018 Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated. Gijzen et al. BMC Psychiatry (2018) 18:124 Page 2 of 12

Background behaviors in adolescents have a high likelihood of recur- Each year, 11.2% of Dutch youth have suicidal thoughts and rence [16]. Adolescents who attempt suicide are more likely 6.6% attempt suicide or engage in deliberate self-harm [1]. to re-attempt suicide compared to adults who attempt Adolescence is a key period with respect to clinical depres- suicide [17]. Thus, it is important to recognize adolescent sion, as the incidence rates of depression rise dramatically suicidal behaviors at an early stage, take them seriously, and from the early to late adolescent years [2]. It is therefore implement strategies to prevent the rise in suicides during crucial that health care policies focus on preventive inter- adolescence. Studies among adults have indicated that pre- ventions that aim to reduce the incidence of depression ventive strategies aimed at suicide can indeed reduce and suicidal behaviors in early adolescence. We will test a suicidal behaviors [18]. Fewer studies have been carried out multimodal stepped-prevention program for 12–15 years in adolescents or young adults. Therefore, the aim of this old and, if effective, implement it in secondary schools in study will be to investigate the effectiveness of a multimodal the Netherlands. The program consists of four modules: stepped-prevention program for suicidal behaviors among early screening and detection, gatekeepers training, pro- adolescents, with the aim to implement the program once gram targeting stigma, and indicated depression prevention. proven effective. The current study will examine the overall effectiveness of The existing research has shown that the most common the multimodal stepped-prevention program for suicidal motive for suicide among adolescents is suffering from behaviors and depressive symptoms using a cluster- mental problems [19]. Most adult disorders have their randomized trial. Through intensive structural collabor- origin in adolescence, with more than three-quarter of dis- ation between municipality health services, schools, mental orders starting before the age of 24 [20]. In this context, health agencies, and national institutes, we will facilitate an depression appears to be a significant risk factor for sui- outstanding situation for the implementation of the multi- cidal behaviors in adolescents [21]. This close relationship modal stepped-prevention program. between depressive symptoms and suicidal behaviors is The problem we address is both widespread and severe. further established by the fact that the rise of suicidal In the Dutch population of 17 million, approximately five behaviors in adolescence coincides with increasing inci- people per day die from suicide [3]. This affects not only dence of major depression [22]. After all, the first onset of family and friends, but also society. Two studies (in Ireland suicidal ideation usually occurs during an episode of and Scotland) were conducted to estimate the burden of depressive disorder in adolescence [21]. In light of this, suicide on society, revealing the cost of €1.5 million per research suggests that treating depression will likely suicide [4, 5]. Suicide before the age of 15 is quite rare, but reduce suicidal behavior as well [23]. Thus, it is important suicide rates rise substantially during adolescence [2]. To to identify adolescents with subclinical depressive symp- illustrate, suicide is the single most important cause of toms. Depression is also associated with several negative death among 15 to 29 years old in the Netherlands [3]. It is outcomes. Adolescents with depressive symptoms are more important to note that suicidal behaviors (i.e., suicidal likely to smoke, binge eat [24, 25], and have school-related ideation, deliberate self-harm, and suicide attempts) are problems, such as low grades and high drop-out [26, 27]. often initiated at the age of 15 years; thus, well before a Once adolescents develop depressive symptoms, they are completed suicide [6]. Adolescents themselves recounted also at risk for depressive recurrences during adulthood that the onset of their suicidal behaviors was when they [28]. When a depressive episode initiates during a younger were 12–16 years old, indicating the need for initiating age, the prognosis is far worse than when the first prevention during early adolescence rather than late depressive episode initiates during adulthood [29]. This adolescence [7]. Therefore, suicidality should in most is especially disconcerting considering that depression cases be viewed as a process wherein initial suicidal behav- rates among adolescents have been rising in the past iors remain unnoticed until death by suicide [8, 9]. Suicidal years [30]. behaviors in young adolescents are sometimes mistaken In sum, integration of suicide and depression prevention for behaviors that early adolescents might outgrow [10]. is both a necessary and valuable approach for adolescents. However, approximately one third of adolescents with The most relevant setting to reach adolescents is the school, suicidal ideation will eventually attempt suicide, generally as school attendance is mandatory in the Netherlands until within one year [11]. Moreover, suicidal ideation in adoles- the age of 18. Few school-based prevention programs cence is strongly related to suicidal behaviors in adulthood actually address suicide prevention. Yet, there are some and is predictive of a range of other adverse outcomes school-based programs aimed at depression prevention [12–14]. To illustrate, a longitudinal study found that have been proven effective in Dutch samples [31, 32]. adolescents with suicidal ideation showed significantly Current approaches to depression and suicide prevention more psychopathology and recurrence of suicide attempts at schools comprise mostly singular interventions. Growing and lower perceived coping skills, self-esteem, and social evidence suggests that depression and suicide prevention connectedness [15]. Moreover, it appears that suicidal warrant the adoption of a multimodal approach to become Gijzen et al. BMC Psychiatry (2018) 18:124 Page 3 of 12

truly effective [33]. The lack of an integrated multi- personal experience with depression seems to be a crucial modal approach is also evident from the available inter- factor for changing stigmatizing attitudes [47]. This true vention options in the National Institute for Public not only for students not showing signs of depression or Health and the Environment (in Dutch: Rijksinstituut suicidal behaviors, but also for those with mental health voor Volksgezondheid en Milieu, RIVM) and Centre for problems (e.g., suicidal behaviors or depressive symptoms) Healthy Living (in Dutch: Centrum Gezond Leven, CGL) when they identify with others suffering from mental database as well as the healthy school database in the health problems [48]. Netherlands [34–36]. Although several programs focus on The identification of and subsequent intervention for the recognition of risk factors for suicide or for the preven- those with elevated signs of depressive symptomatology tion of depression, none of these options offer a comprehen- should also be important components of the prevention sive multimodal approach targeting both depression and of suicidal behaviors [21]. Cognitive behavioral therapy suicide prevention. Integrating different complementary (CBT) is an effective treatment for depression, and many types of prevention (i.e., universal and indicated approaches) indicated prevention programs are based on CBT, such that would include various stakeholders simultaneously (e.g., as ‘Op Volle Kracht (On Full Power: OVK). OVK was teachers, adolescents, parents, youth (mental) health service modeled after the Penn Resiliency Program (PRP) [49]. providers) and using various types of interventions The program successfully reduced depressive symptoms in (screening, education, universal, and indicated interven- adolescent girls with subclinical depressive symptoms [50]. tions) have been suggested as prevention strategies. A modified OVK version (OVK2.0) by de Jonge-Heesen, et The current study will examine the effectiveness of the al. [51] will be implemented and investigated at schools multimodal stepped-prevention program for suicidal behav- with adolescents who have been identified as scoring higher iors and depressive symptoms using a cluster randomized on depressive symptoms, as measured by self-report. trial. This includes a combination of preventive interven- The primary aim of our study will be to investigate the tions, such as (1) early screening and detection of suicidal effectiveness of a multimodal stepped-prevention program behaviors with subsequent clinical referral, (2) a safety net to reduce suicidal behaviors and depressive symptoms. We consisting of gatekeepers at school, (3) universal prevention hypothesize that suicidal behaviors will decline as a result focusing on stigma reduction, and (4) identification of of a multimodal preventive intervention relative to care as adolescents who have elevated signs of depression with sub- usual enhanced with screening and the gatekeepers’ safety sequent indicated prevention. Early detection is important, net. Since most modules of the stepped-prevention pro- as less than half of adolescents engaging in suicidal behav- gram are based on depression prevention program, we iors are known at mental health care services or by other expect that depressive symptoms will decline as well. gatekeepers (i.e., family, friends, teachers and mentors at Secondary aims are to investigate the mechanisms of school, etc.) prior to a suicide [37, 38]. A growing body of change by studying the effect mediators and moderators. evidence suggests that school-based screening adequately Previous research has identified several factors that contrib- identifies students at high-risk, effectively refers these ute to lower treatment response in depression and suicidal students to mental health care, and reduces the risk of behaviors. Socio-demographic factors, such as age, gender, suicide ideation and non-fatal suicidal behaviors [39, 40]. It cultural background/ethnicity, and educational background, is similarly important that a safety net is created at schools. have been found to moderate treatment outcome. There- Mentors should have the knowledge and skills to identify fore, their effects will be examined. Other factors that could adolescents who show signs of suicidal behaviors and know influence treatment outcome are perfectionism, hopeless- how to respond to those students [41]. Previous research ness, and baseline depressive symptoms and suicidal has shown that a gatekeeper training based on Question, behaviors. Furthermore, other factors that are related to Persuade, and Refer (QPR) model can increase knowledge the proposed prevention modules could mediate outcome, of suicide prevention and skills [42, 43]. Another important such as stigma, social connectedness, mastery, worry, and factor impeding identification of suicidal behaviors is the life events. We hypothesize that the factors that impede fact that help-seeking behaviors among youths is very low treatment response will have a similar effect on the out- [44]. Nevertheless, research has shown that help-seeking comes of the multimodal stepped prevention program. behaviors predicts better prognosis [45]. Stigma has been identified as an important factor that impedes help-seeking Methods among youth. Thus, it is important to develop a universal The study methods and results will be reported in accord- strategy aimed at reducing stigma. ance with the CONSORT 2010 statement for reporting Stigma literature suggests that mental health literacy parallel group randomized trials [52] and the CONSORT combined with information related to individuals’ personal 2010 statement: extension to cluster randomized trials experience is more likely to produce a change in attitude [53]. The medical research ethics committee CMO Region and stigma [46]. Moreover, contact with someone who has Arnhem-NijmegeninTheNetherlandsapprovedthis Gijzen et al. BMC Psychiatry (2018) 18:124 Page 4 of 12

study (NL61599.091.17). The study is registered in the in Dutch: HAVO/VWO) because school type is a relevant Dutch Trial Register (NTR6622). prognostic predictor of outcome [54]. Like most psychological and public health interventions, blinding of partici- Design pants is not possible; therefore, it will not be attempted. The presented study is designed as a non-blinded cluster- The assessments will be conducted at baseline (T0), during randomized controlled trial with two parallel groups the intervention phase after the third module (T1), at post- (experimental and control) to evaluate the effectiveness intervention (T2), at 6-month follow-up (T3), at 12-month of a multimodal stepped-prevention program relative to follow-up (T4), and at 24-month follow-up (T5). The (enhanced) usual care. The participants in the experi- overall study design is shown in Fig. 1. mental condition will receive all four modules of the multimodal stepped-prevention program, whereas partici- Participants’ eligibility pants in the control condition will receive modules 1 and 2. Adolescents in their second year of secondary education will Randomization will be conducted at school level (to be be eligible to participate in the study. Inclusion criteria are more precise: at the level of school location because one aged between 11 and 15 years and sufficient knowledge of school can have annexes at multiple locations) to avoid the Dutch language. Exclusion criteria are absence of (par- contamination. An independent statistician will randomly ental) permission and for the indicated module (OVK2.0) assign the participating school locations to the intervention already receiving health care for mood-related problems. or control condition using random.org. Furthermore, randomization will be stratified for (1) education level Recruitment (vocational training as one category, in Dutch: VMBO; Following the randomization at the school level, as men- and higher education / pre-university as the other category, tioned above, students in their second year of secondary

Fig. 1 Schematic Overview of the Study Design Gijzen et al. BMC Psychiatry (2018) 18:124 Page 5 of 12

school (i.e., 8th grade in the US), from vocational training Program modules up to pre-university level, will receive written information The multimodal stepped-prevention program comprises about the screening and the study. Written informed con- screening with subsequent clinical evaluation and/or referral; sent from adolescents and their parents will be obtained gatekeeper training (QPR) for mentors; universal prevention prior to the initiation of the study. focusing on stigma reduction; and identifying adolescents who have elevated signs of the most important risk factor for suicidal behaviors, i.e., depression with subsequent Sample size indicated prevention for screen-positive adolescents. We powered our study to detect a standardized mean These integrated four modules will be compared to the difference d = 0.38 (or larger) on the central clinical out- control condition. Schools in the control condition will come (suicidal behaviors) using a test for independent adhere to the usual curricula and students in these means at α = 0.05 (2-tailed) and power of (1-b) = 0.80 while schools will have full access to usual care offered by the considering the design effect of the cluster randomization regional health services, e.g., the Municipal Health Services with schools as the unit of randomization and students (GGD), Primary Care, and (specialized) Mental Health being nested within schools. The relevant parameters are Care services. In the context of this trial, we will prioritize (1) the intraclass correlation coefficient, rho; the mean the wellbeing and safety of participating students, espe- cluster size, m; the coefficient of variation, cv, of the cluster cially those at elevated risk for suicidal behaviors and/or sizes; and the minimally detectable effect size, d.Regarding depressive disorder. Therefore, the screening and gate- the last parameter, we looked at the well-known SEYLE keeper intervention will also be available to the control study [55], where 2764 pupils in 43 schools were random- group. Consequently, the comparison condition will ized to a universal prevention consisting of a Youth Aware not exactly be “care as usual”; instead, it is described as of Mental Health (YAM) program (comparable to our “enhanced usual care” to guarantee the wellbeing and universal intervention) and compared to a control group safety of participating students. of 2933 students in 40 schools. YAM was associated with a significant reduction in incident suicide attempts I. Screening (OR = 0.45, p = 0.014) and a reduction in severe suicidal All students in their second year of secondary school (i.e., ideation (OR = 0.50, p = 0.025). In our study, we expect 8th grade in US), from vocational up to pre-university similar or better effects because in addition to the uni- levels, will be screened for suicidal behaviors and depres- versal preventive module, we will also offer the safety sive symptoms using the Questionnaire assessing Suicide net provided by the gatekeepers training in addition to and Self Injury (in Dutch: Vragenlijst over Zelfdoding en early detection and referral for adolescent students at Zelfbeschadiging; VOZZ) [59] and the Childhood Depres- elevated risk of suicide. For screen-positive students, an sion Inventory 2 (CDI-2) [60]. The screening will be part additional indicated preventive module will be offered. of a larger health survey conducted by the health services Nonetheless, we conservatively assume that the effect- of the school (in Dutch: GGD). Adolescents identified at iveness of our integrated multi-component intervention risk for suicide are seen within 48 h by the health service will be of equal effectiveness as the YAM program. The of school (in Dutch: GGD). Parents of children identified OR of 0.50 translates into a standardized mean difference, as at risk for suicide by the mental health service of school d, of 0.38 when the outcome is measured on a continuous will be informed. In accordance with parents, the adoles- scale, as we will do [56]. The mean cluster size of m =215 cents will be referred to specialized mental health care, if will be based on student population in the previous year. deemed necessary by the health service of school, and In the last year, the smallest school had 74 students and will be excluded from the indicated intervention module. the largest 340. Accordingly, it follows that coefficient of Adolescents who have been referred to specialized mental variation cv =0.31[57]. In the SEYLE study of Wasserman health care will also be approached to complete the same et al. [55], the intraclass correlation coefficient was esti- set of questionnaires as the children in the experimental mated at rho = 0.01 for severe suicidal ideation. To obtain and control condition to examine the effect of the screen- an estimate of the required sample size, we used Stata’s ing and to monitor their mental health. procedure to estimate sample size needed for an independent groups t-test in a cluster randomized trial [58]. II. Gatekeeper training The corresponding syntax of clustersampsi, samplesize All mentors of participating adolescents will be used as mu1(0.00) mu2(0.38) sd1(1) sd2(1) rho(0.01) m(215) size_ gatekeepers and trained to recognize suicidal behaviors, cv(0.31) indicated n = 645 per condition, or n =1290 in learn to initiate conversations after recognizing suicidal both trial arms. We will also compensate for a maximum behaviors in a student, and learn to refer a student of 30% dropout, which implies that we will need to have effectively and correctly, if deemed necessary. This 1290/(1–0.30) =1844 participants at baseline. training will be based on the Question, Persuade, and Gijzen et al. BMC Psychiatry (2018) 18:124 Page 6 of 12

Refer (QPR) gatekeeper training [61]. Gatekeepers who Table 1 Overview of assessments will complete the QPR will show increased knowledge Screening T0 T1 T2 T3 T4 T5 of suicide prevention, self-reported skills, competencies, Adolescent and efficacy [42, 43]. Further information on the content, Suicide risk (VOZZ-Screen) X background, and structure of the online program is pro- Suicidal behaviors (VOZZ) XXXXXX vided through an open-access protocol paper [62]. Depressive symptoms (CDI-2) X XXXXXX III. Universal prevention Clinical depression (ADIS-C) X The third module that will be evaluated is ‘Moving Stories,’ Stigma (DSS) XXXXXX which will be offered to all participating students in the Social connectedness XXXXXX ‘ ’ experimental condition. The goal of Moving Stories is to Mastery (PMS) XXXXXX increase students’ mental health literacy (i.e., knowledge Worry (PSWQ) XXXXXX of depressive symptoms, effective strategies for dealing with depressive symptoms, and help-seeking strategies) Perfectionism (MPS) X and decrease depression stigma. It consists of an introduc- Life events (TP) XXXXXX tion presented by a researcher and the students’ school School mentor, a serious game, and a debriefing by either an Academic grades X XXXXXX experiential expert only or a mental health professional Drop-out rates X XXXXXX in combination with a filmed experiential expert trained Non-attendance X XXXXXX in the program with support from the students’ school mentor and a researcher at school. The game consists Truancy X XXXXXX of five sessions, approximately 10–15 min per day. In a virtual house, adolescents will be asked to discover useful Screening strategies to help a girl, Lisa, who is showing signs of To assess suicide risk, adolescents will be screened using depression. They can complete five actions each day and the VOZZ [59] and item 8 of the CDI-2 [60]. The earn points with these actions. More points represent VOZZ-Screen is a self-report questionnaire comprising more trust between Lisa and the player, and this enables 10 items. This questionnaire contains ten questions the players to call in help from an adult. For example, assessing thoughts and actions about life, self-harm, students can make drinks or food, call her parents, or suicide, and suicidal ideations in the past 7 days. Items clean the house, among other things. To identify useful assessing the participant’s life are rated on a 5-point strategies, they can talk to the girl in the house. Students scale from 1 (I totally agree) to 5 (I totally disagree) (e.g., will receive feedback on their actions during the day. ‘I feel worthless’). Items about self-harm and suicide are rated on a 5-point scale from 1 (never) to 5 (very often) IV. Indicated prevention (e.g., ‘I have harmed myself deliberately’). Items about The fourth module, namely the indicated module, is suicidal ideation in the past 7 days are rated on a 5-point ‘OVK2.0,’ which will only be offered to screen-positive scale from 1 (never) to 5 (every day) (e.g., ‘I thought that students (CDI-2 ≥ 14) in the experimental condition. It suicide would be a solution for my problems’). A score of consists of 8 lessons of 60 min each. The intervention is ≥23 requires subsequent action in the form of a personal based on the principles of cognitive behavioral therapy conversation to assess acute suicide risk. The CDI-2 is also (CBT) and is a modified version of the OVK-program that a self-report questionnaire comprising 28 items assessing was based on the PRP. It only includes the lessons that are depressive symptoms, each consisting of three statements based on the CBT techniques, as they were considered rated in severity from 0 to 2 (e.g., ‘Idon’tfeelalone’ =0‘I most effective [50]. Details about the content of the often feel alone’ =1,‘I always feel alone’ = 2). Item 8 of the program are described extensively elsewhere [49]. It will be CDI-2 measures the presence of suicidal ideation on a delivered by two trainers, a licensed psychologist who is a three-point scale (0 = I don’tthinkaboutendingmylife, school staff member and a co-trainer who is a licensed 1 = I think about ending my life, but I would never do member of a (mental) health institution. The trainers it, 2 = I want to end my life). The CDI-2 will be used underwent an extensive 3-day training program in the for screening purposes in accordance with the Dutch necessary skills, such as CBT and its theoretical background, clinical guidelines for depression among youth [63]. a training manual, and the intervention protocol. Primary outcome measure Study outcome measures Suicidal behaviors will be measured using the full VOZZ For an elaborate overview of study outcome measures, questionnaire [59]. This questionnaire contains 39 questions see Table 1. assessing thoughts and actions about life, self-harm, suicide, Gijzen et al. BMC Psychiatry (2018) 18:124 Page 7 of 12

and suicidal ideations in the past 7 days. It is a combination a scale of 0 (“never”)to3(“always”). Higher scores indi- of the VOZZ-Screen and 29 additional items. Scoring is cate higher levels of worry. the same as described earlier for the VOZZ-Screen. A Perfectionism will be assessed using the Frost Multidi- score of 86 or above indicates high risk of suicide. The mensional Perfectionism Scale (FMPS) [71]. It consists reliability was high in an adolescent sample (Cronbach’s of 35 items and has good internal reliability (Cronbach’s α = 0.91; r = 0.82) [59]. α = 0.90) [71]. Each item is rated on a scale of 1 (not true at all) to 5 (completely true), with a higher score indicating Secondary outcome measures more levels of perfectionism. It measures six subscales, Depressive symptoms in children and adolescents will namely concern over mistakes, personal standards, parent also be measured with the CDI-2 [60], as described in expectations, parental criticism, doubting of actions and previous section. organization. A higher score indicates greater levels of Clinical depression will be measured by the Anxiety perfectionism. Disorder Interview Schedule for Children (ADIS-C; [54]) Life events will be recorded and assessed using The during a clinical interview. It is a semi-structured diag- Top Problems (TP) [72] measure. It assesses life events nostic interview that is used to diagnose anxiety and that participants consider the most important at that comorbidity. All interviews will be administered by a time. Participants are asked to list three problems about qualified psychologist. which they are most concerned. Furthermore, they are Stigma will be measured using the Depression Stigma also asked to rate the severity of all three problems Scale (DSS) [64]. It measures personal depression stigma separately on a scale from 0 (not at all) to 10 (very, very and perceived depression stigma. Both scales have a good much). As such, it not only measures actual life events, internal consistency (Perceived Scale: Cronbach’s α =0.82; but also accounts for personal salience of life events, as Personal Scale: Cronbach’s α =0.78) [64]. The Personal opposed to most life events measures. Stigma Subscale measures stigma in the respondents’ Hopelessness will be assessed using the VOZZ. Previous own attitudes towards depression. The Perceived Stigma studies have found that correlations between the hopeless- Subscale measures the respondent’s perception about the ness scale by Beck and the VOZZ were 0.79 and 0.58 [59]. attitudes of others towards depression. Both subscales The items 3, 4, 21, 28, and 31 of the VOZZ, which were consist of 6 items and responses to each item are measured established in accordance with the first author of the on a five-point scale (ranging from 0 ‘strongly disagree’ to VOZZ, will be used to assess hopelessness. 4 ‘strongly agree’). Higher scores indicate higher levels of School-related factors, such as academic grades, drop- depression stigma. outs, non-attendance, and truancy will be obtained in Social connectedness will be measured using a single collaboration with the schools. item construct. Research has shown that a single item measure of social identification is reliable [65]. As the Analysis construct has not yet been used in this age group, we The targeted clinical outcome (VOZZ suicidal behaviors) have included three extra items for (potential) increased will be evaluated in agreement with the intention-to-treat reliability of the construct. Social connectedness will be principle using linear mixed modeling with VOZZ at base- measured for the class, the ‘OVK’-group, and school. Items line as covariate. Reporting of the results will be conducted are rated on a scale of 1 (“strongly disagree”)to7(“strongly in accordance with the CONSORT statement [52]. agree”). Mastery will be measured with the Pearlin Mastery Evaluation of secondary outcome Scale (PMS; [66]), which consists of 7 items. It measures The intervention’s effect on the secondary outcome perceived control of one’s life. Each item has the following (CDI-2 depression) will be investigated in the same way response options: (1) Strongly Disagree (2) Disagree (3) as the primary outcome (VOZZ suicidal behaviors); Agree (4) Strongly Agree. It is a widely used measurement hence, using linear mixed modeling with baseline CDI-2 of mastery, with a higher score indicating higher mastery. depression as a covariate. Previous studies have found good reliability of the PMS (Cronbach’s α =0.78)[67]. Analysis of effect mediation Worry will be assessed with the Penn State Worry It is important to determine the mediators that affect Questionnaire for children (PSWQ-C; [68]). It is widely the intervention effect. This helps determine whether used in both research and clinical practice to reliably the intervention works through expected mechanisms assess worry in both clinical and non-clinical samples and which improvements could be effective. Mediation and adolescents [69]. The Dutch version has also been analyses will be performed in Mplus [73], where indirect shown to be reliable for assessing worry in children [70]. effects will be tested with bootstrap methods. More The PSWQ-C consists of 14 items. Each item is rated on specifically, we will test whether (1) mastery, (2) stigma, Gijzen et al. BMC Psychiatry (2018) 18:124 Page 8 of 12

(3) worry, (4) top problems, (5) social connectedness, and worrying (i.e., repetitive thinking) as a proximal risk factor (6) hopelessness will mediate the intervention effect on of suicidal behaviors. Interestingly, mastery has been asso- VOZZ suicidal behaviors and / or CDI-2 depression. ciated with negative effect of repetitive thinking [93], which Increased awareness through stigma reduction will be may also be an important mediator, as mentioned before. the main intervention target of the universal prevention. Another interesting finding is that hopelessness may par- We hypothesize that reduction in stigma will also reduce tially mediate the relationship between repetitive thinking suicidal behaviors and depressive symptomatology and and suicidal ideation [94]. Hopelessness has also often been mediate the effect of the intervention. Moreover, stigma is named as one of the most important predictors of suicide also related to social isolation, which is another important attempts and behaviors [95–98]. Moreover, it is considered risk factor for suicidal behaviors [74]. Research has found to mediate the relationship between depression and suicidal that people often withdraw from social life due to fear of behaviors [95]. Perfectionism is often related to the feelings rejection as a result of stigmatizing attitudes towards of hopelessness [99]. Just like those with lower levels of mental health problems [75, 76]. Hence, reduced stigma is mastery, a person with high feelings of perfectionism may likely to reduce social isolation. Previous research has also be more likely to turn to suicidal ideation in line of negative shown that students who have negative attitudes towards events or feelings of hopelessness [100, 101]. As opposed school and thus feel less connected to the school are at an to mastery, perfectionism is traditionally viewed as a more increased risk for suicide attempts [77]. School-based pro- stable trait [102]. Thus, we hypothesize that perfectionism grams are expected to induce attitude changes in school may moderate rather than mediate the treatment outcome. staff and students. Moreover, group-based interventions In addition, it is also important to record life events, as they could reduce the feeling among adolescents that they are can increase the risk of depressive symptomatology [103]. the only ones experiencing depression, which is often Negative life events are also associated with the occurrence associated with adolescent depression, as youngsters often of suicidal ideation [97]. However, the most important do not share depressive feelings with each other due to variable is the experienced disruptiveness of life events. fear of stigmatization. Furthermore, social isolation can As such, recurrences in depression are often associated also reduce treatment response [78]. In line with this, with negative life events due to kindling and sensitization studies have shown that each social group that a depressed [104–106]. We expect that the multimodal stepped- individual joins decreases relapse rates in depressed indi- prevention program will also reduce the disruptiveness of viduals and results in a more pronounced improvement life events and in turn decrease occurrence of depressive on depressive complaints [79]. Thus, social connectedness symptoms. is an interesting factor to examine as a potential moderator considering both depression and suicide prevention. Explorative analysis of effect moderation Both the interpersonal theory [80] and the integrated- Finally, moderation analyses will be conducted to determine motivational-volitional model [81]theorizethatreduced which adolescents benefit most from the intervention. social connectedness increases suicidal ideation through Additionally, it will help determine whether some adoles- the sense of thwarted belongingness. Christensen [82] cents would be better served by receiving adapted or other found that mastery is also related to the sense of thwarted interventions. We will conduct a series of a priori planned belongingness and in accordance with previous research, moderator analyses to see whether the intervention effect the researcher has found that lower levels of mastery are is moderated (increased or diminished) by the following significantly associated with suicide ideation [83]. In moderators: (1) gender, (2) ethnic descent / cultural back- theory, mastery ensures that people have the ability to ground, (3) level of baseline perfectionism, (4) level of manage negative experiences [84]. Those who feel lack baseline VOZZ suicidal behaviors, and (5) level of baseline of control (i.e., mastery) over situations are more likely CDI-2 depression. We call these analyses explorative to turn to suicidal ideation, much like a self-fulfilling because the data-analytical strategies will be non-parametric prophecy. Moreover, suicidal ideation may give people a in nature, such as bootstrap-aggregated CART analysis and false sense of mastery when they feel they lack mastery random forest methods implemented in the R statistical naturally [85]. package. In addition to social connectedness and mastery, Several socio-demographic factors have been found repetitive thinking is also suggested as a mechanism to affect suicide and depression prevention. Previous affecting depression and suicidal ideation [86]. Research research has found that gender may play an important has found that repetitive thinking predicts not only role in CBT-based programs. Age has also been found to presence of depression and suicidal ideation, but also influence treatment outcome in depression prevention their duration [87, 88]. Furthermore, repetitive thinking studies. Girls and older participants were found to experi- decelerates the recovery after the treatment [89–91]. In ence more beneficial effects of interventions. It is also line with this, Kerkhof and van Spijker [92] identified important to consider ethnic and cultural background. It Gijzen et al. BMC Psychiatry (2018) 18:124 Page 9 of 12

has been well established that ethnicity influences help- symptoms, baseline level of suicidal behaviors, perfec- seeking behaviors and the ways in which suicidal behaviors tionism, gender, mastery, and ethnic descent / cultural or depressive symptoms are expressed [107]. Hence, background. ethnicity might in turn also influence the perception and integration of treatment. Strengths and limitations One of the strengths of this study is that it will include Other study parameters the follow-up assessment of 24 months, providing the Possible baseline imbalances between the two conditions opportunity to evaluate the longer-term effects. Second, in demographic variables, VOZZ suicidal behaviors, and the multimodal stepped-prevention program will be CDI-2 depression will be verified. If any variables show implemented in all secondary schools in a rural region in different distributions across the two conditions, they The Netherlands, with a strong collaboration between will be entered as covariates in all models testing the schools’ and (mental) health organizations. A meta-analysis effectiveness of the intervention. of Brunwasser and Garber [108] on the effectiveness of programs for the prevention of youth depression revealed Interim analysis the need to conduct studies in real-life conditions, like the A planned interim analysis will be conducted to assess one we are proposing. Third, the current study uses a whether one of the trial’s conditions (either intervention between-schools design which minimizes contamination or control) is associated with a significantly higher risk of effects that might occur in a within-schools design. An completed suicides. An independent statistician, blinded to additional strength of the study is that in contrast to the treatment allocation, will carry out the interim analysis at most RCT studies, we will focus not only on the effective- the post measurement. We chose the post measurement ness of the program, but also on the mediators of change as the time point for the interim analysis because it is the (i.e., how the intervention works) and on the characteris- earliest stage in the trial to test whether one of the condi- tics of the student population that may act moderators. tions is associated with a significantly greater number of This will shed light on how the intervention works and for completed suicides. The interim analysis may result in whom it is effective. changes in the study’s protocol or the study might even Several limitations of this study must be noted. As we end due to overwhelming evidence of group difference. evaluate the effectiveness of the multimodal stepped-pre- The interim analyses will be conducted for completed vention program, no conclusions regarding the specific suicides at the primary efficacy end point of the study components of the program can be made. Additionally, obtained from patients in the target population. The we will not complete a clinical interview prior to the inter- appropriate analysis of such count-data (non-negative ventions (only at post measurement). Thus, it is possible integers) is best done with Poisson regression, which is that some adolescents meeting the diagnostic criteria of a also appropriate for analyzing rare events. The statistical full-fledged depression may be included in the study. For analyses will be carried out by setting the alpha-level to those, the intervention becomes the treatment rather than 0.05 for a two-tailed test. The Type I error boundaries the prevention. However, since sub-threshold depression for statistical significance do not need to be adjusted or minor depression is sometimes viewed as a clinical for multiple comparisons because the interim analysis disorder(asisthecaseintheDSM-5),thecurrent will be conducted at a single time point. study might not be classified as the prevention of the imminent onset of a new disorder, but rather as the Discussion treatment of an existing disorder. Additionally, the study The present study protocol describes a RCT on the effect will be conducted in a specific region in The Netherlands, of a multimodal school-based prevention program on which may limit the generalizability of the results to other suicidal behaviors in adolescents. The primary aim is to regions in The Netherlands. investigate whether the multimodal school-based program results in a clinically significant reduction of suicidal Implications for practice behaviors and depressive symptom levels in secondary If the multimodal stepped-prevention program proves to school students compared to ‘enhanced’ care as usual. The be effective in reducing suicidal behaviors and preventing secondary aim is the evaluate pathways that transmit the depressive symptoms in adolescents, then this will call for intervention effect on the primary outcome: (a change in) a broader implementation of school-based suicide and stigma, social connectedness, mastery (i.e., level of internal depression prevention. Moreover, considering all the locus of control), worry, hopelessness, and number of self- stakeholders involved in this preventive program, their reported major problems. Another secondary aim is to strong collaboration could benefit the region and serve identify modifying factors that increase or decrease the as an example for other regions in Europe of how to intervention’s effectiveness: baseline level op depressive organize suicide and depression prevention for youth. Gijzen et al. BMC Psychiatry (2018) 18:124 Page 10 of 12

Abbreviations Received: 22 November 2017 Accepted: 1 May 2018 ADIS-C: Anxiety disorders interview schedule for children; CBT: Cognitive behavioral therapy; CDI-2: Children’s depression inventory 2; CMO: Commissie Mensgebonden Onderzoek (in English: Committee on Research Involving Human Subjects); DSS: Depression Stigma Scale; References FMPS: Frost Multidimensional Perfectionism Scale; OVK: Op Volle Kracht (in 1. Dijkstra M. Factsheet preventie van suïcidaliteit [fact sheet]. http://www. English: On Full Power); PI: Principal investigator; PMS: Pearlin Mastery Scale; trimbos.nl/webwinkel/productoverzicht-webwinkel/preventie/af/af0933- PRP: Penn Resiliency Program; PSWQ-C: Penn State Worry Questionnaire for factsheetpreventie-van-suicide. 2010. Children; QPR: Question, Persuade and Refer; RCT: Randomized controlled 2. Chaplin TM, Gillham JE, Reivich K, Elkon AG, Samuels B, Freres DR, et al. trial; TP: Top Problems; VOZZ: Vragenlijst over Zelfdoding en Depression prevention for early adolescent girls: a pilot study of all girls – Zelfbeschadiging (in English: Questionnaire assessing Suicide and Self Injury); versus co-ed groups. J Early Adolescence. 2006;26(1):110 26. ’ YAM: Youth Aware of Mental Health 3. Gijzen S, Boere-Boonekamp MM, L hoir MP, Need A. Child mortality in the Netherlands in the past decades: an overview of external causes and the – Acknowledgements role of public health policy. J Public Health Policy. 2014;35(1):43 59. – We would like to acknowledge Rian van den Boogaart (project manager at 4. Kennelly B. The economic cost of suicide in Ireland. Crisis. 2007;28(2):89 94. GGZ Oost Brabant) for her contribution to design and practicability of the 5. McDaid D, Kennelly B. An economic perspective on suicide across the five – study, and Anouk Tuijnman (phd student at Radboud University) and continents. Oxford: Oxford University Press; 2009; 359 368. IJsfontein for their contribution in developing Moving Stories. We are also 6. Runeson BS, Beskow J, Waern M. The suicidal process in suicides among – grateful to the collaborating schools (Alfrinkcollege, Carrolus Borromeus young people. Acta Psychiatr Scand. 1996;93(1):35 42. College, Commanderijcollege, Dr. Knippenbergcollege, Jan van Brabant 7. Glenn CR, Lanzillo EC, Esposito EC, Santee AC, Nock MK, Auerbach RP. College, Hub van Doorne, Peellandcollege, St. Willibrord Gymnasium, Examining the course of suicidal and nonsuicidal self-injurious thoughts and Strabrecht College, Vakcollege Helmond, and Varendonck College), the behaviors in outpatient and inpatient adolescents. J Abnorm Child Psychol. – health professionals of the Municipal Health Services “Brabant-Zuidoost”, 2017;45(5):971 83. – mental health professionals of GGZ Oost Brabant, Marianne van Bakel for 8. Van Heeringen C. Suicide in adolescents. Int Clin Psychopharmacol. 2001;16:S1 6. training the experiential experts and the experiential experts we trained for 9. Retterstøl N. Suicide: a European perspective. Cambridge: Cambridge making this research possible. University Press; 1993. 10. Lieberman EJ. Suicidal ideation and young adults. Am J Psychiatr. 1993; Funding 150(1):171. Funding for this study was provided by the municipalities of Asten, Deurne, 11. Nock MK, Green JG, Hwang I, McLaughlin KA, Sampson NA, Zaslavsky AM, Geldrop-Mierlo, Gemert-Bakel, Helmond, Laarbeek and Someren, The et al. Prevalence, correlates, and treatment of lifetime suicidal behavior Netherlands. Moving Stories was funded by ‘Het Stimuleringsfonds,’ and OVK among adolescents: results from the National Comorbidity Survey – was funded by ZonMw. GGZ Oost Brabant and the Trimbos Institute will provide Replication Adolescent Supplement. JAMA Psychiatry. 2013;70(3):300 10. — program materials. 12. Ahrens B, Linden M, Zäske H, Berzewski H. Suicidal behavior symptom or disorder? Compr Psychiatry. 2000;41(2):116–21. Authors’ contributions 13. Groleger U, Tomori M, Kocmur M. Suicidal ideation in adolescence-an MG is responsible for data collection, data analysis, and for reporting the Indicator of actual risk? Isr J Psychiatry Relat Sci. 2003;40(3):202. study results. FS will be involved in the data analysis. SR, FS, DC, and RE read 14. Goldney RD, Smith S, Winefield A, Tiggeman M, Winefield H. Suicidal the manuscript and provided suggestions for improvement. SR, DC, FS and ideation: its enduring nature and associated morbidity. Acta Psychiatr – RE are also supervisors and grant applicants. All authors have read and Scand. 1991;83(2):115 20. approved the final manuscript. 15. Reinherz HZ, Tanner JL, Berger SR, Beardslee WR, Fitzmaurice GM. Adolescent suicidal ideation as predictive of psychopathology, suicidal Ethics approval and consent to participate behavior, and compromised functioning at age 30. Am J Psychiatr. 2006; – The medical ethics committee CMO Region Arnhem-Nijmegen in The 163(7):1226 32. Netherlands approved this study (NL61599.091.17). Written informed consent 16. Kessler RC, Borges G, Walters EE. Prevalence of and risk factors for lifetime from adolescents and parents will be obtained. suicide attempts in the National Comorbidity Survey. Arch Gen Psychiatry. 1999;56(7):617–26. Competing interests 17. Steele MM, Doey T. Suicidal behaviour in children and adolescents. Part 1: Trimbos Institute, Utrecht, has the exploitation rights of the Moving Stories etiology and risk factors. Can J Psychiatr. 2007;52(6):21S. and the ‘OVK2.0’ intervention. Trimbos Institute is a not-for-profit WHO 18. Mann JJ, Apter A, Bertolote J, Beautrais A, Currier D, Haas A, et al. Suicide – Collaborative Centre with the goals to disseminate best and evidence-based prevention strategies: a systematic review. JAMA. 2005;294(16):2064 74. practices. Trimbos Institute may licence third parties to use the Moving Stories 19. Spirito A, Esposito-Smythers C. Attempted and completed suicide in – intervention and OVK2.0 within routine preventive services. FS, and MG are em- adolescence. Annu Rev Clin Psychol. 2006;2:237 66. ployees at Trimbos Institute, but will not have a share in any possible licence 20. Kessler RC, Berglund P, Demler O, Jin R, Merikangas KR, Walters EE. Lifetime revenues. prevalence and age-of-onset distributions of DSM-IV disorders in the National Comorbidity Survey Replication. Arch Gen Psychiatry. 2005;62(6): 593–602. Publisher’sNote 21. Lewinsohn PM, Rohde P, Seeley JR. Adolescent suicidal ideation and Springer Nature remains neutral with regard to jurisdictional claims in attempts: prevalence, risk factors, and clinical implications. Clin Psychol Sci published maps and institutional affiliations. Pract. 1996;3(1):25–46. 22. Avenevoli S, Swendsen J, He J-P, Burstein M, Merikangas KR. Major Author details depression in the National Comorbidity Survey–Adolescent Supplement: 1Trimbos Institute (Netherlands Institute of Mental Health and Addiction), prevalence, correlates, and treatment. J Am Acad Child Adolesc Psychiatry. P.O. Box 725, 3500 AS Utrecht, The Netherlands. 2Erasmus School of Social 2015;54(1):37–44. e2. and Behavioural Sciences, Erasmus University, P.O. Box 1738, 3000 DR 23. Wilkinson P, Kelvin R, Roberts C, Dubicka B, Goodyer I. Clinical and Rotterdam, The Netherlands. 3GGZ Oost Brabant, P.O. Box 3, 5427 ZG Boekel, psychosocial predictors of suicide attempts and nonsuicidal self-injury in The Netherlands. 4Behavioral Science Institute, Radboud University Nijmegen, the adolescent depression antidepressants and psychotherapy trial (ADAPT). P.O. Box 9104, 6500 HE Nijmegen, The Netherlands. 5Child and Adolescent Am J Psychiatr. 2011;168(5):495–501. Studies, Utrecht University, P.O. Box 80140, 3508 TC Utrecht, The Netherlands. 24. Keenan-Miller D, Hammen CL, Brennan PA. Health outcomes related to early 6Department of Clinical, Neuro and Developmental Psychology and adolescent depression. J Adolesc Health. 2007;41(3):256–62. Department of Epidemiology and Biostatistics, Amsterdam Public Health 25. Hasler G, Lissek S, Ajdacic V, Milos G, Gamma A, Eich D, et al. Major research institute, VU University Medical Center, PO Box 7057, 1007 MB depression predicts an increase in long-term body weight variability in Amsterdam, The Netherlands. young adults. Obesity. 2005;13(11):1991–8. Gijzen et al. BMC Psychiatry (2018) 18:124 Page 11 of 12

26. Lewinsohn PM, Rohde P, Seeley JR. Major depressive disorder in older program (‘Op Volle Kracht’) among adolescent girls with elevated adolescents: prevalence, risk factors, and clinical implications. Clin Psychol depressive symptoms. J Abnorm Child Psychol. 2014;42(2):217. Rev. 1998;18(7):765–94. 51. de Jonge-Heesen KW, van Ettekoven KM, Rasing SP, Oprins-van Liempd FH, 27. Fletcher JM. Adolescent depression: diagnosis, treatment, and educational Vermulst AA, Engels RC, et al. Evaluation of a school-based depression attainment. Health Econ. 2008;17(11):1215–35. prevention program among adolescents with elevated depressive 28. Lewinsohn PM, Rohde P, Klein DN, Seeley JR. Natural course of adolescent symptoms: study protocol of a randomized controlled trial. BMC Psychiatry. major depressive disorder: I. Continuity into young adulthood. J Am Acad 2016;16(1):402. Child Adolesc Psychiatry. 1999;38(1):56–63. 52. Schulz KF, Altman DG, Moher D. CONSORT 2010 statement: updated guidelines 29. Thapar A, Collishaw S, Pine DS, Thapar AK. Depression in adolescence. for reporting parallel group randomised trials. BMC Med. 2010;8(1):18. Lancet. 2012;379(9820):1056–67. 53. Campbell MK, Piaggio G, Elbourne DR, Altman DG. Consort 2010 statement: 30. Lewinsohn PM, Rohde P, Seeley JR, Klein DN, Gotlib IH. Natural course of extension to cluster randomised trials. BMJ. 2012;345:e5661. adolescent major depressive disorder in a community sample: predictors of 54. Siebelink B, Treffers D. Anxiety disorders interview schedule for DSM-IV child recurrence in young adults. Am J Psychiatr. 2000;157(10):1584–91. version, Kindinterview [anxiety disorders interview schedule for DSM-IV child 31. Calear AL, Christensen H. Systematic review of school-based prevention and version, child interview schedule]. Lisse: Swets & Zeitlinger BV; 2001. early intervention programs for depression. J Adolesc. 2010;33(3):429–38. 55. Wasserman D, Hoven CW, Wasserman C, Wall M, Eisenberg R, Hadlaczky G, 32. Stice E, Shaw H, Bohon C, Marti CN, Rohde P. A meta-analytic review of et al. School-based suicide prevention programmes: the SEYLE cluster- depression prevention programs for children and adolescents: factors that randomised, controlled trial. Lancet. 2015;385(9977):1536–44. predict magnitude of intervention effects. Journal of consulting and clinical 56. Chinn S. A simple method for converting an odds ratio to effect size for use psychology. 2009; 77(3):486. in meta-analysis. Stat Med. 2000;19(22):3127–31. 33. Weare K. Child and adolescent mental health in schools. Child Adolesc 57. Eldridge SM, Ashby D, Kerry S. Sample size for cluster randomized trials: Mental Health. 2013;18(3):129–30. effect of coefficient of variation of cluster size and analysis method. Int J 34. Rijksinstituut voor Volksgezondheid en Milieu. De Gezonde School, http:// Epidemiol. 2006;35(5):1292–300. www.rivm.nl/Onderwerpen/G/Gezonde_School. Accessed June 20 2017. 58. Hemming K, Girling AJ, Sitch AJ, Marsh J, Lilford RJ. Sample size calculations 35. Centrum Gezond Leven. Interventies zoeken, https://www. for cluster randomised controlled trials with a fixed number of clusters. BMC loketgezondleven.nl/leefstijlinterventies/interventies-zoeken?trefwoord= Med Res Methodol. 2011;11(1):102. &option=any&sort=evaluation&approved=1&review=&theme=A09a_ 59. Kerkhof A, Huisman A, Vos C, Smits N. Handleiding VOZZ & VOZZ screen: DEPRESSIE&setting=A10_ONDWVO&gender=&type=A04_GPREV%20A04_ Vragenlijst over Zelfdoding. Amsterdam: Amsterdam VU; 2015. SPREV%20A04_UPREV%20A04_ZPREV&age=12–17&target=&methodology= 60. Bodden D, Braet C, Stikkelbroek Y. CDI-2: Screeningsvragenlijst voor &materials=&organisation=. 2017. Accessed June 20 2017. depressie bij kinderen en jongeren (Nederlandse bewerking). Amsterdam: 36. Gezonde School. Pychische problemen, https://www.gezondeschool.nl/ Uitgevers H; 2016. voortgezet-onderwijs/gezondheidsthemas-en-criteria-vignet/welbevinden- 61. Quinnett P. QPR gatekeeper training for suicide prevention: the model, en-sociale-veiligheid/psychische-problemen. Accessed June 20 2017. rationale and theory. Retrieved July. 2007;28:2008. 37. Burns J, Morey C, Lagelée A, Mackenzie A, Nicholas J. Reach out! Innovation 62. Ghoncheh R, Kerkhof AJ, Koot HM. Effectiveness of adolescent suicide in service delivery. Med J Aust. 2007;187(7):S31. prevention e-learning modules that aim to improve knowledge and self- 38. Gulliver A, Griffiths KM, Christensen H. Perceived barriers and facilitators to confidence of gatekeepers: study protocol for a randomized controlled trial. mental health help-seeking in young people: a systematic review. BMC Trials. 2014;15(1):52. Psychiatry. 2010;10(1):113. 63. Multidisciplinaire Richtlijn Depressie Jeugd Addendum GGZ. http://www. 39. Gould MS, Marrocco FA, Kleinman M, Thomas JG, Mostkoff K, Cote J, et al. ggzrichtlijnen.nl/index.php?pagina=/richtlijn/item/pagina.php&richtlijn_id=85. Evaluating iatrogenic risk of youth suicide screening programs: a Accessed 7 June 2017. randomized controlled trial. JAMA. 2005;293(13):1635–43. 64. Griffiths KM, Christensen H, Jorm AF, Evans K, Groves C. Effect of web-based 40. Kerkhof A, Huisman A. The risk of including suicidal patients in RCT studies depression literacy and cognitive–behavioural therapy interventions on into the effectiveness of pharmacotherapy and psychotherapy for adult stigmatising attitudes to depression. Br J Psychiatry. 2004;185(4):342–9. psychiatric disorders and iatrogenic effects as a result of asking about 65. Postmes T, Haslam SA, Jans L. A single-item measure of social identification: suicide. Vrije Universiteit Amsterdam AKP: Faculteit der Gedrags- en reliability, validity, and utility. Br J Soc Psychol. 2013;52(4):597–617. Bewegingswetenschappen; 2017. 66. Pearlin LI, Schooler C. The structure of coping. J Health Soc Behav. 1978:2–21. 41. Gould MS, Kramer RA. Youth suicide prevention. Suicide Life Threat Behav. 67. Jang Y, Haley WE, Small BJ, Mortimer JA. The role of mastery and social 2001;31(s1):6–31. resources in the associations between disability and depression in later life. 42. Wyman PA, Brown CH, Inman J, Cross W, Schmeelk-Cone K, Guo J, The Gerontologist. 2002;42(6):807–13. et al. Randomized trial of a gatekeeper program for suicide prevention: 68. Molina S, Borkovec TD. The Penn State worry questionnaire: psychometric 1-year impact on secondary school staff. J Consult Clin Psychol. 2008; properties and associated characteristics. 1994. 76(1):104. 69. Chorpita BF, Tracey SA, Brown TA, Collica TJ, Barlow DH. Assessment of 43. Cross W, Matthieu MM, Lezine D, Knox KL. Does a brief suicide prevention worry in children and adolescents: an adaptation of the Penn State worry gatekeeper training program enhance observed skills? Crisis. 2010. questionnaire. Behav Res Ther. 1997;35(6):569–81. 44. Donald M, Dower J, Lucke J, Raphael B. The Queensland Young People's 70. Muris P, Meesters C, Gobel M. Reliability, validity, and normative data of the Mental Health Survey Report. Brisbane: Public Health Services, Queensland Penn State worry questionnaire in 8–12-yr-old children. J Behav Ther Exp Health, 2000. Psychiatry. 2001;32(2):63–72. 45. Parslow RA, Jorm AF. Improving Australians’ depression literacy. Med J Aust. 71. Frost RO, Marten P, Lahart C, Rosenblate R. The dimensions of 2002;177(7):S117. perfectionism. Cogn Ther Res. 1990;14(5):449–68. 46. Petty RE, Cacioppo JT, Strathman AJ, Priester JR. To think or not to think. 72. Weisz JR, Chorpita BF, Frye A, Ng MY, Lau N, Bearman SK, et al. Youth top Persuasion: Psychological Insights and Perspectives; 2005. p. 81–116. problems: using idiographic, consumer-guided assessment to identify 47. Corrigan PW, River LP, Lundin RK, Penn DL, Uphoff-Wasowski K, Campion J, treatment needs and to track change during psychotherapy. J Consult Clin et al. Three strategies for changing attributions about severe mental illness. Psychol. 2011;79(3):369. Schizophr Bull. 2001;27(2):187–95. 73. Muthén L, Muthén B. Mplus User’s guide. Los Angeles: Muthen & Muthen; 48. Watson AC, Corrigan P, Larson JE, Sells M. Self-stigma in people with mental 1998–2010. CzeglédiE Body dissatisfaction, trait anxiety and self-esteem in illness. Schizophr Bull. 2007;33(6):1312–8. young men; 2016. 49. Gillham JE, Reivich KJ, Freres DR, Chaplin TM, Shatté AJ, Samuels B, et al. 74. Hall-Lande JA, Eisenberg ME, Christenson SL, Neumark-Sztainer D. Social School-based prevention of depressive symptoms: a randomized controlled isolation, psychological health, and protective factors in adolescence. study of the effectiveness and specificity of the Penn resiliency program. J Adolescence. 2007;42(166):265. Consult Clin Psychol. 2007;75(1):9. 75. Link BG, Cullen FT, Frank J, Wozniak JF. The social rejection of former 50. Wijnhoven L, Creemers D, Vermulst A, Scholte R, Engels R. Randomized mental patients: understanding why labels matter. Am J Sociol. 1987; controlled trial testing the effectiveness of a depression prevention 92(6):1461–500. Gijzen et al. BMC Psychiatry (2018) 18:124 Page 12 of 12

76. Link BG, Cullen FT, Struening E, Shrout PE, Dohrenwend BP. A modified 103. Auerbach RP, Bigda-Peyton JS, Eberhart NK, Webb CA, Ho M-HR. labeling theory approach to mental disorders: an empirical assessment. Am Conceptualizing the prospective relationship between social support, stress, Sociol Rev. 1989:400–23. and depressive symptoms among adolescents. J Abnorm Child Psychol. 77. Dubow EF, Kausch DF, Blum MC, Reed J, Bush E. Correlates of suicidal 2011;39(4):475–87. ideation and attempts in a community sample of junior high and high 104. Kendler KS, Thornton LM, Gardner CO. Stressful life events and previous school students. J Clin Child Psychol. 1989;18(2):158–66. episodes in the etiology of major depression in women: an evaluation of 78. Trivedi MH, Morris DW, Pan J-Y, Grannemann BD, Rush AJ. What moderator the “kindling” hypothesis. Am J Psychiatr. 2000;157(8):1243–51. characteristics are associated with better prognosis for depression? 105. Lewinsohn PM, Allen NB, Seeley JR, Gotlib IH. First onset versus recurrence Neuropsychiatr Dis Treat. 2005;1(1):51. of depression: differential processes of psychosocial risk. J Abnorm Psychol. 79. Cruwys T, Dingle GA, Haslam C, Haslam SA, Jetten J, Morton TA. Social 1999;108(3):483. group memberships protect against future depression, alleviate depression 106. Post RM. Transduction of psychosocial stress into the neurobiology. Am J symptoms and prevent depression relapse. Soc Sci Med. 2013;98:179–86. Psychiatr. 1992;149:999–1010. 80. Van Orden KA, Witte TK, Cukrowicz KC, Braithwaite SR, Selby EA, Joiner TE Jr. 107. Goldston DB, Molock SD, Whitbeck LB, Murakami JL, Zayas LH, Hall GCN. The interpersonal theory of suicide. Psychol Rev. 2010;117(2):575. Cultural considerations in adolescent suicide prevention and psychosocial 81. O’Connor RC, Nock MK. The psychology of suicidal behaviour. Lancet treatment. Am Psychol. 2008;63(1):14. Psychiatry. 2014;1(1):73–85. 108. Brunwasser SM, Garber J. Programs for the prevention of youth depression: 82. Christensen H, Batterham PJ, Mackinnon AJ, Donker T, Soubelet A. Predictors evaluation of efficacy, effectiveness, and readiness for dissemination. J Clin of the risk factors for suicide identified by the interpersonal-psychological Child Adolesc Psychol. 2016;45(6):763–83. theory of suicidal behaviour. Psychiatry Res. 2014;219(2):290–7. 83. Fairweather AK, Anstey KJ, Rodgers B, Butterworth P. Factors distinguishing suicide attempters from suicide ideators in a community sample: social issues and physical health problems. Psychol Med. 2006;36(9):1235–45. 84. Vilhjálmsson R, Sveinbjarnardottir E, Kristjansdottir G. Factors associated with suicide ideation in adults. Soc Psychiatry Psychiatr Epidemiol. 1998;33(3):97–103. 85. Hendin H. Psychodynamics of suicide, with particular reference to the young. Am J Psychiatr. 1991;148(9):1150. 86. Watkins ER. Constructive and unconstructive repetitive thought. Psychol Bull. 2008;134(2):163. 87. Eshun S. Role of gender and rumination in suicide ideation: a comparison of college samples from Ghana and the United States. Cross-Cult Res. 2000; 34(3):250–63. 88. Abela JR, Vanderbilt E, Rochon A. A test of the integration of the response styles and social support theories of depression in third and seventh grade children. J Soc Clin Psychol. 2004;23(5):653–74. 89. Schmaling KB, Dimidjian S, Katon W, Sullivan M. Response styles among patients with minor depression and dysthymia in primary care. J Abnorm Psychol. 2002;111(2):350. 90. Jones NP, Siegle GJ, Thase ME. Effects of rumination and initial severity on remission to cognitive therapy for depression. Cogn Ther Res. 2008;32(4): 591–604. 91. Kuyken W, Watkins E, Holden E, Cook W. Rumination in adolescents at risk for depression. J Affect Disord. 2006;96(1):39–47. 92. Kerkhof A, van Spijker B. Worrying and rumination as proximal risk factors for suicidal behaviour. In: O'Connor RC, Platt S, Gordon J, editors. International handbook of suicide prevention: research, policy and practice. Chichester: Wiley; 2011;199–209. 93. Nolen-Hoeksema S, Jackson B. Mediators of the gender difference in rumination. Psychol Women Q. 2001;25(1):37–47. 94. Smith JM, Alloy LB, Abramson LY. Cognitive vulnerability to depression, rumination, hopelessness, and suicidal ideation: multiple pathways to self- injurious thinking. Suicide Life Threat Behav. 2006;36(4):443–54. 95. Minkoff K, Bergman E, Beck AT, Beck R. Hopelessness, depression, and attempted suicide. Am J Psychiatr. 1973;130(4):455–9. 96. Brown GK, Beck AT, Steer RA, Grisham JR. Risk factors for suicide in psychiatric outpatients: a 20-year prospective study. J Consult Clin Psychol. 2000;68(3):371. 97. Mazza JJ, Reynolds WM. A longitudinal investigation of depression, hopelessness, social support, and major and minor life events and their relation to suicidal ideation in adolescents. Suicide Life Threat Behav. 1998; 28(4):358–74. 98. Weissman AN, Beck AT, Kovacs M. Drug abuse, hopelessness, and suicidal behavior. Int J Addict. 1979;14(4):451–64. 99. Hewitt PL, Newton J, Flett GL, Callander L. Perfectionism and suicide ideation in adolescent psychiatric patients. J Abnorm Child Psychol. 1997;25(2):95–101. 100. Blatt SJ, Zuroff DC. Interpersonal relatedness and self-definition: two prototypes for depression. Clin Psychol Rev. 1992;12(5):527–62. 101. Gruen RJ, Silva R, Ehrlich J, Schweitzer JW, Friedhoff AJ. Vulnerability to stress: self-criticism and stress-induced changes in biochemistry. J Pers. 1997;65(1):33–47. 102. Rice KG, Aldea MA. State dependence and trait stability of perfectionism: a short-term longitudinal study. J Couns Psychol. 2006;53(2):205.