Evaluation of a health promoting program in French primary schools: methodological presentation of a mixed methods study MR. Guével 1,2 , D. Jourdan 1, J. Pommier 2 1The ‘health education in schools’ research team PAEDI ; 2EHESP – Ecole des Hautes Etudes en Santé Publique INTRODUCTION RESEARCH PURPOSES AND QUESTIONS Schools’ contribution to students’ health and well-being has been increasingly According to Newman’s research purposes typology (Newman et al, 2003), our research recognized. Nevertheless, the development of suitable evaluation approaches for aims to “add to the knowledge base” of health promotion in the school setting. However, health promotion in schools remains a central question. In France, few studies have since it is a complex research, this general purpose can be further specified and divided been published on the implementation and evaluation of programs aiming to develop in two stages, corresponding to the pilot and generalization stages. The first one focuses health promoting (HP) practices among teachers and school communities. on understanding teachers’ and schools’ HP practices and the second focuses on the Our research focuses on the development of an evaluation design that enables to take measure of change due to a HP program based on teacher training and support. into account the complexity of the implementation of HP programs. We thus developed Based on these research purposes, our research questions can be defined as follows: a design based on:  What are the mechanisms and the contextual factors in a school setting that allow the a systemic approach of evaluation taking into account the mechanisms, the contexts educational community to develop a HP approach? and the outcomes as defined in realistic evaluation that we adjusted to our context What is the impact of teacher training and support in the development of HP practices (Chen, 1990, Pawson & Tilley, 1997); at school and class levels?  the combination of qualitative and quantitative approaches (Tashakorri & Teddlie, What is the impact of these practices on children’s well-being in the school setting? 2003, Creswell & Plano Clark, 2007). What is the impact of these practices on the relationships established with parents? Figure 3 presents the iterative process from research purposes to opportunities to use PRESENTATION OF THE HP PROGRAM PRESENTATION OF THE HP PROGRAM quantitative and qualitative approaches (mixed methods). This initiative aims at: Iterative process between purpose and questions Decision about methods  improving children’s well-being at primary school, Research Theoretical Theoretical Traditional Research questions Opportunities to use mixed methods  developing sustainable HP projects in school settings by promoting, sharing and purposes drive framework approaches - Which resources enable teachers and Understand Analysis of Traditional plus quantitative research schools to develop HP practices? Qualitative enhancing teachers’ and schools’ HP practices. complex Inductive activity using multivariate techniques and taking - How do they use them to develop HP research phenomena (Leplat, 2008) into account multiple points of view Figure 1 presents the main principles and practices that underlie this HP program. practices? - What is the impact of teacher training Principles - The program … and support in the development of HP practices at school and class levels? Theory of Measure -What is the impact of these practices on social change Quantitative Traditional plus qualitative research Integrates public health and Deductive Is based on recommended best change well-being in the school setting? (Schoonbroodt, approaches using open-ended process educational science approaches practices -What is the impact of these practices on 1996) Takes into account specificities of the the relationships established with Is based on partnerships between French school setting parents? professionals and researchers Figure 3 : Iterative process from research purposes to opportunities to use mixed methods Integrates a global and ecological Combines top down and bottom up perspective of health approaches METHODS Considers ethical issues The pilot stage (stage 1) is based on a convergent triangulation design: QUAL+QUAN . Practices – The program … Data collection, analysis and interpretation were concurrent and separate. Results were Sustains local resources Is based on teacher training and support then compared and contrasted during the final interpretation step. Provides tools and encourages teachers to share their practices Links with parents and communities For the generalization stage (stage 2), an embedded design is used: QUAN(qual). It is Is integrated into schools’ practices mainly based on a QUAN approach where qual data provide a supportive role. Data are Figure 1 : Program’s main principles and practices concurrently collected and then analyzed separately. Some qualitative data are In each region, a resource team is trained. It is formed of teacher trainers, pedagogical transformed into quantitative data and introduced in statistical models. The interpretation advisers, members of HP services and members of local HP NGOs. This team then combines both quantitative and qualitative data. trains and supports the teachers. The aim is that teachers integrate HP approaches in In both stages, data are collected from pupils, parents, teachers and school communities. their everyday practices at the class level as well as at the school and community In the pilot stage, there is a 4 year follow-up. In the generalization stage, there is a 3 year levels. follow-up, with data collection at the beginning of the project and then at the end of each year (T , T , T , T ). The project is constituted of two stages: a pilot and a generalization stages. Figure 2 0 1 2 3 presents the localisation and the participation to the 2 stages of the project. Figure 4 details the design of the 2 stages and presents the data collection and analysis

Nord Pas Stage 1 (2003-2007): tools. de Calais 1 French region, Versailles 21 primary schools: Pilot stage: convergent triangulation design (21 schools) Generalization stage: embedded design Rouen Amiens 2,381 pupils and their (160 schools) family, 110 teachers. Basse Lorraine Normandie 16 Bretagne Ardenne QUAN QUAN QUAN data data data interpretation Créteil 16 s 2 usedstage in Data collection Data analysis Interpretation Pays de 16 collection analysis Orléans Bourgogne Besançon la Loire Descriptive Interpretation Tour QUAN Interpretation Stage 2 (2008-2011): Questionnaires statistics QUAN + QUAL based on Inferential Prediction 10 French regions, Observations results Descriptive Poitou statistics Interpretation stage 1 Lyon 160 primary schools: statistics Questionnaires based on stage 2 16 16 5,000 pupils and their Compare Inferential Grenoble family, 800 teachers. and Larger sense statistics QUAN(qual) results results 16 contrast making results Asking questions qual Generalization Midi 16 Contextualizing Prediction Aix Nice Modeling Pyrénées Personal Larger Interpretation Marseille of theory interpretation Open-ended Description sense 16 QUAL QUAL QUAL data process Category making Montpellier data data interpretation Interviews identification Corse collection analysis Pacifique Guadeloupe Martinique Guyane La Réunion Interviews Description Larger 16 16 Documents Category sense Observations identification making

Figure 2 : Localisation and participation to stage 1 (dark purple) and stage 2 (purple) of the project tool and methods for Basisconstructing strategies, This initiative is supervised by a scientific committee formed of health and educational Figure 4 : A mixed methods design to add to the knowledge base of health promotion in the school experts and practitioners. An ethics committee was also settled. settings: pilot and generalization stages DISCUSSION Mixed methods provide a framework to combine qualitative and quantitative data, it helps to understand complex systems and therefore is an option to strengthen evaluation in the field of health promotion. Moreover, a systemic approach of evaluation that aims at understanding how the mechanisms are triggered, in which context and for whom, gives elements that can be decisive in the extension of the program and may help policy-makers and stakeholders to adapt the program to the specificities of their context and population. This research program is supported by the French national institute for health education and prevention (INPES) .