Maskwacis Life Skills Training Program

Using Outcome Mapping to Evaluate a Culturally Adapted Prevention Program in an Indigenous Community Melissa Tremblay1, Natasha Rabbit2, Lola Baydala1, Jennilee Louis2, & Kisikaw Ksay-yin2 University of Alberta; Edmonton, Alberta Nehiyaw Kakeskewina Learning Society; Maskwacis, Alberta Faculty/Presenter Disclosure • Dr Lola Baydala, Kisikaw Ksay-yin, Natasha Rabbit and Melissa Tremblay have no relevant financial relationships with the manufacturer(s) of commercial services discussed in this CME activity • The authors do not intend to discuss an unapproved/ investigative use of a commercial product/device in this presentation Outline • Context and Project background • Outcome Mapping • Stages • Highlights of OM findings • Next steps • Conclusions Canadian Context Employment and Social Exclusion social safety net Access to health Housing and care services food security Aboriginal status Education and colonialism Project Background Maskwacis Project Background • Maskwacis First Nations communities – Recognized need for prevention • Partnership – Members of the Maskwacis Four Nations and University of Alberta researchers • Community-based participatory research (CBPR) – Equitable involvement – Collaboration and co-learning – Aim for social change Maskwacis Life Skills Training Program • Started with the evidence-based LST program • Gilbert Botvin, Cornell University • Partners worked together to culturally adapt, implement and evaluate the program Maskwacis Life Skills Training Program • Delivered in the Four Nations schools by community members Program Evaluation • Questionnaires – Adapted LST questionnaire – Piers-Harris Self-Concept Scale, 2nd Edition • Focus Groups – Elders – Program facilitators – School personnel – Students – Parents • But what about capturing complexity, community change, and unanticipated outcomes? Measuring Complex Outcomes • Interconnectedness occurs between people, systems, and environments. • This interconnectedness leads to interdependence between people, systems, and environments, and gives rise to complex behaviour • Assumptions about cause and effect in open systems Simple Complicated Complex Outcome Mapping • Developed in Canada but used primarily for international development • Participatory planning, monitoring, and evaluation tool • Looks at outcomes in terms of changes in behavior, relationships, actions, and activities • Focus on contributions to outcomes • Focus on outcomes rather than impact Earl, Carden, & Smutylo, 2001 Intentional Design STEP 1: Vision STEP 2: Mission STEP 3: Boundary Partners STEP 4: Outcome Challenges STEP 5: Progress Markers STEP 6: Strategy Maps STEP 7: Organizational Practices Outcome and Performance Monitoring STEP 8: Monitoring Priorities Evaluation Planning STEP 9: Outcome Journals STEP 12: Evaluation Plan STEP 10: Strategy Journal STEP 11: Performance Journal Step 3: Boundary Partners People the People who Project project works benefit from with/through the project Sphere of Sphere of Sphere of concern control influence Step 3: Boundary Partners . Schools . MLST Students . Elders . Facilitators . Education Directors . Parents/ caregivers . Leadership and administration in the 4 Nations . University of Alberta . Funders Step 4: Outcome Challenges • Ideal changes in the behavior, relationships, activities, or actions of boundary partners. – MLST students: • Develop a positive cultural identity with a healthy Nehiyaw lifestyle. – Parents: • Raise kids in a traditional cultural way that is congruent with the aims of the program. Step 5: Progress Markers • Graduated progress markers for each boundary partner illustrate complexity – MLST Students • Expect to see: Attend MLST classes • Like to see: Demonstrate respect for the Elders • Love to see: Speak Cree; Understand the effects of abused substances. Progress Markers • Boundary Partner: Schools • Outcome Challenge: Support and promote the program in the school environment, particularly the cultural aspects of the program. Data Sources: 3 years of meeting minutes; 3 years of post- program focus groups with facilitators, school personnel, students, parents, and Elders; 3 years of facilitator daily reports Progress Markers • Boundary Partner: University of Alberta • Outcome Challenge: Equitable partnerships with community members, practicing authentic CBPR. Step 6: Strategy Maps • Identify strategies used by the program to contribute to achieving an outcome • OM manual – Categorize strategies into a six-cell matrix • Our strategy for strategy maps: – Categorize strategies according to: – Individual vs. community change – Already in place vs. not yet enacted Strategies • Boundary Partner: Schools • Outcome Challenge: Support and promote the program in the school environment, particularly the cultural aspects of the program. Strategies • Boundary Partner: University of Alberta • Outcome Challenge: Equitable partnerships with community members, practicing authentic CBPR. Highlights of Our Findings • Promotion of culture in the schools • Students practicing culture • MLST staff connected to the community • Support from community leadership • University guidelines Conclusions • Benefits of Using OM – Document stories and anecdotes as data – Identify areas of progress – Examine unanticipated outcomes Conclusions • Challenges – Very time-consuming – Steep learning curve • OM has been highly useful for our prevention program as a retrospective evaluation tool Questions? Hai hai! Thank-you! Please feel free to contact us via e-mail: Natasha Rabbit [email protected] Melissa Tremblay [email protected] www.nklearning.ca .

Load more