6/20/17 Presenter Disclosures Measuring Success: Kimi Sakashita, MPH Improving School Health Center Sara Geierstanger, MPH Samira Soleimanpour, MPH, PhD Evaluation Using a Results-Based Accountability Framework The following personal financial relationships with commercial interests relevant to this presentation existed during the past 12 months: School-Based Health Alliance 2017 Convention No relationships to disclose Kimi Sakashita, MPH Sara Geierstanger, MPH Samira Soleimanpour, MPH, PhD Alameda County Health Care Services Agency Center for Healthy Schools and Communities Learning Objectives Alameda County, California • Explain how a Results-Based Accountability (RBA) Framework can be used to evaluate school health centers (SHCs) and “tell the story.” • Describe SHC evaluation results and measures of health and education equity using a RBA Framework. • Discuss how to incorporate the national SHC performance measures into an evaluation that uses the RBA framework. Population: 1,578,891 • Discuss how to leverage existing data systems or implement new School Districts/Schools : 18/389 strategies to collect data. Number of Students: 222,681 Free/Reduced Lunch Percentage: 45% College graduates, persons 25 and over: 42% Median household income: $72,399 Source: American Community Survey, 2013. 1 6/20/17 29 Current School Health Centers A l a m e d a C o unt y H e a l t h C a r e Se r v i c e s A g e nc y Berkeley High School Health Center Emeryville Unified BTA Health Center TechniClinic u Piedmont Wellness Center Chappell Hayes u Health Center Shop 55 Wellness Center West Oakland Middle Hawthorne Elementary u u u School u United for Success u Skyline High School uu u u Frick Middle School Youth Heart Health Center uu u u Youth Uprising Health Center Roosevelt Health Center u u u u Elmhurst/Alliance Campus u u Rising Harte Wellness Center Fuente - REACH AYC ACLC/NEA Campus u Encinal High School u Alameda High School u Fremont Tiger Clinic Why RBA? Havenscourt Campus uu Madison Park Academy u San Leandro High School Health Center u San Lorenzo High School Health Center Hayward High Mobile Clinic u Tennyson Health Center Logan Health Center Alameda County Health Care Services Agency LI N K IN G H E A LT H A N D E D U C A T I ON T O C H A N G E L I VE S AN D AC H I E V E E Q U I T Y 10 00 SA N L E A N D RO B O U L E VA R D S U I TE 3 00 | S A N L EA N D R O , C A 94 5 7 7 | WWW . A C H E A LT H Y S C H O OL S . OR G Center for Healthy Schools and Communities Updated 3/4/16 RBA Framework RBA in Alameda County STEPS 1. Choose RESULTS or conditions of well-being that you want to achieve for youth, families and communities. 2. Identify the overall population you hope to influence and the POPULATION INDICATORS that measure these results at the population level. 3. Identify the people you are directly serving and choose PERFORMANCE MEASURES that measure the impact of your programs/services on them. Find measures that answer: • How much did you do? • How well did you do it? • Is anyone better off? h<ps://clearimpact.com/results-based-accountability/ 2 6/20/17 Alameda County SHC Evaluation Logic Model QUESTIONS 1. Name one overall RESULT (condition) you are working towards. a. Name a POPULATION INDICATOR that matches this result and the population you hope to influence. Remember, the population you choose is LARGER than the clients you serve. 2015-16 Data Collection Tools • Client Services Form (Efforts to Outcomes/ETO) Ø 14,790 clients • Community Functioning Evaluation Form (ETO) Ø 227 behavioral health clients Evaluation Methods & Findings • Student and School Community Activity Log (Activity Log) Ø 113,234 youth • Quarterly Reports Ø 27 SHC directors Alameda County Health Care Services Agency Center for Healthy Schools and Communities 3 6/20/17 Sharing Data: QI, Accountability, Making the Case 2015-16 Data Collection Tools (cont.) • School Health Centers • California Healthy Kids Survey, SHC Module (CHKS) Ø Agency/District Level Reports Ø 11,786 students Ø Site Level Workbooks Ø 58% students from 24 schools • County Level • Client and Youth Program Participant Survey Ø Measure A Report Ø 1,632 clients Ø Insurance Trends Ø 50% of all clients over a 2-month period Ø Program Level Ø Presentations to stakeholders • Making the Case Ø Issue Brief Ø Presentations to stakeholders Sharing 2015-16 Findings with you today! • Access to Care • Reproductive Health Services • Oral Health Services • Behavioral Health Services Access to Care • Youth Engagement ** NQI Measures 4 6/20/17 PERFORMANCE MEASURE How much? Increased clients & visits How much? Access to Care 63,345 Clients Served Clinic Visits 60,780 Client Demographics 56,967 Of the nearly 37,000 Alameda 51,715 52,226 County students with access to a School Health Center, 41,519 41,802 31% were registered clients 33,614 27,078 27,394 SHC student Students who had 14,790 clients 13,017 14,446 Male 11,399 11,813 Female 31% not used 8,968 9,514 42% 6,624 6,642 7,276 58% SHC (as a registered client) 2006-07 2007-08 2008-09 2009-10 2010-11 2011-12 2012-13 2013-14 2014-15 2015-16 69% (11 SHCs) (12 SHCs) (12 SHCs) (14 SHCs) (18 SHCs) (23 SHCs) (23 SHCs) (25 SHCs) (27 SHCs) (28 SHCs) 2015-16 School Health Centers serve: • 2 elementary schools • 12 middle schools • 24 high schools Data Source: ETO Client Service Form Data Source: ETO Client Service Form and California Dept. of Education PERFORMANCE MEASURE PERFORMANCE MEASURE How much? Client demographics How well? Client satisfaction Experiences with the School Health Center % Agree or (n=1,484-1,567) Strongly Agree 40% 43% The people who work there helped me to work through my 98% problem 22% 22% It felt like a safe place 99% 18% 12% 12% 7% 10% The people who work there were easy to talk to 98% 3% LaHno/a African American Asian/Pacific White Bi/MulH-Racial or Islander/Filipino Other § Demonstrating satisfaction… School Health Center Clients (Students only) Student PopulaHon § 62% clients returned for ≥1 visit § 30% returned for a different type of service Data Sources: ETO Client Service Form and California Dept. of Education Data Sources: Client Survey and ETO 5 6/20/17 PERFORMANCE MEASURE PERFORMANCE MEASURE Better off? Increased access How much? Increasing insurance enrollment § School Health Center clients were more likely than • 24 of the 27 SHCs conducted some amount of application non-clients to “always” receive counseling when needed assistance to educate and/or enroll 7,220 families in health coverage and other benefits programs, such as CalFresh. (32% vs. 27%, p<0.001) Experiences with the School Health Center % Agree or (n=1,484-1,567) Strongly Agree They helped to get help sooner than I normally would 97% They helped me to get services I wouldn’t otherwise get 96% Data Sources: CHKS and Client Survey Data Sources: ETO Client Service Form and California Dept. of Education (2015-16) PERFORMANCE MEASURE POPULATION INDICATOR Better off? Increased insurance enrollment Better off? Increased insurance enrollment % SHC Clients with No Reported Insurance % Uninsured at Any Point in Last Year California Alameda County 28% 14% 25% 10% 24% 10% 9% 8% 10% 8% 7% 19% 17% 8% 7% 7% 7% 4% 4% 2001 2003 2005 2007 2009 2011 2013 2011-12 2012-13 2013-14 2014-15 2015-16 Definition: % children ages 0-17 uninsured at any point during the 12 months prior to the survey. Data Source: ETO Client Service Form representing 56% (n=8,235) of clients with insurance documentation. Data Source: California Health Interview Survey, UCLA 6 6/20/17 How much? Reproductive Health Services § Provided during 28% of all School Health Center visits (Client Service Form, ETO) § Contraceptive counseling (87% of reproductive/sexual health services) Reproductive Health Services § STI screening/counseling (37%) § HIV screening/counseling (21%) § Chlamydia screening (19%) § School Health Centers also provided 1,033 HPV immunizations Data Source: Client Service Form, ETO How well? Reproductive Health Services POPULATION INDICATOR Better off? Te e n b i r t h r a t e s § Stay tuned…. California Alameda County 40.2 40.1 38.3 35.4 31.6 28.3 26.2 23.2 27.6 26.0 25.6 24.7 21.8 19.3 17.7 14.7 2006 2007 2008 2009 2010 2011 2012 2013 Definition: Number of births per 1,000 young women ages 15-19. Data Source: CA Dept. of Finance, CA Dept. of Public Health, & CDC 7 6/20/17 POPULATION INDICATOR Better off? Chlamydia rates California Alameda County 1306.5 1316.1 1277.6 1302.2 1125.0 1164.2 973.7 885.1 788.4 Oral Health Services 779.7 815.0 803.0 803.7 816.6 779.3 742.2 725.2 684.4 2006 2007 2008 2009 2010 2011 2012 2013 2014 Definition: Rate of chlamydia and gonorrhea infections per 100,000 youth ages 10-19. Data Source: CA Dept. of Public Health, CA Dept. of Finance, CDC, & US Census PERFORMANCE MEASURE PERFORMANCE MEASURE How much? Oral health visits and clients How much? Oral health screenings 4,484 SHC Dental Visits 3,871 • 18% of all visits (1,341 clients) had a dental service SHC Dental Clients provided for screening exams and cleanings, and 3,072 restoraHve treatment.