6/20/17

Presenter Disclosures

Measuring Success: Kimi Sakashita, MPH Improving School Health Center Sara Geierstanger, MPH Samira Soleimanpour, MPH, PhD Evaluation Using a Results-Based Accountability Framework The following personal financial relationships with commercial interests relevant to this presentation existed during the past 12 months: School-Based Health Alliance 2017 Convention No relationships to disclose Kimi Sakashita, MPH Sara Geierstanger, MPH Samira Soleimanpour, MPH, PhD

Alameda County Health Care Services Agency Center for Healthy Schools and Communities

Learning Objectives Alameda County, California

• Explain how a Results-Based Accountability (RBA) Framework can be used to evaluate school health centers (SHCs) and “tell the story.”

• Describe SHC evaluation results and measures of health and education equity using a RBA Framework.

• Discuss how to incorporate the national SHC performance measures into an evaluation that uses the RBA framework. Population: 1,578,891 • Discuss how to leverage existing data systems or implement new School Districts/Schools : 18/389 strategies to collect data. Number of Students: 222,681 Free/Reduced Lunch Percentage: 45% College graduates, persons 25 and over: 42% Median household income: $72,399

Source: American Community Survey, 2013.

1 6/20/17

29 Current School Health Centers

A l a m e d a C o unt y H e a l t h C a r e Se r v i c e s A g e nc y

Berkeley High School Health Center Emeryville Unified

BTA Health Center TechniClinic u Piedmont Wellness Center Chappell Hayes u Health Center Shop 55 Wellness Center West Oakland Middle Hawthorne Elementary u u u School u United for Success u Skyline High School uu u u Frick Middle School Youth Heart Health Center uu u u Youth Uprising Health Center Roosevelt Health Center u u u u Elmhurst/Alliance Campus u u Rising Harte Wellness Center Fuente - REACH AYC ACLC/NEA Campus u Encinal High School u Alameda High School u Fremont Tiger Clinic Why RBA?

Havenscourt Campus uu

Madison Park Academy u San Leandro High School Health Center u

San Lorenzo High School Health Center Hayward High Mobile Clinic u Tennyson Health Center

Logan Health Center

Alameda County Health Care Services Agency LI N K IN G H E A LT H A N D E D U C A T I ON T O C H A N G E L I VE S AN D AC H I E V E E Q U I T Y 10 00 SA N L E A N D RO B O U L E VA R D S U I TE 3 00 | S A N L EA N D R O , C A 94 5 7 7 | WWW . A C H E A LT H Y S C H O OL S . OR G Center for Healthy Schools and Communities Updated 3/4/16

RBA Framework RBA in Alameda County

STEPS

1. Choose RESULTS or conditions of well-being that you want to achieve for youth, families and communities.

2. Identify the overall population you hope to influence and the POPULATION INDICATORS that measure these results at the population level.

3. Identify the people you are directly serving and choose PERFORMANCE MEASURES that measure the impact of your programs/services on them. Find measures that answer: • How much did you do? • How well did you do it? • Is anyone better off?

h

2 6/20/17

Alameda County SHC Evaluation Logic Model QUESTIONS

1. Name one overall RESULT (condition) you are working towards.

a. Name a POPULATION INDICATOR that matches this result and the population you hope to influence. Remember, the population you choose is LARGER than the clients you serve.

2015-16 Data Collection Tools • Client Services Form (Efforts to Outcomes/ETO) Ø 14,790 clients

• Community Functioning Evaluation Form (ETO) Ø 227 behavioral health clients Evaluation Methods & Findings • Student and School Community Activity Log (Activity Log) Ø 113,234 youth

• Quarterly Reports Ø 27 SHC directors Alameda County Health Care Services Agency Center for Healthy Schools and Communities

3 6/20/17

Sharing Data: QI, Accountability, Making the Case 2015-16 Data Collection Tools (cont.) • School Health Centers • California Healthy Kids Survey, SHC Module (CHKS) Ø Agency/District Level Reports Ø 11,786 students Ø Site Level Workbooks Ø 58% students from 24 schools • County Level • Client and Youth Program Participant Survey Ø Measure A Report Ø 1,632 clients Ø Insurance Trends Ø 50% of all clients over a 2-month period Ø Program Level Ø Presentations to stakeholders

• Making the Case Ø Issue Brief Ø Presentations to stakeholders

Sharing 2015-16 Findings with you today!

• Access to Care • Reproductive Health Services • Oral Health Services • Behavioral Health Services Access to Care • Youth Engagement

** NQI Measures

4 6/20/17

PERFORMANCE MEASURE How much? Increased clients & visits How much? Access to Care 63,345 Clients Served Clinic Visits 60,780 Client Demographics 56,967 Of the nearly 37,000 Alameda 51,715 52,226 County students with access to a School Health Center, 41,519 41,802 31% were registered clients 33,614 27,078 27,394 SHC student Students who had 14,790 clients 13,017 14,446 Male 11,399 11,813 Female 31% not used 8,968 9,514 42% 6,624 6,642 7,276 58% SHC (as a registered client) 2006-07 2007-08 2008-09 2009-10 2010-11 2011-12 2012-13 2013-14 2014-15 2015-16 69% (11 SHCs) (12 SHCs) (12 SHCs) (14 SHCs) (18 SHCs) (23 SHCs) (23 SHCs) (25 SHCs) (27 SHCs) (28 SHCs)

2015-16 School Health Centers serve: • 2 elementary schools • 12 middle schools • 24 high schools Data Source: ETO Client Service Form Data Source: ETO Client Service Form and California Dept. of Education

PERFORMANCE MEASURE PERFORMANCE MEASURE How much? Client demographics How well? Client satisfaction

Experiences with the School Health Center % Agree or (n=1,484-1,567) Strongly Agree

40% 43% The people who work there helped me to work through my 98% problem 22% 22% It felt like a safe place 99% 18% 12% 12% 7% 10% The people who work there were easy to talk to 98% 3%

LaHno/a African American Asian/Pacific White Bi/MulH-Racial or Islander/Filipino Other § Demonstrating satisfaction… School Health Center Clients (Students only) Student PopulaHon § 62% clients returned for ≥1 visit § 30% returned for a different type of service

Data Sources: ETO Client Service Form and California Dept. of Education Data Sources: Client Survey and ETO

5 6/20/17

PERFORMANCE MEASURE PERFORMANCE MEASURE Better off? Increased access How much? Increasing insurance enrollment

§ School Health Center clients were more likely than • 24 of the 27 SHCs conducted some amount of application non-clients to “always” receive counseling when needed assistance to educate and/or enroll 7,220 families in health coverage and other benefits programs, such as CalFresh. (32% vs. 27%, p<0.001)

Experiences with the School Health Center % Agree or (n=1,484-1,567) Strongly Agree They helped to get help sooner than I normally would 97%

They helped me to get services I wouldn’t otherwise get 96%

Data Sources: CHKS and Client Survey Data Sources: ETO Client Service Form and California Dept. of Education (2015-16)

PERFORMANCE MEASURE POPULATION INDICATOR Better off? Increased insurance enrollment Better off? Increased insurance enrollment

% SHC Clients with No Reported Insurance % Uninsured at Any Point in Last Year California Alameda County 28% 14% 25% 10% 24% 10% 9% 8% 10% 8% 7% 19% 17% 8% 7% 7% 7% 4% 4%

2001 2003 2005 2007 2009 2011 2013

2011-12 2012-13 2013-14 2014-15 2015-16 Definition: % children ages 0-17 uninsured at any point during the 12 months prior to the survey.

Data Source: ETO Client Service Form representing 56% (n=8,235) of clients with insurance documentation. Data Source: California Health Interview Survey, UCLA

6 6/20/17

How much? Reproductive Health Services § Provided during 28% of all School Health Center visits (Client Service Form, ETO) § Contraceptive counseling (87% of reproductive/sexual health services) Reproductive Health Services § STI screening/counseling (37%) § HIV screening/counseling (21%) § Chlamydia screening (19%)

§ School Health Centers also provided 1,033 HPV immunizations

Data Source: Client Service Form, ETO

How well? Reproductive Health Services POPULATION INDICATOR Better off? Te e n b i r t h r a t e s

§ Stay tuned…. California Alameda County 40.2 40.1 38.3 35.4 31.6 28.3 26.2 23.2 27.6 26.0 25.6 24.7 21.8 19.3 17.7 14.7

2006 2007 2008 2009 2010 2011 2012 2013

Definition: Number of births per 1,000 young women ages 15-19.

Data Source: CA Dept. of Finance, CA Dept. of Public Health, & CDC

7 6/20/17

POPULATION INDICATOR Better off? Chlamydia rates

California Alameda County

1306.5 1316.1 1277.6 1302.2 1125.0 1164.2 973.7 885.1 788.4 Oral Health Services

779.7 815.0 803.0 803.7 816.6 779.3 742.2 725.2 684.4

2006 2007 2008 2009 2010 2011 2012 2013 2014

Definition: Rate of chlamydia and gonorrhea infections per 100,000 youth ages 10-19.

Data Source: CA Dept. of Public Health, CA Dept. of Finance, CDC, & US Census

PERFORMANCE MEASURE PERFORMANCE MEASURE How much? Oral health visits and clients How much? Oral health screenings

4,484 SHC Dental Visits 3,871 • 18% of all visits (1,341 clients) had a dental service SHC Dental Clients provided for screening exams and cleanings, and 3,072 restoraHve treatment. 2,444 • 2,489 students participated in one of 27 school-wide 1,533 1,339 dental screenings 1,041 763 766 506

2011-12 2012-13 2013-14 2014-15 2015-16 (4 SHCs) (5 SHCs) (6 SHCs) (8 SHCs) (10 SHCs)

Data Source: ETO Client Service Form Data Source: Activity Log and Client Service Form, ETO

8 6/20/17

PERFORMANCE MEASURE PERFORMANCE MEASURE How well? Oral health screenings Better off? Improved oral health

• 91% School Health Center dental clients had a clinical At the 10 sites that provided oral health services: baseline dental assessment § 85% were found to have some dental decay § 85% of clients had suspicious areas of decay identified at first oral health assessment Ø 20% had urgent dental needs identified Ø Decay improved or did not worsen over time in 82% of clients screened at baseline and follow-up

Data Source: Activity Log and Client Service Form, ETO Data Source: ETO Client Service Form

PERFORMANCE MEASURE How much? Behavioral health services

School-Wide Prevention (Tier 1 and Tier 2) # Contacts Restorative justice/circle activities 2,901 School/teacher consultation (i.e., Coordination of Services Team) 2,400 Screening, trauma, 9th and other grades 1,753 School safety/climate presentation/activity 1,710 Behavioral Health Services Social skills/communication/anger management/conflict resolution 1,175

Client Early/Intensive Intervention (Tiers 2 & 3) Services, by type of Behavioral Health Service (>1 service can be selected per visit; N=21,587 visits) Individual therapy 44% Assessment/intake 17% Psychosocial screening 15% Individual contact 8% Plan development 7%

Data Source: Activity Log and ETO Client Service Form

9 6/20/17

PERFORMANCE MEASURE PERFORMANCE MEASURE How well? Behavioral health services Better off? Behavioral health services

Experiences with the School Health Center % Agree or (n=918-1,464) Strongly Agree Feel like I had an adult I could turn to if I needed help or support 97% Feel safe talking about my problems 97% Experiences with the School Health Center % Agree or (n=918-1,464) Strongly Agree § Among 6 OUSD School Health Centers (in high schools) Deal with stress/anxiety better 94% providing trauma-informed interventions: Concentrate better 93% § 21% of unduplicated clients received trauma screenings during Stop using or use less tobacco, alcohol or drugs 91% a visit to the School Health Center Feel less irritable or have less angry outbursts 91% § 35% of those screened positive for trauma and needed follow-up care § 86% were connected to services/supports to address their needs

Data Source: Client Survey and ETO Client Service Form Data Source: Client Survey

PERFORMANCE MEASURE POPULATION INDICATOR Better off? Behavioral health services Better off? Depression rates

% Change in Presenting Concerns, Baseline and Follow-Up (N=227) 30.4% *significant at p≤0.005, ** significant at p<0.0001 30.0% 29.1% Social Relationships 36%**

Health/Basic Needs 35%** 26.9%

Emotional & Behavioral Functioning 32%**

Exposure 24%** 2011-2013 2013-15 Living Arrangements & Family California Alameda County 18%** Functioning Definition: Percentage of public school students in grades 7, 9, 11, and non-traditional students reporting Academic Functioning 17%* whether in the past 12 months they had felt so sad or hopeless almost every day for two weeks or more that they stopped doing some usual activities. Data Source: CHKS & California Student Survey (2011-2015); Note: Alameda County ”2013-15” data are from Data Source: Community Functioning Evaluation, ETO the 2015/16 school year only and should be considered estimates.

10 6/20/17

PERFORMANCE MEASURE How much? Youth engagement

Additional Contacts with Youth # Sessions # Contacts

Health fairs/outreach 214 26,196 School-wide assembly or special event 34 5,222 Yo u t h E n g age m e n t Job training/career exploration (e.g., applying, internships, 305 2,801 shadowing) School Community Supports Youth advisory board/leadership/research/advocacy 299 2,663 Peer health education group/peer counseling/mentoring 238 2,442 Physical activity/recreation/dance/yoga 148 1,877 Acculturation support (for newcomers, unaccompanied youth, 116 1,376 etc.)

Data Source: Activity Log

PERFORMANCE MEASURE PERFORMANCE MEASURE Better off? Youth enrichment Better off? School assets Impact of SHC Services (n=1,042-1,272) % Agree or Strongly Agree Have goals and plans for the future 95% § More SHC clients reported higher levels of school Stay in school 93% assets than non-clients, including: Do better in school 89% § Having caring relationships with teachers or other adults at Have better attendance (cut classes less) 87% school (36% vs. 29%, p<0.001) § Having a teacher or other adult at school who had high expectations for them (49% vs. 41%, p<0.001) Impact of SHC Group Programs (n=370-437) % Agree or Strongly Agree Learn skills that will help in future 98% Feel more confident 96% Make school a better place 96% Feel more connected to people at school 93%

Data Source: Client Survey Data Source: CHKS

11 6/20/17

PERFORMANCE MEASURE PERFORMANCE MEASURE Better off? Seat time POPULATION INDICATOR § After 96% of medical visits, clients were sent back to Better off? Suspension and Graduation rates class potentially saving an absence from school for the rest of the school day Suspension Rates Graduation Rates § With an estimated Average Daily Attendance (ADA) payment of $56/day, this saved school districts nearly $900,000 2010-11 2014-15 % 2010-11 2014-15 % (16,017 medical visits x $56) in ADA in 2015-16 Change Change California 11.3 3.8 -66% 77.1 82.3 +7% Alameda County 9.6 3.3 -66% 78.0 85.0 +9% Alameda County School 18.4 6.3 -66% 72.3 79.5 +10% Health Center schools

Data Source: California Dept. of Education, Data Reporting Office: California Longitudinal Pupil Data Source: Client Service Form, ETO; ADA estimate from OUSD (2014-15) Acheivement Data Systems (CALPADS)

QUESTIONS

2. What is one thing you wish you could measure? National Quality Initiative a. Is there a proxy measure that would not be resource intensive?

12 6/20/17

SBHA National Quality Initiative Evaluation Client Service Form –NQI-Related Fields § School-Based Health Alliance, with support from the Maternal and Child Health Bureau (MCHB), launched the School Health Services National Quality Initiative SCREENINGS/NEEDS AND DIAGNOSES/PROCEDURES (NQI) Providers: complete sections that are applicable to each visit and check all boxes that apply. PHYSICAL AND BEHAVIORAL HEALTH PHYSICAL AND BEHAVIORAL HEALTH § Central to NQI is the adoption SHC standardized performance measures to help SCREENINGS/NEEDS (during this visit) DIAGNOSES/PROCEDURES Annual Risk Assessment (GAPS, RAAPS, HEEADSSS, q Nutrition counseling (E66, Z68.51-4, Z71.3) describe the quality of care delivered in SHCs and demonstrate our collective etc.; 99420): ¡ Yes impact q Physical activity counseling (E66, Z68.51-4, Z71.3) BMI Screening Percentile: ¡ < 5% (underweight) ¡ 5-84% (normal) q Well care visit (99381-5, 99391-5, Z00.00-.01, § Performance measures include: ¡ 85%-94% (overweight) ¡ 95%+ (obese) Z00.121, Z00.129) 1. Annual Well Child Visits Depression Screening (PHQ-2, PHQ-9, etc.; 96127): 2. Annual Risk Assessments ¡ Yes 3. BMI Screening and Nutrition/Physical Activity Counseling 4. Depression Screening with Follow-up Plan SEXUAL/REPRODUCTIVE HEALTH SEXUAL/REPRODUCTIVE HEALTH 5. Chlamydia Screening SCREENING/NEEDS DIAGNOSES/PROCEDURES Is client sexually active (past 3 months)? q Chlamydia screening (86631-2, 87110, 87270, ¡ Yes ¡ No § For more information, visit: http://www.sbh4all.org/resources/quality-counts/ 87320, 87490-2, 87810, Z11.8)

NQI PERFORMANCE MEASURES How well?

% Unduplicated clients with documentation of ≥1 of the Alameda National following during the school year: County SHCs Data Well Child Visit: ≥1 comprehensive well-care visit with a PC or an OB/ 9% 44%* GYN practitioner regardless of where the exam was provided, including documentation of health and developmental history, physical exam, and health education/anticipatory guidance

Annual Risk Assessment: ≥1 age-appropriate annual risk assessment Not documented N/A BMI Screening & Nutrition/Physical Activity Counseling (ages 3-20 8% all clients 58%** Questions? years): • BMI percentile AND counseling for nutrition AND physical activity 37% clients w/ BMI TBD ≥85% • BMI ≥85% AND counseling for nutrition AND physical activity Depression Screening & Follow-up Plan (ages ≥12 years): Not documented 51% (includes ≥1 screening for clinical depression using age appropriate standardized tool adults)** AND a follow-up plan documented if positive screen For more information contact: Kimi Sakashita, MPH Chlamydia Screening: ≥1 test for Chlamydia if identified as sexually active 52% female 48%* female 47% male N/A male Center for Healthy Schools and Communities Alameda County Health Care Services Agency [email protected] Data Source: ETO Client Service Form (2015-16); *DHHS: www.achealthyschools.org https://www.medicaid.gov/medicaid/quality-of-care/downloads/2015-child-sec-rept.pdf; **https://bphc.hrsa.gov/uds/datacenter.aspx

13