Molokai LIVE Program Evaluation School Year 2015-16 June 5, 2015 to May 26, 2016 SuBmitted on DecemBer 21, 2016 By: Laura R. Peterson, Program Evaluator EXECUTIVE SUMMARY The evaluation was designed to determine progress Molokai LIVE has made towards meeting objectives and to determine outcomes. It has been determined that significant progress has been made in all three of the project goals: 1. To support the education provided by the schools in meeting and further supporting student academic development. 2. To develop student interest and aspirations through enriched learning and college and career readiness. 3. To partner with parents, community, and educators to expand student learning opportunities and support. PROGRAM IMPLEMENTATION ⋅ Molokai LIVE was open 16 hours a week, 5 days a week, and 7 weeks over summer 2015, and after school 21 hours a week, 5 days a week, and 38 weeks over the 2105-16 School Year. ⋅ Students were supported as never before, with 174 students served during the Summer of 2015, and 336 students served during the 2015-16 School Year. ⋅ The average increase shown by both 30-day and 60-day attendees was 71% in reading and 47% in math on school universal screeners. ⋅ There were 9% more 60 day attendees that showed growth in Lexile scores and 17% more 60 day attendees that showed growth in math measures than 30 day attendees, indicating that longer attendance in the program leads to more student improvement in reading and math. ⋅ Of the regular attendees, students showed improvement in reading/language, math, and behavior in increasing percentages for 30, 60, and 90-day students (with the exception of 90 day students in math), according to teacher reports and student grades. ⋅ Various Enrichment classes were offered in the categories of Arts and Music and Physical Activities in order to engage students. ⋅ College and Career Readiness supports were offered in the academic class categories of STEM, Tutoring, and Homework Help, as well as a Parent/Student Academy. ⋅ Molokai LIVE engaged 155 adults and family members in various programs. In addition, one-time events became a venue for increased parent and family participation with events such as the twice-annual Family Fair. ⋅ Event surveys consistently reflect high satisfaction, for example in the Parent/Student Academies, 95% expressed high levels of satisfaction with the monthly academies, including overall level of satisfaction, comfort with the college process, and impact on college planning. ⋅ Administrators gave positive feedback for collaborations with the principals and staff, especially excelling in the areas of including students and parents, communicating with all stakeholders, inclusion in decision- making, and timely and appropriate responsiveness. ⋅ The core team continued to cultivate 14 Partnerships to ensure growth and success of the program. ⋅ Molokai LIVE provided extended hours into the evening for Family Learning time. ⋅ Molokai LIVE employed 10 staff members over the summer, 22 in the fall, and 15 in the spring, plus volunteers. Both staff and volunteers consisted of school day teachers, non-teaching school day staff, community members, students, and parents. ⋅ Between the adjacent public feeder schools of MHS and MMS, Molokai LIVE served 68% of the student population, reaching the 60% goal for this reporting period (not including the private school Aka‘ula). ⋅ Including Aka‘ula, Molokai LIVE served 48% of the total student population, an increase from 44% of total students from last reporting period (14-15 SY). ⋅ Just over 20% of enrolled students attained 30-day participation, meeting the 20% goal for this reporting period. ⋅ The demographic information collected reflects the student population, with the students being 81% Native Hawaiian, 15% Filipino, 3% White and 1% Other. ⋅ Molokai LIVE also served Males, Females, English Language Learners, and Special Education students consistent with school population percentages. This indicates that all populations were served and felt welcome. EVALUATION SUMMARY Results will be utilized in order to address challenges for this reporting period: 1. Hiring and retention of personnel for program needs and development. 2. Limited student participation in enrichment class offerings. 3. Building “Family Engagement and Empowerment” to support student learning and achievement. 4. Effective communication about program activities and events that increase participation. 5. Opportunities for student internship, job shadowing, and employment are limited due to the socio- economic factors within the community. 6. Systems in place that expedite timely formative (quarterly) program data. MAJOR FINDINGS AND RECOMMENDATIONS Recruitment: Activate interest by utilizing social media with the Molokailive.com website, a Facebook page, and a quarterly Molokai LIVE Newsletter in order to increase participation. To continue to forge partnerships with MHS teachers and athletic department coaches in order to increase high school participation (11% less than middle school) as a requirement or incentive from within the school. Continue with efforts to innovate pathways for Aka‘ula participation such as hosting a remote site and inclusion in activities that lead to increased student opportunities. Program Offerings: Innovate and restructure the delivery of enrichment offerings in order to boost participation in classes. Try a variety of engagement opportunities for students and experiment with setting, length, duration, and frequency. Incorporate into program planning for classes the concepts of learning new skills for females and sharpening skills for males (from Summer 2015 student survey). 3 College and Career Readiness: Build internship and job shadowing opportunities, create a directory of businesses in the community that align to different career pathways, and create internship possibilities within the program in order to create opportunities for students. To increase college and/or career transition services in order to boost participation of high school seniors (10% of total students served compared to 16-21% of other grade levels), and high school students in summer school. Retention: Consider shorter run classes as well as new classes each quarter; as long running classes decrease in attendance as the year progresses. Track matriculation of students from class to class within the program in alignment with grant goals. Capitalize on the feelings of safety (54% MMS, 20% MHS) students find while in the program through the continuation of strategies such as selling snacks, signing in, ample options for seating, patrolling hallways, and having small adult to student ratios. Personnel: Restructure position duties, offer workshops rather than classes, create student internships, and publicize and recruit committed volunteers in order to meet personnel challenges. Continue to provide additional supports (in the form of personnel that can accomplish selected tasks) for the site coordinator, freeing time to continue the high level of support for instructors. Train staff in relationship building strategies appropriate for high school students in order to increase indicators for “a sense of belonging” with this age group (from “I sort of like the program, it’s okay” to “I really like the program, it’s great”). Also for this indicator, build up social opportunities for enrichment that appeal to high school students. Family Engagement: Create more opportunities during the program’s Family Learning Center time to include College Bound Families workshops, individual or small group support, STEAM Emporium, typing, reading skill building, and technology classes in order to build family engagement. Evaluation: Significantly (10%) more students improved grades in both math and language arts when they had 30 days of attendance accrued by the Fall term rather than the Spring term. This suggests that early regular attendance in Molokai LIVE is more likely to favorably impact grades, underscoring the importance of timely formative data in order to make program improvements 4 PROGRAM DESCRIPTION ORIGIN In its second year, Molokai LIVE continues deliveries at its designated sites: Molokai High School, Molokai Middle School, and Aka‘ula School in a collaborative effort to support the school’s academic and student focuses. MHS and MMS are the only public schools responsible for secondary education on Molokai. Aka‘ula School, a private school serves students in the 5th - 12th grade. Molokai LIVE is based in the library shared by Molokai High School and Molokai Middle School less than one mile away from Aka‘ula School. The 2015-16 combined enrollment was 336 students, of which the largest ethnic group was Native Hawaiian (81%), followed by Asian (primarily Filipino) (15%), this data approximately mirroring Molokai’s population of 7,258 residents (2010 U.S. Census). There were also 155 adults and family members served, both in classes and family College and Career Readiness events. Rural and isolated, Molokai has a unique set of strengths and barriers that set it apart from the rest of the state. The island has earned a statewide reputation for fiercely protecting its resources and a lifestyle rejecting of industry and tourism. At the same time, the educational system is tasked with helping the students on Molokai catch up in 21st Century educational literacy and preparing students to participate in the larger global community. The barriers of economic hardship, limited life experiences, and the deprivation of enrichment opportunities