Activity Name MADE Training Sessions Date Dates throughout Tranche 3 (September 2018 – July 2019) Location 31 target schools and colleges across , , and Somerset (see schools involved) Total number of ≈5500 Students students Sessions also delivered to ≈450 Parents and Carers, and to ≈ 250 teachers Number of target ≈1750 Target Students students Schools involved in All Saints , College, , Brannel the event School, Bridgwater and Taunton College, Buckler's Mead Academy, , Chilton Trinity, Clyst Vale Community College, , College, River Academy, , , Holyrood Academy, Honiton Community College, , , , , Robert Blake Science College, St James School, St Peter’s CoE Aided School, Teignmouth Community School, The Blue School, , , The Taunton Academy, , Academy, Whitstone School

Introduction Throughout Tranche 3, MADE Training worked in collaboration with Next Steps South West (NSSW) to deliver a range of sessions to target and non-target students in years 9-13 across over 30 schools in Cornwall, Devon, and Somerset. All sessions were designed and delivered by MADE’s professional trainers, and covered a range of topics that support students to progress through their studies and on to Higher Education. Sessions delivered included:

 Fast Track to Success  Goals and Aspirations  Exams MADE easy  Rapid Revision  Studying MADE Simple  Advanced Note Taking Following the delivery model of MADE Training, all sessions were interactive, and encouraged students to participate and take away new skills and resources to support their personal development. In addition to the student workshops, MADE Training also provided sessions for students’ key influencers and motivators. Teacher CPD workshops shared effective revision strategies with school staff, whilst parents and carers sessions provided tools and tactics for this audience group to encourage independent learning and exam preparation outside of school. Aims All sessions aimed to support students to address key barriers identified by NSSW relating to Soft Skills and Socio-economic factors, focusing either on study and revision skills and techniques, or on providing students with tools to make informed decisions about their next steps, increasing their motivation and raising their aspirations. Additionally, the workshops provided for teachers, parents, and carers addressed the barrier of Influencers. By providing tools to help students with active and productive study support in the classroom and at home, the sessions aimed to increase influencers’ understanding of how to motivate and support students towards their educational aspirations.

MADE Training Sessions Report Tranche 3 Eleanor Hodgson (Evaluation Specialist) Page 1

Evaluation Design and Data Collection Evaluation of MADE sessions across the NSSW consortium consisted of two evaluation types:

 MADE-issued feedback forms focusing on process evaluation of sessions  NSSW-issued feedback forms focusing on impact evaluation of sessions, with separate surveys for different audience groups and intervention types

Process evaluation data was collected at all MADE sessions run for NSSW schools during the Tranche, representing a sample of 4101 surveys from students (N=3869), parents and carers (N=160), and teachers (N=72). Impact evaluation data was also collected from students (N=2885), parents and carers (N=116), and teachers and staff (N=30).

In order to triangulate self-reported responses from students in relation to the student-facing sessions delivered by MADE, this report also includes data collected from NSSW In School Coordinators (ISCs) regarding their perception of the student workshops and the impact of these interventions on the pupils in their school.

MADE Process Evaluation Data

MADE-issued surveys asked participants in all sessions (including teachers, parents and carers) four questions to gather feedback on how well the workshops had gone. Figure 1 presents responses to each question from the three audience groups engaged in sessions.

MADE Process Evaluation Surveys 4% 2% 1% 1% 1% 4% 2% 5% 6% 10%

95% 96% 98% 94% 99% 99% 99% 100% 100% 90% 96% 98%

Students Parents Teachers Students Parents Teachers Students Parents Teachers Students Parents Teachers Did you enjoy today’s Was the content useful to Did the trainer explain the Will you be using the workshop? you? material clearly? techniques yourself?

Yes No

Figure 1 – MADE-issued process evaluation surveys

Figure 1 indicates that the significant majority (≥94%) of participants in all three audience groups had a positive experience at the MADE sessions, as shown by the proportion who agreed (‘Yes’) that they enjoyed the workshop and that the content was useful to them. Overall, teachers were the most positive in their responses, although the smaller sample size of this group should be taken into consideration when comparing the data in Figure 1.

100% of teachers, parents, and carers, and 99% of students agreed that the trainer had explained material clearly. Furthermore, 99% of teachers, parents, and carers and 94% of students agreed that the content was useful. The only area in which the data suggests improvements could be made is regarding participants’ intentions to use techniques shown in the workshop, as 10% of students stated that they would not be using techniques themselves. Nevertheless, this data presents a very positive picture of the sessions, particularly given the

MADE Training Sessions Report Tranche 3 Eleanor Hodgson (Evaluation Specialist) Page 2 large student sample size, and suggests that MADE successfully delivered appropriate content to support the respective audience groups in a manner that was beneficial to them.

MADE Impact Evaluation Data

In order to gather more nuanced data surrounding the potential impact of MADE sessions on participants, NSSW created separate surveys that were issued to students, teachers, and parents and carers who attended workshops throughout Tranche 3. Two different impact evaluation surveys were distributed to student participants, depending on the focus of the workshop they attended (Study Skills or Aspirations), both gathering pre- and post-activity data in order to ascertain impact. The following sections present data from each separate survey, with analysis highlighting key findings and the evidence they indicate in relation to MADE aims and the NSSW barriers to HE.

Aspirations Survey (Students)

This survey was issued during workshops with a focus on raising students’ aspirations for the future (such as ‘Goals and Aspirations’), and on equipping them with the skills needed to start planning their future career paths (such as ‘Fast Track to Success’ and ‘The Lighthouse’). Students were asked to complete two sections reflecting their attitudes and perceptions before and after the workshop, and Figures 2 and 3 present the data from 1881 survey responses.

Before today's event (Aspirations) 54% 50% 42% 40% 36% 28% 29% 30% 22% 23% 17% 18% 20% 10% 9% 10% 3% 4% 2% 4% 0% I felt confident in planning the next steps I was motivated to do well at I had already thought about going to in my future study/career path school/college University/Higher Education

Strongly agree Agree Unsure Disagree Strongly disagree

Figure 2 – NSSW Aspirations Pre-survey

After today's event (Aspirations) 53% 54% 50% 50% 49% 50% 44% 40% 33% 30% 27% 31% 29% 30% 26% 17% 19% 19% 20% 15% 15% 13% 5% 10% 1%1% 2%1% 3%1% 3%1% 3%1% 3% 0% I am better able to I am better able to I am more confident I am more I am more I am more likely to set goals for the make decisions in planning the next interested in motivated to do consider going to future about my future steps in my future learning new well at University/Higher study/career path information and school/college Education skills

Strongly agree Agree Unsure Disagree Strongly disagree

Figure 3 – NSSW Aspirations Post-survey

Although students reported confidence in career planning (59% agreement) and motivation to do well at school/college (76% agreement) before the workshop, comparison of Figures 2 and

MADE Training Sessions Report Tranche 3 Eleanor Hodgson (Evaluation Specialist) Page 3

3 suggests that these workshops had an impact on these areas. After the sessions, 77% of students agreed that they were more confident to plan their next steps, with 80% agreeing that they were better able to make decisions about their future, and 83% feeling more able to set goals. Students’ motivation to do well at school or college also increased to 82% (+6%), with 81% reporting that they were more interested in learning new information and skills.

This data indicates that these workshops may have had a positive impact on the barrier of Socio-economic factors, particularly surrounding students’ motivation and aspirations for the future. Further evidence of impact is shown by the increase in the proportion of students indicating that they are considering HE study. Whilst 65% agreed that they had thought about higher education before the workshop, 73% agreed or strongly agreed that they were more likely to consider going to university or HE study after the sessions, with only 8% disagreeing or strongly disagreeing with this statement.

Study Skills (Students)

For workshops with a focus on developing students’ study skills, such as ‘Exams MADE Easy’ and ‘Rapid Revision’, it was deemed appropriate to issue students with a pre- and post-activity survey that evaluated impact on the barrier of Soft Skills. Figures 4 and 5 present the data collected from 1007 surveys.

Before today's event (Study Skills) 45% 50% 42% 40% 35% 34% 34% 29% 30% 17% 20% 13% 10% 10% 10% 10% 10% 3% 3% 5% 0% I felt confident in my ability to study I felt confident in being able to achieve I had already thought about going to independently and revise for my exams the grades I need for my future plans University/Higher Education

Strongly agree Agree Unsure Disagree Strongly disagree

Figure 4 – NSSW Study Skills Pre-survey

After today's event (Study Skills) 57% 60% 53% 51% 49% 50% 40% 35% 30% 29% 31% 30% 26% 22% 23% 24% 20% 16% 10% 12% 9% 7% 10% 5% 2% 1% 2% 1% 3% 1% 1% 0% I am more confident in I know more about how I am better able to study I am more confident in I am more likely to my ability to study to revise effectively for well on my own achieving the grades for consider going to independently and exams the next stage in my University/Higher revise for my exams study/career path Education Strongly agree Agree Unsure Disagree Strongly disagree Figure 5 – NSSW Study Skills Post-survey

Comparative analysis of data in Figures 4 and 5 suggests significant impact on students’ confidence in studying for and achieving their desired grades in exams. Although 58% agreed

MADE Training Sessions Report Tranche 3 Eleanor Hodgson (Evaluation Specialist) Page 4 or strongly agreed before the workshop that they were confident in studying independently and revising for exams, 87% agreed that they felt more confident in this area after the sessions. Additionally, 88% agreed or strongly agreed that they knew more about how to revise effectively for exams, with 80% stating that they felt better able to study on their own after the workshops. It can be argued that this data indicates that the sessions helped students to address the barrier of Soft Skills by developing their ability to study independently.

Furthermore, the data shows that these workshops also increased students’ confidence in achieving the grades they need for their planned study or career path. 71% of students agreed or strongly agreed that they felt more confident in this area, showing an increase of 19% compared with responses to the matched pre-workshop statement. Thus, it can be suggested that the sessions may also have helped to raise students’ aspirations for their next steps, increasing their belief in their ability to achieve their future plans and addressing the barrier of Socio-economic factors to HE progression.

Conversely, there was a decrease in the proportion of positive responses relating to students’ intentions to consider Higher Education after the workshops in these Study Skills surveys. Whilst 68% reported that they had been considering HE or university before the sessions, only 57% stated that they were more likely to consider this option after the intervention, and 19% indicated that they were less likely to consider HE after the workshop. This change in responses could indicate that the sessions may have in some way dissuaded students from HE study. However, given the wording of the pre- and post-workshop statements, it could also be argued that, whilst data shows that students were not ‘more likely’ to consider HE, students were not necessarily less likely to consider university study as a result of the intervention. Moreover, it is important to note that the sessions evaluated by this survey focused on study skills, rather than specifically increasing students’ knowledge of and aspiration towards HE, and these findings should therefore not be considered to be an area of particular concern. Parents’ Session Survey

Parent and carers’ workshops focused on providing these influencers with techniques to support students with GCSE preparation and planning their next steps. Figure 6 presents results collected in 116 surveys which were designed to evidence the impact of these sessions on parents and carers’ confidence in supporting their children.

Parents and Carers - Confidence to support child after MADE session

4% 15% 2% 25% 1% 5% 67% 65% 56%

29% 18% 13% After today's session, how confident do you How confident do you feel supporting your How confident do you feel supporting your feel supporting your child with their GCSE child with their study/career choices after child with their study/career choices post revision? GCSEs? 18?

Very confident Quite confident Not that confident Not at all confident Not sure

Figure 6 – Parents’ session impact evaluation survey

Figure 6 indicates that the MADE parent sessions were successful in increasing the confidence of parents and carers to support their children with GCSE revision, with 96% of

MADE Training Sessions Report Tranche 3 Eleanor Hodgson (Evaluation Specialist) Page 5 respondents noting that they were either ‘Very confident’ (29%) or ‘Quite confident’ (67%) in this area after the workshop. In contrast, the data indicates that parents and carers were less confident with regards supporting their child in making their post-16 and post-18 study and/or career choices. A higher proportion (~20%) responded neutrally to the statements relating to these areas, and a small number (~3%) indicating that they were ‘Not that confident’. In particular, Figure 6 shows that parents and carers had lower confidence in supporting their children with post-18 options, with 1% stating that they were ‘Not at all confident’ in this area.

This data suggests that the MADE workshops achieved their primary aim of providing parents and carers with specific tools to increase their confidence supporting their children with revision, thus helping them motivate students towards their educational goals. However, Figure 6 also indicates that some parents would benefit from further information and guidance regarding how to help their children decide their next steps, particularly with post-18 options, including higher education. This data correlates with other data gathered in the survey which showed that 79% of parents and carers did not currently have any other dependent studying at HE level. Overall, this data suggests that NSSW parents and carers may need additional support to be familiar with progression routes for the their children, and although this is not an area directly addressed by the MADE sessions, more information could be built into any future interventions with this audience group.

Teacher Session Survey

In a similar manner to workshops for parents and carers, MADE delivered CPD sessions to teachers during Tranche 3 which aimed to provide these influencers with additional tools to support students with their study and revision skills. Impact evaluation data was collected from 30 surveys at one teacher session, and results are shown in Figure 7.

Teachers - Confidence to support students after MADE session

3% 13% 27% 20%

74% 33% 57% 40% 23% 10% After today's session, how confident How confident do you feel supporting How confident do you feel supporting do you feel supporting students with students with their study/career students with their study/career their GCSE revision? choices after GCSEs? choices post-18?

Very confident Quite confident Not that confident Not sure

Figure 7 – Teachers’ session impact evaluation survey

The results displayed in Figure 7 clearly indicate that these teachers felt very positive (combined 97% ‘Very confident’ or ‘Quite confident’ responses) in their ability to support their students up to their GCSEs after attending the MADE sessions. However, aligning with responses from parents and carers analysed in Figure 6, Figure 7 also indicates that teachers were less confident in supporting students with choosing their post-school study and career paths. Moreover, the data shows that teachers’ lack of confidence (as shown by responses of ‘Not that confident’) increased between giving advice to students on post-16 options (20%) and advising students on post-18 options (33%), with no respondents stating that they were ‘Very confident’ in this area. As with parents and carers, teachers indicated that they were less

MADE Training Sessions Report Tranche 3 Eleanor Hodgson (Evaluation Specialist) Page 6 confident in providing support and guidance to their students about their post-16 and post-18 options, and the measurable decrease in their confidence with each education stage highlights the importance of providing all influencers with support to help them discuss young people’s post-18 career and study options, including higher education. Nevertheless, it is important to remember that the aim of the MADE sessions for both non- student audience groups was primarily to provide these influencers with tools for supporting their children and students with exam preparation, and the data analysed in Figures 6 and 7 suggests that the sessions were successful in meeting this aim. ISC Survey Feedback – Student MADE Sessions In order to counteract self-report bias in the student surveys, we also collected data from NSSW In School Coordinators (ISCs) relating to the student workshops run by MADE Training in Tranche 3. In ISCs surveys conducted at the end of the Autumn, Spring, and Summer terms 2018-19, 22 ISCs provided feedback relating to their perception of the different MADE workshops run in their schools and the impact of these activities on their students. Responding to statements designed to evaluate perceptions of impact across any NSSW activity, ISCs indicated that they believed the MADE sessions were a very positive intervention for the majority of students, as shown in Figure 8.

ISC Survey Feedback - MADE Student Workshops 100% 100% 100% 86% 86% 91% 90% 80% 68% 73% 70% 60% 50% 40% 32% 30% 20% 14% 14% 9% 14%14% 10% 0% The majority of The activity The external staff Students talked I have noticed an It was easy to I would like to students learned raised the who delivered positively about impact from the organise the arrange the same something about aspirations of the the activity were their experience activity on the activity in my activity in my HE or career students who approachable after the activity students (e.g. school/college school/college planning from attended and informative conversations again the activity about HE or careers)

Strongly agree / agree Neither or N/A Disagree / strongly disagree

Figure 8 – ISC Survey MADE Student workshop feedback

This triangulated data supports the findings from the other surveys analysed in this report and highlights the positive reception of MADE workshops, with 91% of ISCs agreeing or strongly agreeing that students talked positively about the sessions. The majority of ISC respondents (86%) agreed or strongly agreed that the majority of students had learned something about HE or career planning and that the activity had raised students’ aspirations, suggesting impact on the barrier of Socio-economic factors to HE progression. Whilst a smaller proportion (68%) agreed that they noticed an impact on students relating to HE as a result of the sessions, it should be noted that this impact is more challenging for ISCs to perceive, and that not all MADE workshops focused specifically on post-18 progression. Figure 8 also shows that although MADE student workshops may not always be easy for ISCs to arrange (14% disagreed that the activity was easy to organise), ISCs are nevertheless very keen to provide this intervention for their students, with 100% agreeing or strongly agreeing that they would like to arrange the same activity in their school again. This positive finding correlates with qualitative data given by ISCs, which included the following comments:

MADE Training Sessions Report Tranche 3 Eleanor Hodgson (Evaluation Specialist) Page 7

 ‘I love the Made Training sessions and so do my students. They are engaged and responsive. The Lighthouse sheets are then useful for staff to promote conversations around GCSE options, careers and HE choices.’ (Autumn survey)  ‘Parents, staff and a majority of students engaged well and thought this was a very useful session.’ (Autumn survey)  ‘We had some difficulties with finding a space big enough for a large group. In hindsight, I think it would have been better to have a smaller group of kids so that we could spend more time with individuals. […] Overall, I feel that the kids gained a lot from the experience.’ (Spring survey)  ‘Fantastic workshop took a fair bit to arrange but well worth it.’ (Summer survey)  ‘The external staff were fantastic and the whole session was great in raising aspirations.’ (Summer survey)  ‘Good activity to get Year 9 talking about future plans and then enables a discussion with tutors during mentoring sessions referring to the worksheets.’ (Summer survey)  ‘Brilliant session run by MADE. Would have been easier to organise if the year group didn't need to be split in half.’ (Summer survey) This qualitative feedback echoes findings in the student surveys analysed in this report, as well as the quantitative data from ISCs, which indicate that the MADE sessions were engaging and impactful for those who attended. Conclusion and recommendations Analysis of process and impact evaluation data from students, teachers, and parents and carers provides a range of evidence to suggest that MADE Training sessions delivered throughout Tranche 3 were well run, engaging, and beneficial for the significant majority of those who attended. Feedback from all audience groups shows a very positive reception for the workshops. 100% of surveyed ISCs noted that they wished to arrange MADE interventions in their schools in the future, and their qualitative comments highlighted the longer-term impact of sessions for students, who were able to build on and reuse the material covered during the workshops elsewhere in school (for example in mentoring sessions with tutors). Overall, it can be strongly argued that the MADE Training workshops met their aim of supporting students to develop their study and revision skills and to raise their aspirations towards and ability to plan for their future study and career goals, whilst also helping key influencers with their confidence to support young people preparing for exams and further study. This data highlights potential impact on the barriers of Soft Skills, Socio-economic factors to HE progression, and Influencers, as indicated throughout this report. The only areas for concern raised by analysis of the data relate to some comments from ISCs regarding organisational and logistical challenges in arranging the sessions in schools. It is recommended that IOs and ISCs organising MADE Training workshops in Phase 2 are aware of these potential difficulties, and liaise with schools to ensure that they are aware of the logistical requirements for the event well in advance. Whilst some data in the impact evaluation surveys of teacher CPD sessions and parents and carers workshops indicates that influencers were lacking in confidence regarding how to support students with post-16 and post-18 career and study choices after the MADE interventions, it should be noted that the workshops did not aim to specifically address this area of influencers’ knowledge and confidence. Indeed, the primary recommendation to be made regarding this data is to review the evaluation in place for any MADE Training interventions delivered to influencers in Phase 2. Some of the data analysed in the impact surveys distributed to influencers did not directly relate to the aims of these sessions, thus resulting in some of the negative findings discussed above. For Phase 2, it is recommended that impact evaluation is refined so that it closely aligns with the specific session objectives for

MADE Training Sessions Report Tranche 3 Eleanor Hodgson (Evaluation Specialist) Page 8 this audience group. Furthermore, it would be beneficial to collect qualitative data from students in order to better explore the impact of the workshops, such as asking students to provide examples of specific skills or techniques they have learnt. Aside from these points, the data indicates that MADE Training sessions have supported NSSW students in addressing key barriers during Tranche 3, and it is hoped that they will continue to do so with the core offer MADE provision under the Phase 2 Progression Framework.

MADE Training Sessions Report Tranche 3 Eleanor Hodgson (Evaluation Specialist) Page 9