Final Evaluation of Building the Future Together

Final Report

July 2013 Final Evaluation of Building the Future Together wavehill

Report prepared by: Simon Hartwell Oliver Allies

Any queries or questions about this report should be referred in the first instance to Oliver Allies at Wavehill: t: 01545 571711 | e: [email protected]

Client contact: Diane Hall/Allyson Griffiths 01443 680599/01443 680596 [email protected]/[email protected]

Report version: Final

Contents

Executive Summary...... 4

1. Introduction ...... 10 1.1. An Introduction to Building the Future Together Project...... 10 1.2. The Evaluation...... 11

2. Project Provisions and the Place of BTFT...... 14 2.1. BTFT Provisions ...... 14 2.2. The place and value of BTFT provisions...... 17

3. Project Progress and Outcomes...... 19 3.1. Overall Progress against key indicators ...... 19 3.2. Cross cutting themes ...... 20 3.3. Spend ...... 235

4. Project Management, Processes and Delivery ...... 26 4.1. Managing BTFT: The Central Team ...... 26 4.2. Project Administration and Evidencing...... 28 4.3. Working on a funded project...... 30 4.4. Embedding BTFT School Provisions ...... 31 4.5. Referral processes...... 32 4.6. College Provisions ...... 32

5. The Impact of BTFT ...... 33 5.1. Impact on Participants ...... 33 5.2. Wider Project Impacts ...... 54

6. Conclusions ...... 59

Executive Summary

Introduction

In May 2013, Wavehill were appointed to undertake a qualitative and quantitative evaluation of the effectiveness of the Building the Future Together Project (herein referred to as BTFT) in achieving its aspirations, aims and objectives. This document represents the final evaluation report.

Building the Future Together was launched in July 2009 and is a £15 million project part funded by the European Social Fund (Priority One). The project provides a participant centred approach to tackling the major issues of disengagement of 11-19 year olds in Rhondda Cynon Taf (RCT). The project seeks to address the barriers to learning that are currently ingrained and systemic by providing a range of integrated services to improve the educational, employment and life outcomes of young people at risk of underachieving. The project is delivered by RCT Council in partnership with Coleg Morgannwg. Although originally scheduled to finish in September 2012, the BTFT project was extended for an extra year (the 2012/13 academic year). The extension of the project was designed as a transition phase between the original project and the possible integration of the roles and functions within core service areas.

Wavehill were commissioned to undertake the final evaluation of the BTFT project in May 2013. A mid-term evaluation was completed by People and Work Unit in April 2012. This means the primary focus of this evaluation is on the period following this, particularly the latest iteration of the project during the extension phase (academic year 2012/13). The overall aim of the final evaluation is to assess the effectiveness of the project. In terms of the focus of the evaluation, as identified in the scoping phase, the evaluation should ‘look beyond’ the hard-edged deliverable outputs of the project (as important as these were seen to be) and also to take account of softer measures of beneficiary progress.

In terms of the main methods of enquiry for this report, overall, there were five key inputs:Participant consultations (199), delivery staff consultations (29), Central Team and Intervention Managers Interviews (7), stakeholder consultations (7) and desk research.

BTFT Provisions

• Youth Coaches (Learning Coaches)work with young people in schools with a focus on Learning Pathways. They have the responsibility of working closely with school staff to develop effective Individual Learning Plans (ILPs). Despite the educational focus, the role has naturally expanded to encompass more of the general support aspect once the number of key workers was reduced as part of the project extension.

• Key Workersidentify and support young people in school, helping to re-engage them with education through a holistic young person centred approach, supporting their learning and any other additional needs. The role of the key worker is to work closely with school staff and multi agency partners to support the young person through a range of interventions available to them.

• Detached Youth Workersoffer street-based engagement and support for disengaged young people, providing one to one and group work, accredited learning and employability opportunities for the most hard to reach young people.

• Positive Futures is an alternative curriculum intervention offering accredited courses for looked after young people, those disengaging from mainstream education, those who cannot engage with mainstream education and the most hard to reach NEETs who have been referred from the Pupil Referral Units.

• The emphasis of Coleg Morgannwg provision is on recruiting and taking referrals of young people who are NEET, and have previously struggled to engage with formal education at an earlier age. Tailored support is provided for these young people in class and includes e-learning. In addition to this, the College offer Gateway Vocational Courses, an Alternative Curriculum to help enable more young people to re-engage with education.

The place and value of BTFT provisions

The justification in terms of need is argued to be strong. As highlighted in the business plan, RCT is an authority with a number of relative challenges in terms of deprivation and attainment in school. This was also expressed by interviewees discussing the place of BTFT. Although some schools within the authority serve areas of higher deprivation than others, it was argued that BTFT is relevant to all schools as there will always be pupils requiring these types of support.

It is clear that support would not have been provided without BTFT- there were no other scenarios available other than the ‘do nothing’ approach. There were limited similar provisions in place prior to BTFT, for example, 6 detached youth workers covering the whole local authority. As such, the value of BTFT was highlighted by respondents as the expansions of provisions, widening access to support across the authority. In addition to the expansion of provision, the BTFT project offers a more ‘joined up’ approach to supporting young people in the authority.

Project Progress and Outcomes

Outcomes

There are five key indicators for BTFT and 3 months short of project end the project has achieved in excess of the target for four of the five key indicators, with more participants engaged than anticipated and more gaining qualifications, entering further learning or entering employment than targeted. The mid-term evaluation highlighted that, although progress was being made to ‘catch up’ from a slow project start, there were still concerns that indicator targets would be met. As such, the evidence presented here represents the overcoming of this challenge by the management and delivery staff on the project.

Spend

Financial data submitted by the project to WEFO in April 2013 (Claim report) demonstrates that the project is on target to achieve its financial indicators. It must be noted that this only indicates the project has progressed well in relation to the latest version post re-profiling, where finances were re- assessed and include actual figures from previous periods.

Cross cutting themes

The project has adhered to existing local authority policies and EU requirements regarding issues relating to equal opportunities and environmental sustainability and proactively worked with the participants to put policies into practice. The mid-term evaluation highlighted that more work could be undertaken by the project relating to STEM qualifications. This was taken on board and the Central Team co-ordinated activities and events to expand STEM provisions. Taken together these represent a significant expansion of STEM related provision and address this shortfall.

In terms of the participants, there are no significant discrepancies between the demographic characteristics of those engaged and the overall population of the authority, enough to state that the project has not excluded any one group and has ‘fair’ representation of different groups. In addition to this, broadly the participant breakdown of each indicator reflects the overall characteristics of the participant population. This demonstrates that impacts are felt across groups, with no one group excluded from achievement.

Project Management, Processes and Delivery

Managing BTFT: The Central Team

The Central Management Team for BTFT are responsible for the overall management and co- ordination of the project. The overriding emergent theme during consultations regarding the role and effectiveness of the team was that the management of the project has ‘settled’ in the last year or so (especially since the extension period) and is currently at its most efficient and effective. There are a series of factors driving the more effective management of the project in recent years including an increased stability within the team, the change in internal LA placement of Central Team, the streamlining of the project, afocus on compliance and a general project settling.

Project Administration and Evidencing

Project administration within a European context and a group of interventions this size is always going to be a challenge for both those establishing and managing the processes and those delivering interventions. In the case of BTFT, the administrative requirements - and the fulfilling of them by staff- emerged as a key issue that has been present throughout BTFT, but one which has improved in the past year with the current management team in place. There are two aspects to this issue: the clarity and consistency of guidance and processes, and the abilities of staff to complete requirements. Administration was a key issue emerging from the BTFT project that has impacted on the project itself. This could be mitigated in future projects partly by providing training at the outset of the project for all delivery staff. This would cover all administrative duties including the evidencing of outcomes and help to give clarity and confidence to delivery staff. In addition to this, the project would have benefitted from a period of pre delivery where processes and guidance was fully developed and in place prior to delivery. If the management team were in place prior to delivery for a suitable period of time, this would enable processes such as those relating to evidencing outcomes to be fully developed.

Working on a funded project

Delivery staff who were working on a funded project for the first time having previously been core staff within the authority, and in some cases, undertaking a similar post/ role for a number of years, found the change of culture and requirements challenging. The nature of working on a funded project, with a specific end date, was raised as a challenge for both delivery staff and line managers. With relatively short term contracts, there is often uncertainty which can hit morale, particularly near the end of funded contracts where it is unclear whether jobs are going to be safeguarded moving forward.

Embedding BTFT School Provisions

A series of challenges around embedding BTFT provisions, particularly within schools were raised by a range of staff consulted (managers and delivery staff across all school interventions). Although more of an issue earlier on the in the project, this was raised as a key issue in the context of the project as a whole and something that should be recognised in the final evaluation. There are two central facets of the challenge: becoming ‘accepted’ and understood by school staff (e.g. teachers) and becoming embedded within the school itself.

Referral processes

With BTFT, broadly, participants are targeted if it is believed that they are at risk of becoming disengaged with education. Various risk factors were utilised to identify relevant young people, the most common being low attendance, risk of under-achievement, lack of basic skills or if the young person was of concern to education welfare. Overall, the perception amongst project staff at all levels and other stakeholders was that the referral systems were effective and that the right young people were being engaged.

College Provisions

Generally, fewer challenges were experienced with the Coleg Morgannwg arm of the project, primarily this was identified to be because the provisions have been largely consistent throughout the project lifetime and focused on one institution (the college) rather than a group (21+ schools). Delivery staff consulted did not identify any specific issues in undertaking their role other than the challenge of working with the participants themselves. Project management within the college is perceived to be excellent, aiding the smoothness of delivery.

The Impact of BTFT

At the current time, RCT’s Education and Lifelong Learning division are undertaking an extremely in depth and extensive analysis of the wider impacts of the project. As such, the report is considered essential reading alongside this evaluation once it is released shortly after project end. Where it focuses on the use of detailed statistical analysis to explore impact, this report will focus on the extensive consultations with participants and delivery staff- another key indicator of impact.

Impact on Participants

Considerable evidence of impact on participants was found across the project interventions. Many of the impacts relate to ‘soft skills’ such as building confidence and developing a young person’s social skills. What is important is that the evidence shows impact beyond the headline indicators and reflects the level of support provided by the project and the difference it can make to young people at risk of disengaging. What could be perceived as ‘soft outcomes,’ for these young people, represent the intermediary stages on the way to achieving a hard outcome. They included for example, thinking skills such as improved problem-solving abilities, personal attributes such as improved self-confidence, or practical work-focused skills. They also often represent the main outcomes achieved by the most disadvantaged ESF client groups. The results of the participant interviews express a significant gain in soft outcomes, and it becomes evident that young people need to overcome their initial barriers, in order to gain hard outcomes, such as training opportunities, a place in college or employment.

Wider Project Impacts

Impacts were also identified beyond the participant, usually as a result of the impacts on the participant identified previously. Four key areas were found:

• Impact on families and siblings: across all interventions, delivery staff and participants highlighted impact on families and siblings, arising as a product of positive changes in the participants themselves. Mostly, this seemed to be related to the overall improvement in the behaviour and mood of the support young person, but also was sometimes routed in the reduction of pressure on families in terms of supporting the young person.

• A range of interviewees identified wider impacts within the schools in which BTFT operates. The main benefits were argued to be a reduction in the pressure on teaching staff and a reduction of behavioural issues and problems within the classroom.

• Impact on communication and integration within the authority

• There has been a significant mainstreaming of activities and support as a result of the project.

Conclusions

As BTFT moves toward its end, the project is delivering tangible outcomes and benefits to young people in an efficient and effective manner. There is a strong perception that extension period of BTFT (2012/13 academic year) represents the most effective and efficient iteration of the project, supported by the Central Team and processes that are now embedded and working well. With the end of the academic year near, the project has achieved in excess of the target for four of the five key indicators and given the issues raised in the mid-term evaluation and here regarding the slow start and the ambitious aims of the project, this reflects well on those involved in the management and delivery of the project. It is widely recognised that that the management by the Central Team (and Coleg Morgannwg) of the project has ‘settled’ in the last year or so (especially since the extension period) and is currently at its most efficient and effective. The dominant perception was that the central team now performs their role extremely well considering the challenges of managing and delivering the project.

The evidence collected as part of this research has shown that the support provided by BTFT has had a significant impact on participants. In addition to hard outcomes (gaining a qualification, entering education for example), significant evidence of impact on soft outcomes have emerged. The complex and challenging backgrounds of the participants means that support is not being targeted at ‘easy wins’, rather, young people facing many challenges are being supported to achieve both hard and soft outcomes. Based on the evidence and discussion in this report regarding the complex and ingrained challenges faced by participants of the project, the current set of possible outcome indicators is not considered fairly reflective of what represents ‘success’ for this group: staying in education for example is a big challenge which the project may support the participant to achieve.

Recommendations

• Provide training at the outset of similar future project for all delivery staff. This would cover all administrative duties including the evidencing of outcomes and help to give clarity and confidence to delivery staff. • Similar future projects will benefit from a period pre delivery where processes and guidance is fully developed and in place prior to delivery. If the management team are in place prior to delivery for a suitable period of time, this would enable processes such as those relating to evidencing outcomes to be fully developed. • A participant remaining in education and/ or increasing attendance above a certain level (say, 90%) should represent a key (claimable) outcome in future projects with similar provisions to BTFT.

1. Introduction

1.1. An Introduction to Building the Future Together Project

The project seeks to address the barriers to learning that are currently ingrained and systemic by providing a range of integrated services to improve the educational, employment and life outcomes of young people at risk of underachieving. It seeks to empower young people to make a successful transition into employment and /or higher education through raising their skills levels and aspirations. This has been enabled by the appropriate infrastructure allowing effective collaboration between a multi-agency partnership. The major focus is on the most hard to reach and most vulnerable and disengaged young people within RCT, ensuring that they get the help, support and services they need to reengage them with mainstream activity, whether that be education, training opportunities or even employment, once they have gained the necessary skills.

The project is delivered by RCT Council in partnership with Coleg Morgannwg. The seven main objectives are outlined below:

Objective 1 - Supporting young people to re-engage in learning and achieve better attainment levels at KS 3, 4 and 5

Objective 2 - Increase the number of young people engaging in universal and targeted interventions most appropriate to their needs

Objective 3 - Young people are helped and supported to improve or maintain their emotional and physical wellbeing

Objective 4 - Increase the numbers of the most hard to reach groups achieving recognised qualifications

Objective 5 - Increase the numbers of young people (including the hardest to reach groups) progressing onto further learning or the jobs market appropriately qualified.

Objective 6 - Expose young people to a wider choice of opportunities and experiences than possible through school curriculum or other means

Objective 7 - Enabling the accelerated improvement in quality and provision of services

Although originally scheduled to finish in September 2012, the BTFT project was extended for an extra year (the 2012/13 academic year). The extension of the project was designed as a transition phase between the original project and the possible integration of the roles and functions within core service areas. The final year of BTFT project involves a streamlined delivery model (see Chapter 2) and plan supported by ESF grant funding of £1,173,897 and match funding being provided by the Local Authority at the same intervention rate as the original project (56.6% grant: 43.4% match funding).

A total of five interventions currently make up the project:

• Key workers • Youth Coaches (Learning Coaches) • Positive Futures • Detached Youth Workers • Support for students and alternative curriculum provisions at Coleg Morgannwg

1.2. The Evaluation

Wavehill were commissioned to undertake the final evaluation of the BTFT project in May 2013. A mid-term evaluation was completed by People and Work Unit in April 2012. This means the primary focus of this evaluation is on the period following this, particularly the latest iteration of the project during the extension phase (academic year 2012/13). The overall aim of the final evaluation is to assess to effectiveness of the project, and to this end, this report has been thematically categorised to cover:

• Chapter 2: Project Provisions and the place of BTFT • Chapter 3: Progress and outcomes • Chapter 4: Project Management, Processes and Delivery • Chapter 5: Impact (on participants and beyond)

Chapter 6 draws conclusions from the previous evidence and thematic discussions.

The initial phases of the evaluation consisted of a series of in depth scoping interviews with each of the project’s intervention managers and the central team (7 were completed in total). The scoping interviews helped the evaluators gain an overview of the project’s background and initial development, gain a detailed insight into the models of delivery for each intervention, and identify issues for consideration in the evaluation.

In terms of the focus of the evaluation, interviewees were unanimous that they wished the evaluation to ‘look beyond’ the hard-edged deliverable outputs of the project (as important as these were seen to be) and also to take account of softer measures of beneficiary progress, such as confidence, motivation, attitudinal changes and changes in participants’ relationships with others by talking to participants. This was the key theme emergent from scoping interviews about the direction the research should take and as such drives the participant focus of the design of the research. These scoping interviews, along with an analysis of project management information and the research brief, drove the development of anevaluation framework, which set out the focus and design of the research.In terms of the main methods of enquiry for this report, overall, there were five key inputs:

Participant consultations (199)

A total of 199 participants were consulted during site visits conducted in June, covering all interventions. Due to the extremely tight timescales for delivery of the evaluation (taking into account the commission date, end of school term and report delivery date), and despite every effort being made, fewer consultations were conducted with Positive Futures Participants than desirable.

A flexible approach to consultation with participants was adopted by the research team. Managers for each intervention were asked how best they felt the consultation should be delivered (based on the knowledge of the particular participants available for consultation). In summary the consultations sought to gather:

• An overview of the background of a participant; • Perceptions of the quality of the support received and any improvements that could be made; • Self-perceptions of the impact of the support received; • Data to test the “counterfactual” – what participants would be doing without BTFT support, and; • Any other comments or observations participants might have.

Delivery staff consultations (29)

In depth face to face interviews were conducted with a total of 29 staff involved in the delivery of BTFT (i.e. those who interact directly with participants) across all interventions. These interviews were conducted during site visits and included a strong focus on the impact of support on participants, but also covered issues relating to the management and delivery of the project.

Central Team and Intervention Managers (8)

In addition to delivery staff, three members of the Central Team and all five intervention managers were interviewed. These interviews covered both the scoping and main evaluation questions.

Stakeholder consultations (7)

Seven in depth telephone interviews were conducted with head teachers of schools in RCT and strategic managers sitting on the project board. These interviews were bespoke, focusing on any issues the interviewee was able to discussion from an informed standpoint.

Desk research

Finally, extensive desk research activities were undertaken to analyse and explore a range of data sources, includingthose collected as part of the research and project management and monitoring data.

2. Project Provisions and the Place of BTFT

This Chapter explores the provisions in place as part of BTFT in more detail and draws out the key differences between them, including an introduction to the approaches undertaken to engaging and supporting participants. In addition to this, the discussion that follows explores the ‘place’ of BTFT – how it creates added value within RCT.

2.1 BTFT Provisions

As noted in the introduction, for the 2012/13 academic year extension of BTFT, there are five key interventions. The general approach and aspirations of the provisions are summarised below, with more detail provided on the exact engagement approaches in Chapter 4 during discussions of impact.There are obviously some similarities in these approaches across the project, but this section helps to understand how the provisions and approaches are different and distinct whilst remaining coherent.

2.1.1 Youth Coaches (Learning Coaches)

BTFTYouth Coaches (often referred to as Learning Coaches) work with young people in schools with a focus on Learning Pathways. They have the responsibility of working closely with school staff to develop effective Individual Learning Plans (ILPs). These plans reflect the young person’s individual needs and provide the structure for support, guidance, coaching and mentoring by setting and working towards individual targets. Despite the educational focus, the role has naturally expanded to encompass more of the general support aspect once the number of key workers was reduced as part of the project extension.

Youth Coaches work closely with BTFT Key Workers to identify and work towards removing barriers to learning and promoting young people’s active participation in positive social and educational activities. By supporting them to develop their learning skills they aim to maximise their development and fulfil their potential. Youth coaches also commonly act as a bridge/ conduit to other support services, both BTFT and external to the project (for example, social services).

Participants are referred by school staff – heads of department, heads of year, heads of key stage, pastoral or senior management team as appropriate; based on CATS/PASS test results, progress in class, NC levels achieved or predicted GCSE grades using a standard referral process that identifies need.

The extension period of BTFT includes provision for 21 youth coaches (one in each school), funded for 15hrs per week. In a number of cases, this is ‘topped up’ by the school to make the post full time. Previously, BTFT provided one full time post per school.

2.1.2 Key Workers

Key workers identify and support young people in school, helping to re-engage them with education through a holistic young person centred approach, supporting their learning and any other additional needs. The role of the key worker is to work closely with school staff and multi agency partners to support the young person through a range of interventions available to them. The young person’s development needs are addressed through one to one and group sessions enabling them to take ownership of their learning and to break down any barriers which prevent them from achieving their potential.

The majority of referrals to key workers are made by schools, with additional referrals via other support services such as Children’sServices and the Youth Offender Service. With Learning Coaches more academically focused, and Detached Youth Workers working with the most disengaged young people, Key workers can be seen as bridging the gap between the two.

The final year extension of BTFT includes provision for 3 Key workers, working geographically across the authority. The previous iteration of the project included 22 Key workers (one placed within each school) and so this represents a significant reduction in resource.

2.1.3 Detached Youth Workers

Detached Youth Workers offer street-based engagement and support for disengaged young people, providing one to one and group work, accredited learning and employability opportunities for the most hard to reach young people. Like the Key Workers and Youth Coaches within the schools, the Detached Youth Workers support tracking of young people who are deemed at risk of NEET etc. with key partners to ensure support pathways into education, training and employment opportunities in line with each young person’s needs.

The team’s flexibility and mobility makes it possible to develop positive relationships with those who, for a variety of often complex reasons, do not use centre-based provision, such as youth centres and E3+. The street is the main focus of their work.

The extension for BTFT includes provision for 15 detached youth workers.

2.1.4 Positive Futures

Positive Futures is an alternative curriculum intervention offering accredited courses for looked after young people, those disengaging from mainstream education, those who cannot engage with mainstream education and the most hard to reach NEETs who have been referred from the Pupil Referral Units. The provision focuses on both Looked After Children and those at risk of going into care with specialist interventions combining social care, youth work and outdoor education theories.

Funding through BTFT provides activities such as:

• Therapeutic group and individual work to learn through outdoor and other activities. • Academic and other vocational qualifications such as outdoor education qualifications approved by national governing bodies, e.g. the British Canoe Union, BTECs, OCN, ASDAN etc. • Opportunities to re-engage young people into mainstream provision and link them into other appropriate provisions, such as further learning and employment.

The majority of referrals come directly from schools or school based support systems (such as BTFT Youth Coaches and Key workers).

2.1.5 Coleg Morgannwg Provisions

The emphasis of Coleg Morgannwg provision is on recruiting and taking referrals of young people who are NEET, and have previously struggled to engage with formal education at an earlier age. Support targets participants who on initial assessment have

In addition to this, the College offer Gateway Vocational Courses, an Alternative Curriculum to help enable more young people to re- engage with education. The programmes are practical, hands on in nature and offer the most disaffected young people a structured integrated programme of vocational training coupled with alternative activities, providing them with valuable work experience and qualifications.

2.2 The place and value of BTFT provisions

A key question which links to the nature of the provisions emerges: is/ was there a place for BTFT in RCT- i.e. where does the value lie? Although a great deal of the value is highlighted in the Impact Chapter of this report, it is important to address the question of the wider fit and value of the project.

Firstly, the justification in terms of need is argued to be strong. As highlighted in the business plan, RCT is an authority with a number of relative challenges in terms of deprivation and attainment in school (see Business Plan V7). This was also expressed by interviewees discussing the place of BTFT:

“This type of support is absolutely crucial in an authority like RCT where there are high levels of deprivation and a huge number of pupils. There is a big strain on the school system and [the BTFT interventions] represent a valuable resource, teachers for example, do not have the time to dedicate to providing this support to young people”.

“It is a really deprived area and the funding has helped a phenomenal amount”.

Although some schools within the authority serve areas of higher deprivation than others (where there may be a greater number of ‘typical’ participants- see Section 3.2.2), it was argued that BTFT is relevant to all schools as there will always be pupils requiring these types of support. BTFT was seen to also reach young people who may not face multiple challenges (e.g. a disruptive home life, drug and alcohol issues), but were in need of support due to issues such as low confidence and self esteem.

If the support is needed, as is suggested, would it have been provided elsewhere during BTFT’s lifetime? In this case, it was clear that there was no other scenarios available other than the ‘do nothing’ approach, it was confirmed that provisions would not have been funded elsewhere during the project lifetime. Although some BTFT provisions are to be mainstreamed at the end of the project (see Section 5.2.4), this has been driven by the value of the interventions to schools:

“The value of this type of intervention is widely recognised (e.g. within most schools), but the other funding is not there to support them”.

There were limited similar provisions in place prior to BTFT, for example, 6 detached youth workers covering the whole local authority. As such, the value of BTFT was highlighted by respondents as the expansions of provisions, widening access to support across the authority. Considering the size of the authority and the number of schools and pupils, the expansion of services equates to equal access to support (for example, one youth coach in each school) across the area, whereas the limited pre BTFT provisions could not provide widespread access:

“Key benefit expansion of the Detached Youth Workers provision from only 6 to 20 (though down to 15 now). This is huge expansion of capacity and resource".

“BTFT has expanded our provision massively, allowing us to cover the whole geographical area effectively”.

In addition to the expansion of provision, the BTFT project offers a more ‘joined up’ approach to supporting young people in the authority. This was identified by the mid-term evaluation, which highlighted a greater integration of services arising as a result of the project:

“Evidence strongly demonstrates that the project has had a positive impact upon the integration of services, most notably through improving referral processes and communication between services, settings and interventions”.

In the time since the mid-term evaluation, relationships have further developed between key parties, for instance, schools and youth workers.

3. Project Progress and Outcomes

This chapter explores the progress of BTFT toward achieving its stated outcomes/ indicators. It also covers cross cutting themes and explores the demographics of participants and those who achieved the key outcomes. Data for the project is held on a central participant database, which was utilised for this analysis. This allows clear disaggregation and comparison of variables such a demography and achievement, but does not include a mechanism by which progress and achievement can be disaggregated by intervention type.

3.1 Overall progress against key indicators

There are five key indicators for BTFT, summarised in table 3.1, below. With the project nearing closure, progress is assessed against final (re-profiled) indicators. The database is complete to July 2013, 3 months short of project end, but near to the end of the school/ college term for the year (and so very close to the final figures).

Table 3.1: Progress against key indicators

End of Project Achievement % achieved target to date Total Participants 11080 11243 101.5 No. Gaining qualifications 11 – 19 3440 3528 102.6 No. Entering further learning 16 – 19 678 685 101.0 No. Gaining other positive outcomes 11 – 19 5021 4587 91.4 No. Entering employment – 16 – 19 28 32 114.3

Table 3.1 shows that the project has achieved in excess of the target for four of the five key indicators, with more participants engaged than anticipated and more gaining qualifications, entering further learning or entering employment than targeted. The numbers gaining other positive outcomes is approximately 10% short of the final target, however, at the time of this report the final three months of the project outcomes are yet to be accounted for.

It is worth noting that these are re-profiled indicators from the original targets set in the business plan in 2009. Reductions in targets were agreed mid way through the project as a result of delays in the project starting delivery and over optimistic figures initially submitted (see mid-term evaluation). The mid-term evaluation highlighted that, although progress was being made to ‘catch up’ from a slow project start, there were still concerns that indicator targets would be met. As such, the evidence presented here represents the overcoming of this challenge by the management and delivery staff on the project.

In addition to this, as noted by a number of the staff consulted, the extension period has represented a period whereby there has been a significant increase in the focus on hard outcomes within the project. With this added pressure, the results achieved to date reflect a project that is ‘in full flow’ and able to reach stiff targets. It was also generally agreed that this additional focus on targets was underpinned well by the central team (see Chapter 4) and that the settling of this function has enabled the project to reach these goals.

3.2 Cross cutting themes

The project has adhered to existing local authority policies and EU requirements regarding issues relating to equal opportunities and environmental sustainability and proactively worked with the participants to put policies into practice, integrating them as far as possible into the mainstream operation, delivery and monitoring of the project.

3.2.1 Equal Opportunities

The project set out ambitions to deliver against the equal opportunities objectives set out for the convergence programme:

Objective 1: Increase the number of individuals who have multiple disadvantages accessing employment and self-employment:

The project engaged with 9,901 participants who, at the time of engagement had no qualifications whilst a significant proportion of these (7,107 or 72%) were under 15 and thereforeyet to meet the NQF Level 2 threshold, the remainder (28%) were 15 or over and many of these were already disadvantaged by the fact that they hadn’t gained NQF Level 2 following Year 11 exams. Furthermore, 710 participants (7%) were described as economically inactive and could therefore be considered as NEETand therefore at multiple disadvantages towards accessing employment.

Multiple project interventions, developed through collaboration with a host of other partner organisations included work experience, social responsibility, key skills and mentoring schemes, combined with basic skills provision and access to further or higher education have helped facilitate access to employment and self-employment amongst these target groups.

Objective 2: Increase the number of women, black and minority ethnic people and disabled people securing training and employment in higher paid and higher skilled sectors and self-employment:

The project supported 66 participants of black or minority ethnic origin to gain qualifications and supported 14 participants of BME origin into further learning. A further 103 BME participants secured other positive outcomes whilst one BME participant successful secured employment as a result of the BTFT project.

The participant database has been analysed in greater depth to explore the demographics of those achieving the key indicators summarised above. This helps to assess if the project is inclusive.

In terms of the participants, the breakdown of the group overall is given in table 3.2, below. There are no significant discrepancies between the characteristics of those engaged and the overall population of the authority, enough to state that the project has not excluded any one group and has ‘fair’ representation of different groups. These figures can also be used to analyse the extent to which each group has achieved within the project, and lead to a further assessment of engagement. To this end, the remaining columns in the table breakdown the characteristics of those participants achieving key outcomes.

Broadly the participant breakdown of each indicator reflects the overall characteristics of the participant population. This demonstrates that impacts are felt across groups, with no one group excluded from achievement.

Table 3.2: Indicator breakdown Characteristics

% of those % of those % of those % of those gaining % total who gained entering Gender entering another participants a further employment positive qualification learning outcome Male 48.8 49.0 43.8 56.5 47.5 Female 51.2 51.0 56.3 43.5 52.5 Black/Minority Ethnic Group? No 97.6 98.1 96.9 98.0 97.7 Yes 2.4 1.9 3.1 2.0 2.3 Considered Disabled No 95.3 95.8 93.8 97.4 94.1 Yes 4.7 4.2 6.3 2.6 5.9 Migrant? No/Not Given 99.7 99.8 100.0 99.9 99.7 Yes - EU 0.3 0.2 0.0 0.1 0.2

Objective 3: Challenge occupational segregation by increasing the number of women and men training or re-training in non-traditional areas, focusing on areas where there are skills shortages:

The mid-term evaluation highlighted that more work could be undertaken by the project relating to Science Technology Engineering and Maths (STEM) qualifications (STEM subjects are typically considered as non-traditional subject areas for training amongst women). The findings from the mid- term evaluation were taken on board and the Central Team co-ordinated activities and events to expand STEM provisions. Taking all factors into consideration this represent a significant expansion of STEM related provision and address this shortfall.

Table 3.3 below provides a breakdown of the numbers and proportion of participants who received training in Science, Technology, Engineering or Maths (STEM) subjects. The table highlights that 10% of females participated in training in STEM subjects compared to 15% of male participants. Whilst males therefore remained more likely to participate in STEM subjects than females, the variance of just 5 percentage points suggests a more balanced gender representation than typically encountered.

Table 3.3: Gender Analysis of Training in STEM Subjects

Participant type Number of Participants Percentage of Copyright Wavehill Ltd 2013 Page | 21 Final Evaluation of Building the Future Together wavehill

female/male participants Total Females 5490 n/a Total Males 5753 n/a Total Females training in STEM subjects 552 10% Total Males training in STEM subjects 850 15%

Activities undertaken by the team to encourage females to train in STEM subjects included the following:

• 43 Young females attended GirlsGetSet (GGS) event, which was a GE initiative to encourage young girls to consider a career in Science, Engineering & Technology (SET). Through an exciting mix of activities and tours which were led by interns undertaking a work placement, the girls were given an insight into GE business and the aviation industry. Activities on the day touched upon real applications of topics covered in school.

• 18 young people from St.Johns took park in a STEM – Multi Media Project. The sessions had been designed to give learners practical, hands on experience of multi-media technology, film and editing software, sound recording and story development linked to themes taken from STEM subjects. They were given the opportunity to present their own ideas and stories as a series of short digital films linked to STEM and they were given the brief of “30 second STEM”.

• 37 female pupils attended an event “Building Bridges – Working together with Capita Symonds and Wales MillenniumCentre” – the project created an arts and science based project, the day consisted of various workshops and the participants had the chance to meet the inspirational architects/engineers/sound engineers.

• 18 students participated in a project with the Big Learning Company, the company having won a tender from Channel 4 were working with the young people creating Apps and short films, which we have termed “30 second STEM” based around their chosen STEM themes.

• 24 participants attended a Robotic and Rocketry Residential Challenge which entailed various activities which carried accreditation in STEM subjects

• A STEM taster day was organised for participants whereby they could try their hand at a number of different activities related to the subject.

• 24 participants attended “Launch My Future” event whereby BTFT promoted this project to schools and businesses with a view to creating a pilot project that could lead young people from model boats to real boats, from working with cardboard to working with wood and from Aberdare Park to the open sea and to raise their aspirations and awareness about what life has to offer. Volunteers from local businesses came from GE Aviation Wales and Norgine. Students had to use the volunteers to find out what can help make people successful. The start point was to form teams from students at two schools; Bryncelynnog (SEN unit) and Maes Gwyn Special School and the volunteers from GE and Norgine.

Objective 4: Increase the number of employers and training organisations that develop equality and diversity strategies, including monitoring systems and methods of feeding in improvements:

The organisations that deliver elements of the project have been supported in their continual monitoring and improvement of their equal opportunities policies and practices. The Local Authority seeks to raise awareness of equal opportunities for all and will challenge any traditional assumptions in participating interventions and organisations.

3.2.2 Environmental sustainability

The project also set out plans at an early stage for series of activities and process that adhere to the six environmental sustainability objectives for all ESF Priority 1 Convergence projects.

Objective 1:Reducing Emissions of greenhouse gases to help limit the extent of climate change and help to adapt to its effects

The project team have sought to promote a range of activities targeted at reducing greenhouse gas emissions, including minimising travel or where travel was necessary to promote sustainable travel (car sharing and the use of public transport where at all possible).

Objective 2: Promoting Sustainable Transport All Service deliverers were encouraged to develop a travel plan that reduces car travel for work and for travel on businesses.

Objective 3: Promoting the Efficient use of Resources The project also sought to promote the efficient use of resources, maximising the use of ICT wherever possible and encouraging the adoption of the Eco-Code amongst all partners and across all interventions and encouraging each partner organisation and/or intervention to adopt their own eco-code and display this at their premises (thereby promoting awareness of activities that should be adopted to reduce greenhouse gas emissions).

Eco-Code

An eco-code was developed and distributed by project management for key project staff and providers early on in the project (that they could either adopt or to guide them in the development of their own eco-code):

1. We will reduce, reuse and recycle waste wherever possible by:

• Only printing out paper copies if absolutely essential • Photocopying on both sides of the paper • Ensuring paper, cardboard, plastic and cans are re-cycled using the facilities at Ty Trevithick central office, schools and other delivery venues.

2. Minimise the use of transport wherever possible by:

• Car sharing for journeys to and from the workplace • Reducing the need for travel through the use of ICT • Arranging project meetings/courses at locations which will minimise travel times • Car sharing to attend meetings • Using public transport wherever possible

3. We will endeavour to buy products and materials that do not harm the environment when they are made or used:

• Purchasing fair trade items • Purchasing products made from recycled/biodegradable materials

4. We will aim to lower energy consumption at work by:

• Turning off equipment when not in use i.e. monitors/printers

5. We will promote environmental sustainability to others by:

• Displaying our eco code at the central team premises • Encouraging project partners and visitors to become more aware of ways in which they can help the environment.

Furthermore, the project activity included training provision that involved education visits to RCT’s landfill site on a number of occasions.

Objective 4:Improving the quality of the local built environment and opportunities to access green space Healthier lifestyles (through the promotion of opportunities to access green space) have been promoted through project activities, the case study below provides such example – through the recycling of bikes whose subsequent distribution to the community will enable them to access green space more easily.

Case Study: Recycle-cycle Project

The recycle-cycle project was founded by the BTFT team at Pontypridd High School. The project engaged Sixth Form students who spent their free time bringing old bikes back to life for the people of Pontypridd. Supported by RCT Street Care Department who bring scrapped bikes to school and who donated cash to purchase tools, the aim of the project was to fix and then distribute the newly refurbished bikes to the community. A brand new bike was donated to the project and raffled at the Christmas Fair in order to raise funds for more tools to be purchased to develop the scheme further.

The project was headed by Building the Future Together Key Worker, Jim Sullivan. Reporters from BBC Wales and BBC Radio Wales visited Pontypridd High School to film and interview the team and stories covering the “Re-cycle cycle Project” were featured on S4C’s news programme “Ffeil” and BBC Radio Wales’ Breakfast Show.

Objective 5: Minimising the risk of pollution and other environmental hazards thereby safeguarding the health of communities and the environment

Again the promotion of the eco-code amongst delivery partners and interventions is the clearest example of how the project has sought to deliver against this objective whilst amongst delivery partners;staff have also been encouraged to undertake environmental awareness training.

Objective 6: Promoting biodiversity and the sustainable management of the land, sea and inland waters Course content within provision offered through BTFT promotes diversity examples include: • Participants from Rhydywaun created an area within the school ground to encourage wildlife and to use as an outside classroom area. (progress report 31st March 2012); • Participants from Bryncelynnog created an allotment for all within the school to use; • Participants from Hawthorn worked on a memorial garden including stepping stone pathways, shrub and tree planting;

3.2.3 Welsh Language Provision

Throughout the project, all materials associated with communication were produced bilingually. Key examples of this include externally available stakeholders’ reports and website information about the project for public consumption.

BTFT works across the 4 Welsh medium secondary schools in Rhondda Cynon Taf. In all Welsh medium secondary schools every subject is taught through the medium of Welsh, other than English as a subject in Key Stages 3 and 4, and BTFT bilingual Key Workers and Youth Coaches worked within these schools to ensure bilingual learning opportunities were accessible outside mainstream school. The participant database identifies 570 (5% of) participants who stated that Welsh is their preferred language, and 414 (4% of) participants received provision in Welsh across the project.

3.3 Spend

4. Project Management, Processes and Delivery

This chapter focuses on the management, processes and delivery of BTFT. As noted previously, there is an explicit focus on issues relating to the latest iteration of the project (i.e. the last academic year), however, this does not discount longer running themes and issues that were emergent and can represent key learning points. Indeed, as the general theme was that the extension period has been that with the least challenges, many of the themes regard looking back to learn from the project as a whole. Through the development of the evaluation framework and scoping phase, and during research, a number of themes and issues regarding project management and delivery emerged and these drive the topics covered here. In summary they are:

• Managing BTFT: The Central Team • Project Administration and Evidencing • Working on a funded project • Embedding school provisions • Referral processes • Coleg Morgannwg Provisions

4.1 Managing BTFT: The Central Team

The Central Management Team for BTFT are responsible for the overall management and co- ordination of the project. The overriding emergent theme during consultations regarding the role and effectiveness of the team was that the management of the project has ‘settled’ in the last year or so (especially since the extension period) and is currently at its most efficient and effective. The dominant perception was that the centralteam now performs their role extremely well considering the challenges of managing and delivering the project (explored further later in this Chapter). Those consulted pointed to a range of facets of the Central Team role and performance, including communication, guidance, and support:

“They have been great really, very communicative and approachable”.

“It was turbulent pre 2011 but the current team do a good job of a difficult role, they provide a bridge between WEFO and the delivery level”.

“[The central team] are very effective after a number of personnel changes which did little to maintain stability in the early years of the project. The lead body has provided strong, focussed direction over the last two years”.

There are a series of factors driving the more effective management of the project in recent years.

Stability within the team

The last 18 months represents the most consistent period with regards staffing within the management team. Previously there had been a number of changes of key personnel for example, the project lead and the project manager, and consultations revealed that this had previously impacted on the coherent management of the project:

“The most difficult issue was the changes in the PM, it was never settled or coherent previously”.

“When there was changes in PM the quality of support would vary from one to another, that is not the case now”.

“It is now very effective in terms of management and processes, everything has settled once the staffing did”.

Those who noted this as a challenge identified that the stability which now exists had aided the management and delivery of the project.

Internal Placement of Central Team

For the majority of the delivery period the Central Team has been within the Education Division of the Council due to the nature of the project. However, for the academic year 2012/2013, the Central Management Team was moved to the Corporate Department. This move seems to be one of the contributory factors in the improvement of the central management team as the team are now more focused on their role and independent of those deliveringin the same sector (education).

Streamlining of the project

In its previous form, BTFT encompassed more than the current five key interventions. In addition to those that make up the current package of interventions, the project included another 26 separate interventions. The overriding perception amongst those consulted was that the extension iteration of the project is more targeted and streamlined. This has made it more manageable in terms of the role of the Central team in handling day-to-day management as well as developing project wide processes:

“I always thought the initial project was too big – it is very difficult for them to manage such a huge range of interventions that are all different”.

Focus on compliance

Recent years have seen the management team focus more closely on project compliance, increasing their agency and prominence in ensuring evidence is collated and other similar tasks. This represents a centralising of processes relating to compliance and marks a shift from more of a focus on support for intervention managers earlier on in the project. Those consulted felt that this was the right route to take – there is less of a need for support at this stage of the project and the need had shifted to ensuring all targets were met and the general compliance of the project. This re-focusing can be seen as a key reason for the improved management of the project.

General Project Settling

Finally, there is a more general influence at play in regards to the more general ‘settling’ of the project. What is meant by this is that the project has now been underway for a number of years and has become more embedded in terms of processes and delivery. Staff are more experienced by this stage in working within the European Project Environment (see Section 4.3 for further discussion), and are more embedded within their roles having performed them for a long time.

4.2 Project Administration and Evidencing

The extension period of BTFT was identified as the most effective and efficient in terms of administration and processes such as those relating to evidencing outcomes. Prior to this period, changes in guidance from WEFO and a lack of clarity , for example regarding eligibility of activities, had meant this aspect of managing and delivering the project had been particularly challenging:

“Processes are now in place that are much clearer about what is required. This was not the case with paperwork in the past, there was no clarity regarding eligible spend for example”

“When there were changes to processes and guidance from WEFO this was a very difficult thing to translate to delivery staff”.

“The changes in requirement and paperwork over the first few years of the project was the key challenge, now it has settled it has much improved”.

Despite improvements in processes, managing the balance between time with participants and time spent completing other requirements was perceived to be a challenge by both existing/ core (in post prior to BTFT) and new staff (brought in as part of the project). In most cases this related to the balance between time with participants and working on administrative requirements:

“The amount of paperwork is challenging, I want to spend as much time as I can with the participants”.

“I understand you have to do it, but the paperwork takes a lot of time and limits our ability to do other things”.

“I've spent a week gathering paperwork on pupils that BTFT want and I didn't seen any pupils that week”.

In a few cases, there were other mitigating factors:

“I'm 50/50 employed with the school and BTFT so I don't get a lot of time for the BTFT paperwork. With the school I'm doing the BTEC side and there's lots of paperwork there too. It’s a challenge to balancing the paperwork for both”.

Helping and ensuring delivery staff complete requirements and do so correctly was identified as a key challenge for line managers and particularly the central team.As noted, some delivery staff found it difficult to meet deadlines and keep the required detail of records, and although case loads are high and staff need and want to spend as much time as possible with participants, in some cases this also seems in part due to a slight reluctance and resistance to fulfil the requirements:

“Delivery staff were doing the work but found it hard to evidence and keep up with that side of things”.

“There were those who would not keep up with deadlines and others who were not completing materials correctly at times, they want to and are used to focusing on the young people”.

Chasing and trying to ensure forms are returned is a resource heavy exercise for the Central Team and line managers, who also face the challenge of collating and summarising this data (for example for quarterly WEFO returns). One of the further impacts of this is that it has resulted in some of the work undertaken being ‘lost’:

“The full volume of work they do is not reflected as they have not been able to produce enough evidence in some cases”.

This is then clearly a key issue emerging from the BTFT project that has impacted on the project itself. This could be mitigated in future projects partly by providing training at the outset of the project for all delivery staff. This would cover all administrative duties including the evidencing of outcomes and help to give clarity and confidence to delivery staff. Those consulted were virtually unanimous in suggesting that this would be the best approach in future projects and should be a point of learning taken forward from BTFT.

In addition to this, the project would have benefitted from a period pre delivery where processes and guidance was fully developed and in place prior to delivery. If the management team were in place prior to delivery for a suitable period of time, this would enable processes such as those relating to evidencing outcomes to be fully developed. This could then be translated to delivery staff in training sessions mentioned previously. Of course, if processes are subject to change from a higher authority (Europe or WEFO), or guidance is not clear for the management team at the outset of the project, this is unachievable, and so this also represents a key piece of the jigsaw for WEFO and the Local Authority in ensuring a more coherent and workable administration system.

4.3 Working on a funded project

Linked to the above discussion of evidencing is the more general challenge identified by delivery staff who were working on a funded project for the first time having previously been core staff within the authority, and in some cases, undertaking a similar post/ role for a number of years.

There is a distinctly different culture and series of requirements associated with working on a European funded project compared to say, working for a local authority 10 years ago. Differences that were raised as challenges include less flexibility and more administration. This is not to say these processes etc. are not necessary of benefit, just that they represent a new way of working that some long term staff take time to adjust to (and in some cases are reluctant to commit to).

One of the issues arising was that previously core staff were of the perception that they were different to ‘project’ staff and should be treated differently, for example, when it came to assess which positions/ employees would be kept for the downsizing during the extension period.

“Those previously core staff see themselves as different to ‘project staff’ – in reality they are not and are all treated the same by the project regardless”.

In addition to this issue, the nature of working on a funded project, with a specific end date, was raised as a challenge for both delivery staff and line managers. With relatively short term contracts, there is often uncertainty which can hit morale, particularly near the end of funded contracts where it is unclear whether jobs are going to be safeguarded moving forward. This was noted by delivery staff who felt it was a challenge to keep motivated and confident at times, and also for line managers who helped delivery staff with these concerns:

“Before the extension was approved, staff were days away from potentially not having a job... we’re in a similar situation now and it knocks confidence and enthusiasm sometimes, we work hard to address this, but often can’t give a confident answer”.

“A fixed term project can lead to fix term attitudes and a lack of commitment – I don’t think this has really been the case but it is there if you are not careful”.

The key is that there is a clear communication of information regarding the future of the project. Generally, this seems to have been the case within BTFT, with the issue being that there is no information to pass on at times as decisions making and project development can take time and be beyond the control of the project. In these situations, keeping staff as up to date as possible and reassuring them is the only feasible approach.

4.4 Embedding BTFT School Provisions

“Schoolstaff don't always accept our role at the beginning, you have to prove yourself to everyone”.

“Getting teaching staff on board, they first see their classes being disrupted but later on they're asking for our help as they have seen the benefit of the work we do”.

“This project was the first of its kind in RCT and indeed SE Wales... [one of] the main challenges has been embedding the Youth Coach function into schools and defining the role so that it met school and BTFT needs”.

A number of respondents also noted that some schools early on in the provisions were keen to involve youth coaches and key workers (when these were school based) in non-project activities and divert some of their time to school requirements not covered by BTFT. In these cases, it took careful management and negotiation on the part of the BTFT line manager to ensure schools used the resource in the correct manner. In these cases, the learning coach or key worker had also been assigned a line manager within the school (in addition to their BTFT manager) and working with these people helped to overcome the issue.

In terms of the Positive Futures Programme, not embedded within schools but working closely with them, a similar pattern was identified. There was a feeling that the relationship between the intervention and schools had matured and developed as time went on. This was identified to have led to better identification of suitable participants and made management of the project more straightforward.

The settling of provisions and staff within school was argued to have improved once school staff had seen some of the benefits of BTFT provisions to pupils engaged, and others in the school (for example, peers of participants):

“Some schools were not enthusiastic at the start of the project but once they realised the impact of the provisions they were on board and committed”.

As relationships have developed over time, it was argued that this has improved and made delivery easier. One example of this relates to the negotiation between BTFT staff and teachers regarding the best time to take a young person out of class. Participants can not miss core subjects and so there are limitedopportunities to engage with BTFT provisions. This relies on the BTFT intervention worker working out logistics with teachers and is heavily reliant on trust and respect between the two.

4.5 Referral processes

How participants are identified for support in a project such as BTFT is a key aspect of the process, ensuring that provisions are reaching the ‘right’ kind of young people – those who will benefit the most from the support and who fit the profile identified by the project. With BTFT, broadly, participants are targeted if it is believed that they are at risk of becoming disengaged with education. Various risk factors were utilised to identify relevant young people, the most common being low attendance, risk of under-achievement, lack of basic skills or if the young person was of concern to education welfare. Although exact referral processes vary between the interventions (see Chapter 2), this identification process runs as a common thread.

Overall, the perception amongst project staff at all levels and other stakeholders was that the referral systems were effective and that the right young people were being engaged:

“Definitely; the combination of tests does a good job ensuring that the young people who participate in the project are targeted”.

“[The referral system] works well, referrals from partner agencies are very good now, they have a clear understanding of the work we do as the provision has been there for a number of years”.

“It is common for the ‘formal’ tests to reiterate the perception of referrers such as school staff”.

Ultimately the evidence for whether this was the case relates to the impact and outcomes of the project themselves

4.6 College Provisions

Many of the challenges relating to the management and delivery of BTFT referred to at this point relate primarily to provisions based at school age participants. Although administration issues stretch across the project, other factors such as engagement with schools and embedding staff into the funded project environment have not been encountered within the college based provisions (for obvious reasons).

5. The Impact of BTFT

Chapter 2 explored the progress of BTFT against key outcomes of the project. This started to highlight some of the impact of the project, for example, the number of participants gaining a qualification as part of their involvement in the project. This chapter moves on to explore the impact of the project further by focusing on the impact on participants beyond outcomes, including soft outcomes, and the impact beyond participants.

At the current time, RCT’s Education and Lifelong Learning division are undertaking an extremely in depth and extensive analysis of the wider impacts of the project (for example on attainment and attendance) within the authority, including disaggregation by intervention. This work uses access to a range of educational data within the authority to explore results on attendance, attainment, etc. between BTFT and non BTFT pupils and come to an assessment of the impact of the project using these detailed statistical indicators. The work also includes development of a scale of pupil need (based on FSM Eligibility, SEN status, In Care, LAC, and School Exclusions.)

There is no value in the work being either repeated or regurgitated during this evaluation, and as such, the report is considered essential reading alongside this evaluation once it is released shortly after project end (an early draft report was available at time of writing). Where it focuses on the use of detailed statistical analysis to explore impact, this report will focus on the extensive consultations with participants and delivery staff- another key indicator of impact.

As noted in the introduction to this report, 199consultations were conducted with participants covering all active interventions. Although focus obviously falls on those interventions active in the extension year of BTFT, the data refers to impact accruing over the lifetime of the project (some participants have been supported for a number of years).

5.1 Impact on Participants

A key aspect of measuring the impact and progress of the BTFT project is the overall opinion and judgement of the young people who participated. Though gaining access and information from this often hard to reach and sometimes alienated group had its challenges, the results that follow reflect the advantages of including this additional qualitative information. A total of 199 in depth, face to face interviews were conducted through a flexible approach, based on the knowledge of the particular participants available for consultation that day and the advice of the intervention worker available on the day. Whilst a significant number of participants were consulted it was felt that typically, consultations on a one-to-one basis (with or without supervisors, as appropriate), elicited a more complete response. Some group interviews were also conducted.

5.1.1 Detached Youth Workers

Approach

Detached Youth workers take targeted referrals from partners agencies (e.g. the youth service) but also engage young people in situ, on street corners and local ‘hang out’ spots where young people who may not be attending school can be found. Often it is not on the first contact that young people are engaged, the detached youth worker may visit a group on the street a number of times in order to gain their trust and encourage them to engage with the support. This mechanism can be considered a key strength of detached youth work, catching young people who are not commonly in school and not often engaged in other state provisions.

“The street based aspect is the key – there is a definite need there to support those young people who are not commonly in school”.

“We do street based work with young people, going into communities and finding detached kids who don't attend youth clubs etc.”

Another strength of the street based aspect of the work is that workers are engaging young people in ‘their’ space rather than somewhere like school where they often do not feel at ease and are guarded and suspicious of those working in and around the school system. Another aspect of this approach based in young people’s ‘worlds’ is the use of mobile phones to keep in contact with participants. By allowing participants to contact them in this way, detached youth workers are always available for support, often out of hours:

“The idea of using the mobile a lot is that is gives the young people the feeling you are there and they are using a method to contact you that they feel comfortable with”.

If referred to the provision, the first step is to assess the needs and aims identified by the referrer. This is then supplemented by discussions with the participants, and the development of an action plan. Often these are adjusted as the participant develops and re-assesses their goals.

Support for the participant can include a range of activities, including signposting, engagement on activities and help with employment/ education, but often entails much more. Emotional support, advice, and general life support is common. The level of disengagement often found (see below) mean that detached youth workers have to focus on developing their relationship with participants over a long period, gaining the trust and confidence of the young people.

One of the other key distinctions compared to other BTFT provisions is the regularity with which the detached youth workers focusedon groups rather than just individuals. The advantage of such an approach is that peers of participants are also engaged and are more likely to keep the young person involved in the provision than if no others in the friendship group were engaged and were pulling the participants away.

Nature of Participants

The majority of young people supported by detached youth workers are at the more disengaged end of the scale compared to those engaged by school based provisions. Generally, these young people face multiple challenges that have led to a disengagement not only with education but also other support mechanisms and sources of help. The majority consulted have a lack of positive or supportive adult relationships:

“There’s only my mum and me and we aren’t getting on, there’s no one else really and mum can’t cope with me”.

This means they can typically be hostile toward adults prior to engaging and illuminates why the street based aspect of the work is so important. Such a mechanism allows the detached youth workers to distance themselves from schools and other provisions such as social workers, whom the young people are no longer engaged with and can even be hostile towards. This was noted by both participants and detached youth workers:

“When you first approach them [the participants] most won’t even talk to you, they are not only suspicious of adults but also actively hostile”.

“I didn’t want to talk to adults, I thought they are just telling you what to do and trying to catch you out like report you to the police, but after a while of talking to us on the road we started trusting her [the DY worker]”.

Even before their teenage years many of the participants are engaged in potentially damaging activities such as drug use (including alcohol), risky sexual behaviour, and violence. This often revolved around peer group activities, out and about in the street and away from supervision (for example that might occur at a youth club). Breaking this cycle and behaviour is difficult as it is embedded and the norm for many of the participants:

“We did just hang out, do the MKHAT [drug] and that’s just what we did, we didn’t do much else that was our nights you know”.

Peer influence was a significant factor in these young people’s lives. They commonly referred to others within their peer group and undertook most activities with them. There was a recognition that this environment could lead to more destructive behaviours, but this was not of concern to the respondents:

“We all do stuff together like one does it and then everyone will, even if you know you shouldn’t everyone else is doing it”.

The importance of being part of these groups and fitting in with whatever is happening was a recurrent theme. It emphasises why the group based activities undertaken by Detached Youth workers are essential, reaching groups of young people and ensuring those supported do not slip back into destructive patterns.

Perception of Impact on Participants

All of the participants consulted who had received support from a detached youth worker were positive about their experience and about the impact the support has had and how it had changed their lives. These responses were categorised into a series of impact categories:

Graph 5.1: Detached Youth Worker Participants: Support Impact (%)

Nearly all (over 90%) of participants mentioned factors relating to confidence, aspiration or motivation. These respondents discussed how the support had made them more confident to address not only general life problems, but also those relating to education and employment:

“I’m more confident you know to try things and now I want to do stuff rather than be scared of trying”.

“I don’t feel like sitting around all the time I think I can do things if I want to, I am good enough to do them”.

In terms of basic skills, when first engaged by the project, some of the participants could not take care of themselves. Many had issues with personal hygiene, some had drug and alcohol addiction, the majority could not cook and lacked general skills that many take for granted:

“I’m better with coping like getting myself food rather than not bothering or just not bothering to go into school”.

It can be argued that any progression within their lives could not happen without first being equipped with basic skills. This is also true of social skills – the kind of interactions that are necessary in the workplace of educational establishments where participants will have to deal with both peers and adults. Over 80% of participants referred to social skills, particularly relating to adult interactions:

“I wouldn’t have done this talking to people like you [the researcher] I would have just been quiet or not come in”.

“I trust that all the adults aren’t not on my side anymore and I can talk to them easier”.

Over half of participants consulted had also been engaged in some education leading to a qualification, or had gained a sense of attainment from work with their detached youth worker:

“I want to be a teacher but haven’t got the grades so she helped get me on this course which will help me I didn’t think I would be able to cos I never did well in school”.

What was important/ successful about support?

An important facet of the impact of support is the perception of participants regarding why it helped them: what is it about the support that is important/ creates impact? Three key themes emerged from detached youth participants.

Firstly, nearly half of participants referred to the importance of having someone who is always there and how this makes a big difference to them. Detached Youth workers give participants a point of contact where they are able to call anytime, just for a chat or to get advice or guidance. This accessibility and consistency should not be underestimated as it creates a point of support for youngpeople which is always available – the person to call if something in a time of need, which many participants do not, by their own admission, have outside of the detached youth worker.

“I can get hold of her like text her and she will give me advice when there is no one else to talk to who isn’t my friends”.

“She’s the only one I know I can rely on”.

Secondly, participants repeatedly referred to ‘respect’ and having someone listen to them. Generally, participants felt that teachers and other adults did not see things from their perspective and did not treat them with ‘respect’:

“He respects me he sees what I think and doesn’t just think I am naughty”.

“I get to be listened to which never happens no one wanted to listen to me”.

“[The DY worker] will listen to me, she lets me vent everything out and calms me down”.

Finally, one group in particular felt that the space they met and interacted with the detached youth worker represented a sanctuary where they could get away from pressures both in school and out:

“We can come here rather than have all the other kids giving us stick and picking on us”.

“It’s safe in here... from all the bad stuff”.

Counterfactual: What would have happened without support

Participants were also asked whether their life would be different in any way if they hadn’t received support from their detached youth worker. At least 80% of all the respondents from felt that their life would be different with many stating that they would no longer be in education; a number acknowledged that they would just be hanging around on the streets or at home.

5.1.2 Youth (Learning) Coaches

Approach

Once referred a young person meets with the Youth coach on a one to one basis to discuss and define their specific needs. This allows the development of a Learning and support plan, identifying aims and how they are going to achieve them.

Participants are either worked with on a 1:1 basis or in a group as appropriate. In many cases there is progression from 1:1 to group interventions- when it is felt that the participant has reached the point they are ready to join activities (particularly important for social skills), they are introduced following a more intensive one to one period. As well as general support and educational support, participants may be signposted on to other interventions if required.

An important facet of the overall approach highlighted by a number of youth coaches consulted is that it is flexible and based on an individual’s needs:

“I try to do it on an individual basis... one way doesn't suit all children”.

Youth coaches are based in school permanently and this brings with it a series of advantages.Firstly, this is the place (in theory) that young people spend a lot of their time. By being school based, interventions are able to be based at a site which plays an important part in a young person’s life, a place where they often are.

Secondly, being permanently based in schools gives youth coaches the ability to provide a constant in young people’s lives- someone who is accessible and a point of stability. Participants know how and where to find a youth coach, something that is not necessarily true of other adults in their lives (see below).

Lastly, locating youth coaches within schools encourages and aids relationships with teachers and senior staff within schools. It provides a good environment for close working and allows school staff to gain a full and informed understanding of the provisions on offer.

Nature of Participants

“Participants face many and multiple challenges – attainment, attendance, SEN & EAL needs, disengagement, issues in school e.g. peer issues, issues at home, pastoral issues and so on”.

As illustrated above, participants engaged by youth coaches face a multitude of barriers to engaging with education (and ultimately, employment). These issues are often tied to home circumstances, where participants face issues such as transience (moving from one home to another), domestic violence, drug and alcohol abuse and physical abuse. Within school, participants struggled as a result of a lack of basic skills:

“Literacy and numeracy is a huge issue in the comprehensives of RCT”.

“Poor literacy skills and low reading age. They don't do any reading at home”.

Where participants lack basic skills and lack achievement, this impacts on their confidence and self esteem. Low on confidence and feeling disengaged, participants have behavioural and anger issues in the classroom, further impeding their learning:

“Some kids are low ability so they misbehave in class as they can't do the work”.

These challenges often amounted to a disengagement from school and a lack of positivity about the future (in terms of gaining employment for example). Participants do not see school as a productive activity and were not engaged in the current approach to mainstream schooling, thinking it would not serve them in the future. In this context, attendance in school was seen as an outcome for this group:

“Poverty, drug abuse etc, the kids have a poor background so for some just being in school is an achievement”.

Even when participants are in school, many struggle with relationships with staff. When behavioural issues start, a cycle can follow leading to more bad behaviour and a disintegration of the relationship between teacher and pupil:

“Barriers with teachers in school, some pupils just don't like their teachers and their teachers don't like them back so they don't want to go in to lessons”.

Perceptions of Impact on Participants

All participants were asked about whether the support provided by their youth coach had impacted on their lives. No negative responses were received and participants were almost unanimous in identifying at least one impact, for the majority, more than one. Answers were coded by key categories, summarised in the graph below.

Graph 5.2: Youth coach participant impacts (%)

Over 80% of participants identified that the project had increased their confidence or motivation, or impacted their aspirations positively. A number of participants reflected that they had not thought about their future prospects until receiving the support.Although thinking about thefuture had been daunting at the beginning, the support had given them a sense of purpose and an awareness of their aspirations:

Around 80% of participants consulted also referred to qualifications or other attainments that they felt had been achieved due to support from their youth coach. In most cases this related to improving their school attainment, for example, getting better homework marks and coursework marks. In some cases, this related to qualifications after school:

“I'm coming back now to the 6th form and studying a teaching assistant course. I wouldn't have done that if I hadn't had the help”.

“I revise now and my grades have got better. I sat my last GCSE on Friday”.

Whilst some were achieving qualifications and improving their overall school work, others were attaining in a different way. More basic skills were improved amongst those participants who needed to take smaller steps. This still represents attainment and underpins further educational achievement:

“Yes, my reading and writing has improved. It’s helped me in other lessons so when I'm in my Welsh class I can read better”.

“In Year 7 my reading wasn't very good and she's helped me to read better. I sometimes like reading because I can read better”.

“Out of school, I am better now in shops because I can count money when shopping”.

In terms of social skills, mentioned by more than half of respondents during their interview, a number of participants made the link between improving their confidence and making more friends. The knock on of this is that the young people felt happier and were more adept at dealing with the social pressures of school making them more likely to attend and engage:

“I’m more confident... I go out with my friends more and socialise more, it makes school better”.

“Yes, I am more confident now. I would walk past a group of people and would find another way round them. I would cross the road to avoid them but now I don't mind passing groups of people in the playground or in the street so my life has changed a lot since I've been helped”.

“It could make it easier for me in job interviews now as I am more confident to talk to people I am not familiar with”.

Another strong emergent impact related to personal development, particularly anger issues. Anger, often born out of frustration was a key barrier to participants that meant that they were unable to deal with situations in school:

“I got so angry before, I had to get out of my lessons, but now I don't miss lessons and my grades have gone up much more than what they used to be in all my subjects”.

“I've not been so angry lately. I've been able to control how I feel a lot easier”.

What was important/ successful about support?

Despite being school based, youth coaches were perceived by participants to be “different to teachers” and as such were able to overcome the barriers felt by participants in relation to teachers and other school staff. Much of this seems to be related to the time devoted to developing a relationship with the participant. Youth coaches are able to devote considerably more time than a teacher to building a relationship with the participant and overcome any initial barriers:

“I build up a relationship so that they trust me, they can't rely on anybody else - everyone has let them down and nobody believes in them”.

Removing the participant from class for a time was a valued aspect of the support and allowed participants time to concentrate in a new environment,

“Mostly, the teachers didn't like me so I stopped going to class. Doing the project meant I could have lessons without having to see those teachers”.

Finally, it seems significant that support is participant led. Young people play a key part in identifying their needs and driving their own path rather than a top down imposing – ‘we think you need to do this or this’, which can alienate participants early in the process.

Estimating the counterfactual

Just under half of those consulted said that they would not have achieved as much or would be in a worse position without youth coach support. In around a third of cases the participant said that they would either be excluded, or not be going on to further education:

“I'd probably be excluded from school. I was misbehaving in class because I didn't understand the work”.

“Yes, because I wouldn't be going to college. I'd probably have to re-sit and waste a year”.

“Without the support I wouldn't have had the chance of going for the apprenticeship”.

Another significant group of respondents referred to changes in their confidence and the implications of this:

“I'd probably be still coming up here and not talking to many people. I'm a lot happier now because I've gained confidence in myself”.

Finally, around a quarter of respondents referred to being better at basic skills, with implications for their learning:

“My reading, spelling and writing wouldn't be as good. I'm getting better at doing these now”.

Case Study: Youth Coach

DB was referred for Youth Coach support by the Head Teacher as he was at risk of underachieving.He was persistently not completing homework, disruptive and often doing inadequate work. The aims of the Referral were to promote Organisation Skills and Behaviour Management. DB has completed an Individual Learning Plan with the Youth coach through the B.T.F.T booklet which can lead to a Key Skills L2 in Improving Own Learning and Performance. He is working toward two targets: To complete homework and bring it in on time, and to raise his self esteem.

“Together we have looked at different ways in which DB could organise himself and he enjoyed making a poster for his bedroom, to remind him to look in his School Planner each night and do any homework and put it in his bag ready to bring to School. It was evident from a Self Esteem Questionnaire that DB completed for me that he had very low self-esteem so I worked on the O.C.N Level One on Developing Personal Confidence and Self – Awareness with him. DB also disclosed to me that he was being bullied by a few boys in his year group, calling him smelly and other names. I brought this to the attention of DB’s Head of Year who dealt with the issue”.

DB has expressed that he is enjoying the Developing Personal Confidence and Self – Awareness O.C.N and that he is beginning to learn more about himself and how to deal with situations in a positive manner. His self-esteem is starting to grow and as a result he is feeling more confident. This has also resulted in an improvement in his behaviour. DB has learnt strategies to organise himself and bring his homework in to School on time. Through the Youth Coach involvement DB is getting into less trouble in School he has more self-esteem and is feeling more confident in himself Copyright Wavehill Ltd 2013 Page | 42 Final Evaluation of Building the Future Together wavehill

5.1.3 Key Workers

Approach

At referral, a key worker undertakes a baseline assessment of the participant (with the young person involved), as informally as possible, and starts to identify issues and needs. This leads on to working on a plan with the participant as to what they can do/ achieve. This in turn develops into a joint action plan with clear aims and tracking progression so the young person can “see just how far they have come”.

Support is holistic but generally encompasses 1-1 sessions working with participants on whatever is needed, signposting/ referrals to other support, confidence building and social development.The focus of key workers is less academic than youth coaches, who focus more specifically on direct support for educational attainment.

The reduction in the number of key workers from 22 to 3 (with one post currently unfilled) as part of the extension of BTFT means that resources are spread more thinly and workers have less time with participants and can reach fewer:

“I find we have less time now to do the initial groundwork - less chance to get to know the individual”.

As such, the importance of signposting and referring to other provisions has increased. These outside provisions allow key workers to access specialist support for participants they are working with.Key workers were school based within previous iterations of BTFT but currently work geographically. When based within schools, the key workers were subject to the benefits highlighted in relation of youth coaches (See Section 5.12). Many of the participants consulted had worked with key workers within their school before the project adjustment.

Nature of Participants

Both participants themselves and key workers identified that they faced many barriers and challenges. Many have behavioural issues in school and have turbulent relationships with teachers and other pupils. Behavioural issues were often a reflection of chaotic and unsettled home lives, with some young people living within single parent families where the parent could not cope with the young person and or their siblings. These families did not only include single mothers or fathers, but were sometimes run by grandparents or other members of their extended family.

The level and number of challenges faced mean that it takes a long time to start to make progress with participants:

“Our participants are very disengaged, it takes a long time and a lot of work to overcome their issues and start to think about accredited outcomes”.

Most seriously was that a number of those consulted identified that a culture had emerged within RCT (particularly within schools) around self harm and in some cases, suicide. Once a culture and trend started to emerge, this seems to have driven others to follow a similar path:

“There’s self harm, a lot of attempted suicides, all the dark stuff”.

Perceptions of Impact

As a result of the differing needs and backgrounds of the young people, the key impacts of the support tended to vary according to the individual. Having said this, the vast majority identified positive impacts relating to key categories:

Graph 5.3: Key worker participant impacts (%)

A lower proportion of participants supported by key workers identified impact relating to qualifications and attainment compared to those supported by the youth coaches, but a higher proportion identified impacts relating to confidence, aspiration and motivation (90%) and social skills (78%) than youth coach participants. This evidences the notion that youth coaches are more academically focused, with key workers more holistically focussed. It is not to say that these are low proportions either – nearly two thirds of those support by a key worker said that they had either got a qualification as a result of support or had done well at something and had a sense of achievement, for example:

“The course I did with them was the first thing I’ve got a certificate for and now I’ve got that it feels good”.

“Most of the achievements I have got now I need for college and my work placements, thanks to the project”.

In addition to these categories, over half of respondents said that they had started attending school more often due to the support they had received. This was confirmed by key workers. Participants felt that they were more confident attending school and felt more settled in lessons:

“I come to school more now, I know there’s things I can do well at”.

“She was the one who got through to me about turning up”.

Generally, participants spoke of being more engaged with school and education more widely, but many also specifically referred to changes in their personality such as becoming more outgoing or being able to control anger better:

“I don’t get the rage now I can just talk to her and work it out”.

“I’m more outgoing and less of a grumpy personality”.

“My behaviour changed. I used to be really aggressive and angry but I've calmed down a lot now by chatting to [the KW]”.

What was important/ successful about support?

Many participants found interactions with teachers and other educational staff difficult prior to working with a key worker. The way key workers are able to position themselves slightly outside of this group and as such engage with participants seemed to be a key aspect of the support. By positioning themselves in such a way, key workers were able to bridge those initial barriers:

“She’s not like teachers, it’s something better than that”.

“No not a teacher, more like someone who cares but also does that teacher stuff with you like working”.

This concept also links to young people’s willingness to follow advice and guidance from key workers. If the ‘non teacher’ relationship, built over a long period of time, did not exist, participants would not have respected the opinion of the workers and taken their advice as regularly:

“I would do it if a teacher said but I trust [the KW’s] opinion like what she thinks I should do”.

Another important aspect of the impact of support relates to the holistic nature of the key worker function. This allows the key workers to respond flexibly to each participant and means that there is support for a variety of issues:

“I know they will help with whatever, that’s a biggy”.

Estimating the counterfactual

As with other participants, key worker supported respondents were asked how their life would be different without the support they had received. Just under half answered that they would not be attending school as often or at all and a similar proportion said that they would not have achieved as much (academically and non academically) without the support.

There was also a strong theme amongst answers relating to maturity and the path young people were on before support from a key worker. A number of respondents said that they were “more grown up” and were no longer on a path that would be destructive:

“I'd probably still be a little wild child”.

“I’ve grown up, if I hadn’t I’d be in jail the way I was going”.

Participants also referred to being able to achieve things that would not have been possible without support. These responses regularly focused on employment, showing the distance travelled by participants who would not have considered that part of their future prior to support:

“I wouldn't be able to get into college and I wouldn't have had an apprenticeship. I wouldn't be able to work on a construction site”.

5.1.4Positive Futures

Approach

The majority of referrals to Positive Futures come directly from schools or school based support systems (such as BTFT Youth Coaches and Key workers). Once referred, the individual is assessed by the Positive Futures Programme Manager using the evidence provided and through a discussion with the referrer. A meeting is then undertaken with the participant to explain what is on offer and assess their needs.

At this point, the participant is engaged on one of the alternative curriculum courses. These are very practical focused, often outside, and are designed to give participants an enjoyable educational experience, building their confidence and sense of achievement through non classroom based activities. As well as improving vocational skills and softer skills, these courses are accredited, giving participants an additional benefit.

When attending activities as part of Positive Futures, participants tend to be ‘hand held’ throughout the process- this means being picked up for the activity and taken home for example. This helps to overcome other barriers faced by the young people, in this example, a lack of ability to catch public transport or lack of guardian who would otherwise assist them getting to and from activities. This helps the intervention to be as inclusive as possible and reach the most difficult to engage young people.

For many of the participants, other school based interventions have not been successful and this drives referrals to Positive Futures. The programme is able to take these young people away from the school environment and give them more intensive support than is available in class.

Nature of Participants

Positive Future participants are a diverse group, although commonly these are young people for whom other interventions in school are not helping, who are disengaged with the school system and ‘traditional’ learning (classroom, teacher based), and who face multiple problems. A number of these participants are in care or at risk of going into care, with others also facing turbulent family situations.

“You see alcohol dependence, abuse, other home factors that really underpin everything else”.

Lack of engagement with mainstream education is reflected in, and driven by, a lack of achievement amongst the participants. Some lack basic skills, are illiterate, or not confident enough to express their skills.

“They aren’t confident as they don’t believe they can achieve. Often this is routed in a lack of basic skills”.

In addition to this, a number are socially isolated and lack social awareness. This leads to them being fearful of contact with others and an inability to trust people:

“Overcoming fears and getting them to learn to trust people”.

Perceptions of Impact on participants

All of the participants consulted who had been engaged by Positive Futures were positive about their experience and about the impact the support has had and how it had changed their lives. As with other interventions, these responses were categorised into are series of impact categories:

Graph 5.4: Positive Futures participant impacts (%)

Significantly, all of those participants consulted referred in some way to attainment or gaining a qualification as an impact of support from Positive Futures. Being part of a course or activity that they were able to complete and do well was important to the participants, many of whom did not experience this sense of achievement in school. This gave them a sense of achievement and overcoming challenges that they were not used to:

“I think I’ve done well, they tell me that and I feel good because it was hard”.

“I was scared of heights but I managed to do the high ropes and zip lining and I wasn't scared”.

This attainment went alongside changes in confidence and self esteem, identified by 80% of those consulted. Confidence itself can also be closely linked to social skills, with participants noting that they were interacting with people they normally wouldn’t:

“I've made more friends in school and outside of school as I'm more independent and confident. Its good fun working outside and with different groups”.

All of these factors (sense of attainment, confidence, social skills) are important in helping young people engage with education.

What was important/ successful about support?

One factor stood out as vital to the impact of Positive Futures: the fact that provisions are out of school, ‘hands on’/ practical, and challenging. This was seen as the most important facet by participants who expressed that removing them from the school environment for a time was important as it was an environment where they were not engaged and not achieving:

“They identify kids like us who want to be outside and not stuck in a classroom. I learn a lot more outside doing hands on stuff than in the classroom”.

“I feel different out here, like I can achieve things and its easier going back to class even though I don’t really want to”.

The challenging nature of activities was important as it seemed to drive participants’ sense of achievement- they viewed the activities as ‘hard’ and a such felt good once they had completed them.

Estimating the counterfactual

Participants were largely unsure how they would be different if they had not been part of Positive Futures. With prompting, most referred to their confidence and said that they are probably doing better in school as a result. In addition to this, others referred to having more friends and being more outgoing.

Case Study: Positive Futures

CM is a 14 year old female living in a socially and economically deprived area within the South Wales Valleys. C lives in a single parent household with her mother who is alcohol dependant. C’s daily routine is chaotic given the aforementioned factors, which have a negative impact on C and her development. C is currently place on the Child Protection Register.

C is registered at a mainstream comprehensive school, however her attendance was at 40% at point of referral. C perceived school as “scary”. C’s confidence was at an all time low and C was socially isolated as her lack of attendance at school hindered her forming and maintaining friendships. This was exasperated as C was reluctant to bring friends home given her mother’s health needs which led to an inactive lifestyle for C. C has been diagnosed as being Morbidly obese. There were concerns around C spending too much time on the internet and potentially placing herself at risk.

C has attended the core group programme for 2 days a week. Since initial engagement C has gained in confidence and has made sound age appropriate friendship groups. C has participated in a range of activities from Orienteering, Walking, Climbing. Bush craft and First Aid. C has gained a 4 Hours First Aid Life Saver Certificate but is working with the group on achieving further accreditations next term. Development has been slow but consistent given C’s individual needs.

Intensive support has been provided to C to return her to mainstream school for the remaining 3 days of the week. C is no longer as “ scared “ of school. C has pushed herself out side of her comfort zone through participating in Outdoor activities. C is participating in an active lifestyle and enjoying taking a healthier approach to life. C is addressing issues around healthy eating and working on making healthier food choices.

5.1.5Coleg Morgannwg Provisions

Approach

At the start of the academic year, those who are enrolled in the college are assessed and those below level 2 in basic skills are approached to be engaged in the BTFT support. In addition to this, there is a focus on recruiting and taking referrals of young people who are NEET, and have previously struggled to engage with formal education at an earlier age. Tailored support is provided for these young people in class and includes e-learning.

The college also offer Gateway Vocational Courses, an Alternative Curriculum to help enable more young people to re-engage with education. The programmes are practical, hands on in nature and offer the most disaffected young people a structured integrated programme of vocational training coupled with alternative activities, providing them with valuable work experience and qualifications.

“Getting people on to Gateway courses and completing them is a key step toward higher qualifications and gives them a sense of achievement, many of them are not used to this”.

More general support is provided to students undertaking courses, generally this will be one to one. It is best considered a mentor role, where delivery staff are able to “support participants emotionally and socially” as well as educationally:

“By doing our mentoring role one to one. We have an emotions board so the students, as some of them can't tell us how they're feeling, can put a post-it note on happy, worried, sad, anxious, cross, sleepy and they do this daily. A couple of the students are carers so may be just tired and not able to work properly”.

Support is based on building a relationship with the participant and guiding them through their qualification.

“I show them respect and hope to get it back. In less than a month most realise we are on their side and want them to achieve”.

Nature of Participants

Low levels of achievement are common amongst participants engaged by the college as part of BTFT. The nature of courses offered (low level, entry qualifications) are specifically targeted at this group. With many having had low levels of achievement it was clear this was often not because of a lack of ability of ‘intelligence’, but was routed in confidence issues and a lack of belief:

“trying to build their confidence - they say they can't do things when we know they're capable”

“When I first started I was really shy and lacking in confidence. I used to be bullied in school and my self esteem was very low”.

As with other BTFT interventions, participants generally face multiple challenges. These include the most basic of life skills and affect these young peoples’ ability to engage in education and with peers:

“Some don't have breakfast or are eating the wrong breakfast, simple things”.

Many have challenging lives with a lot to balance, untypical of other college students including young children and other similar responsibilities.

It is very hard for these participants to balance outside challenges with educational ones. In many cases, work needs to be done with participants before they are at a place where they can attempt to pass courses. This is often more to do with factors other than academic ones:

“As students they have to go out on work placement, therefore, we have to build their social and basic skills before they can do this. A work placement is an integral part of the Level 2 of the child care course for example”.

Perceptions of Impact

All of the participants consulted who had been engaged by the college were positive about their experience and about the impact the support has had and how it had changed their lives. As with other interventions, these responses were categorised into are series of impact categories:

Graph 5.5: College participant impacts (%)

Over 80% of participants focused on issues relating to confidence, aspirations and motivations. This is encouraging considering this was highlighted as one of the key barriers faced by participants.

“I was very shy and lacking in confidence. The breakfast club has meant that I am able to eat in front of people now and I can talk to other people and my tutor now”.

“It’s helped me and the course. I am in a child care placement and it’s helped me talk to the children and the staff there”.

Overcoming this lack of belief was key to achieving impact relating to attainment and qualifications. Over 90% of respondents said that the support had either helped them to enrol on a course, complete a course, or gain another form of attainment. In some cases it is not only confidence but also academic barriers to achievement (such as spelling) that were being addressed:

“I struggle with the work sometimes. I get help with a lot of the classes, they help with my reading and if we don't know how to answer the questions they'll give us an idea”.

What was important/ successful about support?

Participant interviews alongside staff consultations gave a clear idea of the key aspects of the support, which underpinned impact. Firstly, the atmosphere created by delivery staff and the overall ‘informal’ approach allows young people to be relaxed and comfortable. Spending time with BTFT provisions provides a sanctuary from pressures and a forum where issues can be discussed. Within this atmosphere, participants were able to emerge socially and became more confident as a result:

“Talking to people in a relaxed atmosphere has helped me with my confidence”.

“Being in a relaxed atmosphere and talking to people, I have built up a friendship and my confidence has got better and better”.

Participants trusted staff and valued the fact that they felt that they could always access support, when needed. This is an important aspect of the work as the chance to help may be missed if there is an inaccessibility to support:

“That they are always there for you to go to”.

“You know that you've always got help”.

Because of this confidence in BTFTV college staff, participants were willing to share sensitive issues. This is significant as it means the participant is willing to discuss issues which are impeding their educational attainment but which they would not usually tell college staff about, being personal and/ or seen as embarrassing:

“We are more approachable regarding sexual health, they can talk to us”.

Estimating the counterfactual

Respondents saw a close link between the support they had received and completing courses at the college. Nearly all of those consulted felt that they would not have passed without support and/ or they would no longer be engaged in education:

“If I hadn't have had the help I wouldn't have passed. I would have struggled with the course if I hadn't had the support”.

“I think I would have failed the course - dropped out by now”.

These are robust statements that the support has been the primary factor in success and engagement in education. In these cases, participants are not school age and so can more easily become disengaged if things are not going well.

Case Study: Coleg Morgannwg

DD left school with no GCSE’s and became disengaged from education. She was referred to Coleg Morgannwg by her Careers Adviser, and plucked up the courage to enrol on the “Let’s Go Programme” funded by the Building the Future Together Project aimed at helping young people re-engage with learning and giving them better prospects for employment.

As part of the timetable DD was offered additional learner support via Learndirect, which she found extremely helpful, especially in Literacy. Let’s Go included motivational /confidence building stages and employability skills.

DD had set her sights on a career in Childcare and enrolled on to the Cache Level 1 award in Caring for Children course. She said “Before I enrolled on the course I lacked confidence in my ability to learn, especially in Literacy. The Lets Go scheme has helped me become a more confident person and I’ve rediscovered books and reading! I’m really excited for the future. My family have also seen a positive change in me”.

DD’s tutor said “Initially she was nervous about the sessions, especially Literacy; but after initial extra support and using Learndirect as a learning tool, she gained in confidence. She’s been really focussed and is a great example of how you can pick yourself up, and with a bit of courage to have a go, can begin to achieve your ambitions, I’m glad that I’m having the opportunity to support her on her learning journey”.

5.1.6Discussion of findings

Many of the impacts relating to participants highlighted in this section are common across all BTFT interventions. The majority of these relate to ‘soft skills’ such as building confidence and developing a young person’s social skills. What is important is that the evidence shows impact beyond the headline indicators and reflects the level of support provided by the project and the difference it can make to young people at risk of disengaging. Whilst some impact is evidenced through project outputs, the feeling amongst BTFT staff was that this did not truly reflect the impact of the project. One of the key concerns was that soft outcomes were important but not being sufficiently captured. The evidence collected shows that the project has had significant impact in terms of soft outcomes with participants.

What could be perceived as ‘soft outcomes,’ for these young people, represent the intermediary stages on the way to achieving a hard outcome. They included for example, thinking skills such as improved problem-solving abilities, personal attributes such as improved self-confidence, or practical work-focused skills. They also often represent the main outcomes achieved by the most disadvantaged ESF client groups. The results of the participant interviews express a significant gain in soft outcomes, and it becomes evident that young people need to overcome their initial barriers, in order to gain hard outcomes, such as training opportunities, a place in college or employment.

“The reason the soft skills are so important for young people is that they can directly impact their future lives- confidence, the ability to interact with others and so on, these are all skills they will need to work in the future”.

“Small steps need to be taken before the big ones”.

Overall, different interventions within the project have similar but not exactly the same impact. An example of this would be the more academic/ attainment focussed support of youth coaches and Positive Futures, compared to the slightly more holistic impacts of detached youth workers and key workers. This demonstrates that interventions follow similar aims, but with slightly different approaches and results, all valid.

5.2 Wider Project Impacts

The previous section explored the impact of the project on participants and was driven largely by their perceptions of change. Impacts were also identified beyond the participant, usually as a result of the impacts on the participant identified previously. Four areas are explored:

• Impact on families and siblings • The wider impact within schools • Impact on communication and integration within the authority • Mainstreaming of provisions

As noted in the introduction to this chapter, at the current time, RCT’s Education and Lifelong Learning division are undertaking an extremely in depth and extensive analysis of the wider impacts of the project within the authority, including disaggregation by intervention. This work uses access to a range of educational data within the authority to explore results on attendance, attainment, etc. between BTFT and non BTFT pupils and come to an assessment of the impact of the project using these detailed statistical indicators. As such this will not be covered in this report but should be cross referenced at this point.

5.2.1Impact on families and siblings

The first impact additional to participants relates to the families and siblings of these young people. Across all interventions, delivery staff and participants highlighted impact on families and siblings, arising as a product of positive changes in the participants themselves. Mostly, this seemed to be related to the overall improvement in the behaviour and mood of the supported young person, but also was sometimes routed in the reduction of pressure on families in terms of supporting the young person:

“I behave better at home - not so cheeky to my mother”.

“It seems to take the pressure away from the families, there is someone else helping and finding a way to communicate with the young person”.

In rare cases, impacts on the siblings were identified that went beyond an improved home life to including inspiration to undertake education:

“The link has been there with the parents. Families tell me they are more sociable. Siblings - one girl has gone on to university and her sister is now starting Level 1”.

5.2.2Impact within schools

A range of interviewees identified wider impacts within the schools in which BTFT operates. What is meant by this is an impact of the support that extends beyond the participants themselves. The main benefits were argued to be a reduction in the pressure on teaching staff and a reduction of behavioural issues and problems within the classroom.

With regards to teachers, the BTFT provisions were argued to have taken the pressure off and allowed them to focus on other priorities such as teaching large groups. BTFT provisions not only increased capacity within schools to work with low attainersbut also increased the time available for teachers to support pupils who were not participants. The project has also led to changes in the perceptions of young people supported amongst staff:

“Teachers have told me that the project has helped them see some of the young people in a new light – see that they are changing and that they are worth fighting for”.

“Schools are obviously full of teachers but you need people with different skills to be able to deal with issues that young people are affected by”.

“BTFT are able to deal with issues that go above and beyond what a teacher could deal with without disrupting their teaching”.

With respect to the classroom, it was argued that classes where some pupils had received BTFT support had become more settled following the improvement in the behaviour of those supported.

“Anecdotally I have heard that there has been an improved atmosphere in some of the classes where we have worked with young people – helping just one or two seems to have a wider impact on the other pupils on class”.

These are just the two main impacts highlighted. As the quote below demonstrates, changes in attendance and behaviour at the school level were anecdotally suggested. It will be interesting to see if this translates into statistical validity as part of the study currently underway with the authority.

“We have seen an improvement in attendance and behaviour and changes in attitude. We recently did a review of well-being in the school and that had improved. We have young people actually turning up to school and some have improved confidence and they feel like they have someone on their side, exam results have also improved. The support has probably avoided things happening but of course we can't be sure of this”.

5.2.3Impact on communication and integration within the authority

As noted in Section 2.2 of the report, the mid-term evaluation highlighted that the BTFT project offers a more ‘joined up’ approach to supporting young people in the authority.

“The evidence strongly demonstrates that the project has had a positive impact upon the integration of services, most notably through improving referral processes and communication between services, settings and interventions”.

These processes and relationships have now had another 18 months to mature and embed. They were again highlighted by respondents as a key impact, in line with the mid-term report. In addition to this, it was argued that the relationship between the college and local authority had improved as a direct result of the project, and that this may lead to future impacts:

“There is now a much better relationship between the college and RCT, the project has been the reason for this and there are now other joint ventures resulting directly because of this for example, we are looking into a school leavers support scheme”.

5.2.4Mainstreaming of provisions

An important indicator of impact and legacy for the project is the extent to which BTFT support has been mainstreamed. Those provisions mainstreamed are summarised below, firstly those relating to schools and then those relating to Coleg Morgannwg. Together, these demonstrate significant mainstreaming of activities, a key impact of the project.

5.2.5School provisions

The BTFT project was originally due to finish at the end September 2012, and at this point, some of the structural and cultural services that were introduced through BTFT had been accepted into the formal and informal education delivery structure with the Local Authority financially committing to mainstream the following interventions:-

• Special School Programme – schools have significant benefits of a central co-ordinator, overseeing their extra-curricular activities. The four special schools will collectively fund the co- ordinators post on a rolling annual contract, with the aim of building upon the work conducted during the life of the Project to result in accredited outcomes being achieved by their pupils, something they could not have envisaged previously

• Teenage Parent Support – Early Years and Family Support Services have facilitated the support and childcare element of the Intervention post September 2012 to ensure that all young parents have the opportunity to re-engage with education whilst pregnant and when returning from having a baby.

• E3 and Holiday Programme – Whilst these Interventions were in existence prior to the BTFT project being approved, many changes have taken place as a result of the Project developing and these have been continued beyond the life of the funding. Accredited courses, World of Work opportunities, STEM activities are just some of the changes that were introduced, which on reflection has clearly increased the impact and improved the programme for both the Authority and the young people and so will continue as a legacy of the project. Established non stigmatising pathways for targeted support will continue.

• Generic Employability – this has proven to be very well received by participants and prospective employer alike, with other Projects (both internally and externally) choosing to implement the programmes from their own funding streams. Referrals into these programmes will continue to be made for any participants where it is deemed appropriate and beneficial to do so.

• Essential Skills – schools have embraced the format that this intervention was delivered in and have made the decision to employ the staff delivering the provision themselves. Therefore, these staff members are now school staff but continue to work in the way they did whilst on the Project, as a separate entity to the teaching staff. They will continue to use the CATS testing and BKSB package to identify and subsequently assess young people for their gaps in knowledge and use the process to rapidly improve their literacy and numeracy skills.

• Care2Work – the nature of Care2Work Intervention has always been regarded as innovative. The internal work placement programme for care leavers will continue with the additional strand of placing young people in placement with external employers continuing as an additional arm of the aftercare service.

• Young Carers – the Support Worker employed through the project has assisted several young carers to consider opportunities that they would not have previously contemplated, the benefits have been welcomed by the Authority and the role will continue raising aspirations and supporting young carers to achieve their goals

From September 2012, WEFO agreed to a further 12 month extension to the project for a transition period focussing on identified key areas, giving time to fully realise the impact of the project:

• Detached Youth Workers – The Detached Youth Team will continue with delivery in its current form up to the end of the calendar year, whereby this will be reviewed as part of the current re- organisation being undertaken by the Department

• Youth Coaches – All 19 secondary schools will continue to have a youth coach function in schools, part funded by 14-19 network with some schools funding the remaining hours from their core budgets

• Positive Futures – The successful elements of the project will be integrated with the services current provision

5.2.6 Coleg Morgannwg Provisions

There are two key provisions within the college, learning support and Gateway courses. In terms of learning support, during the life of the project, the college piloted various methods of delivering basic skills support to students who were assessed under level 2. The most successful methods are due to be mainstreamed when the project closes as the College feels that it has given students the extra support required and improved participants’ chance of retention, increased their chances of passing the course and increased basic skills levels.

Gateway courses have enabled lower level learners to enrol on a college course at their ability, giving them the opportunity to progress onto a level 1 course after completing the 1st year. These courses will still be available once the project is complete as the college has identified that they have fulfilled a need and that this need still exists. Whereas the BTFT project was only able to focus on a few areas of the curriculum, the college has now developed a foundation tier for all curriculum areas ensuring that any potential learners who do not have the qualifications to enter at level 1 have the opportunity of starting at a level which suits their abilities.

6. Conclusions

Progress and Outcomes

With the end of the academic year near, the project has achieved in excess of the target for four of the five key indicators, with more participants engaged than anticipated and more gaining qualifications, entering further learning or entering employment than targeted. The numbers gaining other positive outcomes is approximately 10% short of the final target, a shortfall that is largely explained by changing definitions within this indicator (eligibility) and is common across a number of other ESF projects.

Given the issues raised in the mid term evaluation and here regarding the slow start and the ambitious aims of the project, this reflects well on those involved in the management and delivery of the project.

Management,processes and delivery

It is widely recognised that that the management by the Central Team (and Coleg Morgannwg) of the project has ‘settled’ in the last year or so (especially since the extension period) and is currently at its most efficient and effective. The dominant perception was that the central team now performs their role extremely well considering the challenges of managing and delivering the project. Several factors have come together to make this happen: there is more stability within the team; the team is now part of the corporate dept. rather than Education; the project is now more streamlined; there is an explicit focus on compliance within the Central Team; and the project has more generally ‘settled’ into delivery.

One of the key issues running throughout the delivery of the project is the completion of administration, for example, evidencing. Although the situation has improved, changes in guidance and a lack of clarity with guidance (for example over eligibility of activities) earlier in the project have provided a challenge for those managing and delivering the project. Delivery staff find the balance between time spent with participants and completing project administration requirement difficult, but there is also evidence of a slight inertia on this issue within the staff that has proved difficult to manage for the Central Team and line managers who are responsible for collating materials in a timely manner.

For delivery staff, working within a funded project has been challenging in a range of ways. Along with administration, uncertainty over the future of roles and retention of morale is difficult.

Recommendations

• Provide training at the outset of similar future project for all delivery staff. This would cover all administrative duties including the evidencing of outcomes and help to give clarity and confidence to delivery staff • Similar future projects will benefit from a period pre delivery where processes and guidance is fully developed and in place prior to delivery. If the management team are in place prior to delivery for a suitable period of time, this would enable processes such as those relating to evidencing outcomes to be fully developed. This could then be translated to delivery staff in training sessions mentioned previously.

Impact

Based on the evidence and discussion in this report regarding the complex and ingrained challenges faced by participants of the project, the current set of possible outcome indicators is not considered fairly reflective of what represents ‘success’ for this group: staying in education for example is a big challenge which the project may support the participant to achieve.

Recommendation

A participant remaining in education and/ or increasing attendance above a certain level (say, 90%) should represent a key (claimable) outcome in future projects with similar provisions to BTFT