Society of Chief Librarians

Evaluation of the Public Library Universal Information Offer workforce development programme: Supporting Digital Access to Information and Services

Executive Summary / Advocacy Report

20th March 2015

Authors:

Sheila Moorcroft

Andrew Myers

Oakleigh Consulting Limited Suite 626 Sunlight House Quay Street Manchester M3 3JZ

Tel: 0161 835 4100 www.oakleigh.co.uk 1. Introduction and aims

1.1. The Digital Information Skills for Library Workforce Development Programme This report summarises the findings, conclusions and recommendations of an independent evaluation of the Digital Information Skills for Library Workforce Development Programme (LWDP) developed in the early part of 2014 and rolled out to library staff in September 2014. Oakleigh Consulting Ltd was commissioned by the Society of Chief Librarians (SCL) and the Arts Council England (ACE) to undertake this evaluation between January and March 2015. The evidence to support the findings is based on a 360o appraisal, with inputs from 22 external and internal stakeholders; 4,432 staff responding to a staff survey; 41of 54 trainers delivering the programme; opinions from 32 customers; and an assessment of the feedback on the e-learning modules staff had undertaken (over 12,000 as of March 13th 2015). The implementation of the LWDP continues till the end of March 2015. However, the findings in this report are relevant in measuring the success of the programme to date and include a series of recommendations highlighting issues or areas to consider for the future. A more in-depth report with eight appendices providing the detailed findings from the research is also available.

1.2. The aims of the evaluation The aim was to evaluate the process, outcomes and impact of the Digital Information Skills for Library Workforce Development Programme and to explore wider issues and opportunities. Among other things, we looked at: the cascade approach, the impacts on staff confidence, and the relevance of and the ability to apply learning. 2. Key achievements of the LWDP 1. A consistent national programme was important. All trainer respondents agreed that a national standardised programme was important; 88% that the e-learning modules supported consistency. 2. Staff engaged well with the LWDP. Over 12,000 staff (about 75-80% engagement of all staff across England) had registered for the programme as of 13th March 2015. 3. The regional cascade approach worked well in several ways. All the trainers enjoyed their role; 88% agreed they had been able to make their own decisions with regard to roll out; local support from colleagues was rated highly by the majority of staff and trainers. 4. Overall, confidence has grown. Since the first Skills Audit of 2013 overall confidence among staff has grown by +8.6% on average across the skills measures. 5. Confidence around using Government transactions and sources of information has grown most. The higher shifts of confidence were reported for 'Government transactions - doing things online' (+14.3%) and 'Government sources of information' (+13.1%). 6. The training modules were relevant to the majority. More than two thirds of staff (67%) and 82.3% of trainers agreed that the modules are relevant to their day to day job. 7. Staff are applying the learning. 91% indicate that they have been able to apply the learning. 55% said they were able to apply at least half the learning when helping customers. 8. Staff feel better placed to help. 73% of trainers felt staff are better placed to help customers; 865 staff respondents provided examples of being better able to help customers, as a result of the LWDP.

Oakleigh Consulting Ltd Page 2 of 8 9. Recommendation is an indicator of relevance and overall quality. An overwhelming pro- portion of staff completing the training (88%) said they would recommend the course to colleagues. 10. Customers value the services and the help they receive. All the customer respondents rated the help they received as Very good/Good, and their experience of using online services overall at over 8 out of 10 – where 10 equals excellent. 3. Conclusions

1.1.The LWDP process - an overall success The LWDP process has been a success on a number of fronts such as the number of people completing the modules, the regional approach, and the enthusiasm of staff and trainers. Aspects which respondents regarded as contributing to the success of the LWDP process include:

• Having a national programme provided a useful baseline and a level of consistency. • The regional trainer structure and support, including the ‘train the trainer’ sessions, which enabled flexibility. • The numbers of people completing the modules, and indeed their willingness to provide feedback on the modules and for this evaluation. • The quality and range of the content. • The quality of the face to face sessions, for those who attended them. • Levels of support when people had queries, especially local support from colleagues and other trainers as well as from Learning Pool.

1.2.The regional network – a valuable resource The regional trainers all enjoyed their role and were regarded as professional and knowledgeable by staff in the face to face sessions which they ran. This network is a valuable resource which should be developed. They could play a role in identifying training needs, developing more effective co-creation of training and skills development programmes, and be encouraged to look to the future. Their efforts and commitment should be recognised; their own capabilities developed and enhanced.

1.3.Staff confidence rising on several fronts The LWDP has had a significant impact in many areas, and has improved staff confidence on many fronts.

• Confidence levels relating to government services, transactions and sources of information, as shown by the update of the 2013 skills audit measures, have increased. • Confidence around some of the key learning objectives in the modules, such as sign- posting versus referral, has also increased significantly. However, confidence was lowest about helping people to complete transactions online. • Overall, staff were very confident about helping customers, on many of the essential soft skills such as being friendly and listening. Giving online support remained the area where confidence was lowest.

Oakleigh Consulting Ltd Page 3 of 8 1.4.Relevance and ability to apply learning, high The content of the LWDP was relevant to people’s day to day work and the vast majority already felt able to apply their learning to some extent. However, the full impact of the LWDP may not emerge for some time, even for those who have completed the training.

• Staff who started the training in September / October 2014 reported higher levels of relevance / ability to apply their learning than those who began in January / February 2015. • Likewise, those who have completed five, rather than only one, modules report higher levels of relevance and ability to apply the learning. • Staff gave a wide range of examples where the training had enabled them to be more confident and do things that previously they might not have felt able to do. • Some of the trainers felt that it was too early to assess impact, and were conducting their own evaluations later. These assessments should, if possible pursue similar approaches, and findings shared widely.

1.5.Impacts on customers, potentially significant Assessing the full extent of the impact of the training for customers requires a major piece of research. Based on the limited time and resources available within this project, it is clear that online access / services in libraries is very important to users and that without them the majority of them would be digitally excluded – because they have no computer and / or internet access at home.

• The mini case studies in the main report demonstrate the range of usage including: the essential – job searching; the practical – buying things; enhancing quality of life – making friends; skill development – learning to manage folders and source material.

1.6.Regional variations The research showed consistent regional variations on a number of measures, such as the ability to apply learning. It was not clear what was contributing to these differences, but they should be investigated and any critical success factors shared with other areas. The success stories and ability to innovate effectively, which were mentioned by stakeholders in some regions and libraries, also needs to be better understood and replicated elsewhere. SCL needs to work with PLAs and regions to investigate critical success factors, find effective mechanisms to increase sharing of learning, and examples of success so that that success can be replicated elsewhere.

1.7.Contract readiness improved The public library network is a major access point to online services for thousands of users, many of whom appear to use the services regularly. Government departments implicitly rely on libraries as a delivery channel for their digital services. Likewise, local authorities in many areas see libraries as part of their move to digital services and part of wider cost reduction and efficiency measures. The LWDP has raised the base level of skills and increased confidence among staff in many areas. This has strengthened the libraries’ capabilities and ability to deliver support for digital services. However, that strengthening of contract readiness is undermined by:

• Inadequate technology infrastructure, e.g. a lack of universal Wi-Fi, slow PCs and internet, as recognised in the Sieghart review. • A lack of entrepreneurialism and leadership in some areas, as raised by stakeholders.

Oakleigh Consulting Ltd Page 4 of 8 • Some staff not being as confident about the potential of digital capabilities, technologies and content.

1.8.LWDP a good start, but needs embedding and extending The LWDP is a good first step, but more and ongoing investment in skills and training is needed locally, regionally and nationally. New digital services and transactions are being announced by Government departments at regular intervals, with the expectation that libraries can deliver. Each of those services will require staff training so that staff can, in turn, support customers.

• Updating and maintaining staff skills, confidence and capability on the areas covered by the LWDP, will be necessary. People will forget the detail and the learning if they do not use it frequently or do not have time to refresh their skills. • Future training packages should include more differentiated learning and more ambi- tious content, so that the more experienced and confident can be challenged as well as the less confident. • The rate of technology changes, the number of new government digital services coming on stream, and staff turnover will all require regular and extensive skills updates. • Leadership, entrepreneurialism and innovation are essential at every level of the library network. While they are in clear evidence in some areas, others lack them. Investment in encouraging such capabilities is critical to future success of the public library network as a whole.

1.9.The LWDP process - not a good experience for all In any major undertaking there are elements that do not run as smoothly as hoped; the LWDP was no exception. An online training programme, especially one designed to enhance confidence among potentially reluctant or not very confident staff in delivering online services and support, should demonstrate the benefits of digital through a positive experience. For many LWDP users this was not the case; in this regard, the LWDP implementation process failed to ‘walk the talk’. The critical factor was the reliability of the online content / technology interface – repeated problems around completions, certificates, and links, and complaints about typos and grammar undermined staff perceptions and experience of the process. The problems were reflected in the numbers of comments about the technology / content interface on the module feedback, the forum, in the stakeholder interviews and the staff and trainer surveys. Other issues, which marred the success and potentially undermined the impact of the LWDP, include:

• The timescale – initial delays, followed by what was seen as a very rapid roll out, which put people, PLAs and local budgets under pressure. • Sometimes poor communication from SCL, especially around the requirement to use the online modules and the December changeover, but also a lack of clarity and slow decision making with suppliers. • The emphasis on e-learning while not a problem for the majority was an issue for some who preferred different approaches. It is also questionable to what extent behaviours and skills can be learnt via e-learning modules, rather than by observation, discussion and actual practice. When developing future training programmes, SCL, Tinder and Learning Pool will need to take these issues on board and to ensure that testing and implementation are more effective. SCL

Oakleigh Consulting Ltd Page 5 of 8 should also revisit the e-learning emphasis to accommodate more learning styles and to enable more effective behaviour change and soft skill development.

1.10. SCL capacity stretched SCL is to be congratulated on implementing a major programme on limited resources, particularly when all key officers have ‘full time day jobs’. However, if SCL is to take a more strategic role in leading on such projects in future, that capacity needs to grow; some of the communication issues may be the result of overextension of key people. There needs to be a small core, dedicated team who are able to lead on projects, raise the profile, create dynamic communications and arguments, and especially represent libraries’ interests on the forthcoming taskforce.

1.11. Value for money The evidence shows that there are a number of positives arising from the LWDP such as: the overall success of the programme in terms of design and roll out, the improvements in staff confidence, the relevance of the programme and staff applying the learning into practice. It is for SCL to decide whether the amount of the original investment has brought significant returns with regard to skills enhancement and changes in behaviours in the way staff deal with customers,

1.12. The taskforce - a once in a life time opportunity The forthcoming national taskforce provides a once in a lifetime opportunity for libraries to make the case, raise their profile, and win the investment they need to upgrade the technology and skills. SCL needs to be well prepared and able to speak with one voice, reinforced across the network. A technology upgrade is needed. Investment in technology is high on the agenda in the wake of the Sieghart report. SCL and its partners must find ways to capitalise on this momentum to best effect. Cuts to library services and library closures have dominated the public library narrative; this emphasis needs to change. Library services need to raise their profile, make the case so that they are recognised as part of the solution to innovative service delivery, knowledge generation and community development; not a potential target for cuts.

1.13. The 360o view shows a powerful base to build on Responses to six statements indicate that libraries have a strong base on which to build (see Figure 1 below). In summary: 1. The majority of respondents agree that libraries will need to innovate in future to meet the demands of the digital agenda, but external stakeholders were much clearer about it; 100% agreed strongly. 2. All stakeholders and the vast majority of staff and trainers see that the reputation of libraries in the community makes them the ‘go to’ place to meet the needs of the digitally excluded. 3. There is almost universal agreement that library staff are effective in supporting people to go online. 4. The LWDP is seen as making a significant impact on staff capabilities by all groups. Staff are the least convinced, with 69% agreeing overall. 5. The vast majority of respondents agree that libraries are well placed to provide digital services in the community in the future.

Oakleigh Consulting Ltd Page 6 of 8 6. Economic pressures are seen by all groups to be having an impact on what libraries are able to deliver, receiving over 70% agreement, and over 90% agreement among staff. Figure 1: Shared statements – a 360o view

4. Recommendations 1. When developing future training programmes SCL, and suppliers such as Tinder and Learning Pool, will need to ensure that testing and implementation are more effective, so that the technology / content interface problems are not repeated. 2. SCL should revisit the e-learning emphasis and build in more blended learning, with more opportunities for face to face learning. Where there is a strong emphasis on e-learning this should be supported by mentoring and coaching, especially to enable more effective behaviour change and soft skill development. 3. In developing future training programmes SCL, and its suppliers, should take into account the range of existing skills among staff so that the least confident can improve, and to ensure that the more confident and experienced also have their thinking challenged. 4. PLAs need to ensure that the skills represented in this LWDP become embedded in library staff induction, development and appraisal procedures. They will also need to be updated. • The existing LWDP should remain accessible so that staff can refresh skills, volunteers can enhance their skills, and new staff can confirm their skills. 5. SCL needs to work closely with government departments as new digital services are developed, so that relevant training materials are available in good time. • Before launch, Government departments need to provide ‘dummy accounts’, FAQs, and test sites complete with basic test cases so that library staff can develop the relevant expertise and confidence ahead of launch. 6. All government departments need to contribute to / invest in the cost of maintaining the technology infrastructure and staff training in libraries, which they implicitly rely on as part

Oakleigh Consulting Ltd Page 7 of 8 of developing digital services and reducing government costs. SCL needs to work via the Taskforce and directly with Government departments to achieve this. 7. SCL should invest in and capitalise on the skills and enthusiasm of the regional trainers. They represent a valuable resource which should be developed to a) expand training and mentoring capabilities for the existing LWDP; b) be the backbone for future training programmes; c) be developed as pro-active regional leaders in digital skills, advocacy and innovation. 8. SCL and the PLAs need to consult its network, e.g. the regional trainers to identify other training needs. Priority areas emerging from this research include communication – promoting the services, making the case; hands on experience of using new technologies; personal communication skills and awareness to adapt to different customer needs – language, literacy problems, hidden / learning disabilities; entrepreneurialism; leadership; and innovation. 9. SCL has achieved a lot, but needs to expand its core capacity if it is to expand its role. If SCL is to expand its role in taking a strategic lead on training and other areas, there needs to be a dedicated team, working only on SCL projects. They should then lead on training projects, create dynamic communications and arguments, raise the profile, and especially represent libraries’ interests on the forthcoming taskforce. • The taskforce is a once in a lifetime opportunity to effect significant change and achieve major investment for libraries. SCL needs to strengthen its capacity to make the case effectively. 10. SCL and / or the PLAs and / or external research experts should conduct further analysis of the evidence base generated by the update of the skills audit and the staff survey. In order to improve local awareness and strengthen local understanding of library staff skills, bench- mark each of the PLAs against a national average on key measures. Data from the surveys includes regional and PLA questions which can be used as filters to conduct regional and PLA analyses. 11. Invest in research to identify critical success factors. Regional differences in confidence and the ability to apply the learning emerged in the research, with two regions being con- sistently above average. These differences should be investigated further for critical success factors, as well as a wider piece of research examining critical success factors in spe- cific areas where innovation and digital capabilities are in advance of others. The findings should form a basis for skills development elsewhere. 12. Invest in customer research. Some local libraries and library services already commission their own research into library users, and develop case studies. • SCL should commission research to develop extended case studies, and wider research to assess changing needs and experience of services. • SCL should encourage open source approaches to publishing / sharing local research findings so that it can be used for meta-analysis or eventually big data analysis. 13. The need for libraries to innovate was seen as very important; but not seen by stakeholders as a core skill among all libraries. Process and service innovation often has more impact than product innovation. Libraries are being forced to do more with less; innovation and new business models are critical. Many of these options are known and some exist in some areas; but implementation and the necessary collaboration to achieve them elsewhere is often lacking. SCL needs to work with its stakeholders and on the national Task-

Oakleigh Consulting Ltd Page 8 of 8 force to identify the full range of options and to promote new ways of working. New training programmes will need to be part of this process. 14. In order to encourage innovation and to capitalise on the knowledge of trainers and key staff within the network, SCL should build a regional / national horizon scanning process for capturing and promoting forward looking opportunities. Such a network would identify new opportunities / threats, support innovation, provide a forum for learning and strategy development, help prioritise new developments and create a shared perspective across regions. It would also help to identify and nurture forward looking staff at every level. 15. SCL, PLAs, and the regional networks need to capitalise more effectively on the power of the national brand. The election on May 7th provides a significant opportunity for lobbying for investment, raising libraries’ profile, shouting about the successes; in short defining the debate rather than being defined by others’ agendas. • SCL needs to develop a clear communications strategy and a set of materials which the regions and PLAs can use. This needs to be an ongoing programme, not just a one off. • SCL needs to develop a clear digital media strategy both to deliver and to promote and share services and access. • Share and promote the good news stories locally, regionally and nationally within and outside the public library network. The reports contain examples of opportunities and success stories involving digital technologies. These and many others need to be shared more effectively. Options such as libraries as innovation centres, extended learning support centres for MOOCs with other organisations’ content e.g. language courses, research spaces, digitising local content, community hubs as discussed in the Sieghart review. The following table summarises the main organisations we see involved in carrying out the recommendations. Table 1 – Recommendation summary

Recommendations 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 SCL ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ PLAs ✓ ✓ ✓ ✓ ✓ Learning Pool ✓ ✓ Tinder ✓ ✓ Government ✓ ✓ ACE ✓ ✓ ✓

Oakleigh Consulting Ltd Page 9 of 8