The creative role of research The creative

2017 role of research Understanding research impact in the creative and cultural sector Authors Dr Kirstie Hewlett, Researcher, Cultural Institute and Policy Institute, King’s College London Katherine Bond, Director, Cultural Institute at King’s College London Dr Saba Hinrichs-Krapels, Senior Research Fellow, Policy Institute, King’s College London

Visualisations provided by Soapbox www.soapbox.co.uk

i King’s College London Foreword

There is no doubt that the UK excels at culture understood. This report provides evidence of the and creativity. People come from all over the immense scale of the collaboration that is going on, world, and also visit digitally, to experience, and the wide range of academic disciplines that are enjoy, and learn from our first class museums, involved, but it focuses most closely on the impact theatre, radio, music and the rest. The UK that these activities have within the world. is also at the forefront of the creative industries, Impact takes many forms. It is not only a high-growth sector of the economy that is now about the direct effect of research – delving essential to the nation’s prosperity. into creative practice in order to hone and Culture and the creative economy are by improve it – because it extends, significantly, definition driven by innovation and new into the public understanding of creativity knowledge, and University research is a primary and culture. And impact is felt in many places, source of new knowledge. Although under pressure with Higher Education itself changed by from international competitors, UK universities collaborative research processes. Research still fare well in global research league tablesi. produces new products and services, but Putting the cultural and academic sectors it also drives talent development. In addition, together, academic research plays a pivotal role it aids policy development: this report itself should in the cultural and creative ecosystem, generating be used to shape policy, providing as it does a full new ideas, creating new networks and nurturing recognition of the importance both of research and talent development. However, as the Arts and the creative industries, and a better understanding Humanities Research Council notes: “The UK of the connections between them. boasts a world-leading position in both arts and In particular, this report disabuses us of the humanities research and the creative industries. notion that the connection between research [But] realising fertile connections between the and the creative economy is a simple matter two is neither inevitable nor easy.” (AHRC) of linear transfer from idea to implementation. I therefore welcome this new report from Collaborations between Higher Education King’s College London into the Creative Role and the creative sector are complex, and their of Research; a report that improves our knowledge success depends on the quality of relationships, about how research affects and impacts both the creation of networks, and the stimulus the cultural and creative sectors and the wider generated by collaboration as much as on the world, thereby helping us to understand how those formality of research processes and methodologies. ‘fertile connections’ can be increased in number I hope that the report will inspire researchers and quality. from a wide range of academic disciplines to The report analyses 1582 impact case studies embrace new opportunities to partner with the that were submitted nationally to the Higher creative and cultural sector and, at the same time, Education Funding Council for England illuminate for artists and creative professionals (HEFCE) as part of the 2014 Research Excellence the many ways in which academic research can Framework. It provides a taxonomy for eight inform and develop their practice. areas of impact, including influencing industry Research is an adventure, a step into the realm practice, transforming public understanding, of uncertainty; and it often leads to unexpected developing audiences, informing curricula and conclusions and unlooked-for destinations. It is pedagogy, promoting social change, developing that spirit of open inquiry that produces advances policy, growing the economy, and improving in the human spirit, which ultimately are more health and wellbeing. It discusses the activities important than financial return. Impact emerges and partnerships that gave rise to those impacts, in this report as not only a utilitarian measurement and offers an account of the forms of evidence (which we need), but also as a multi-faceted used to support the impact claims. enquiry, full of nuance and interest. Academics and creative practitioners have always i timeshighereducation. com/world-university- forged relationships, collaborated, borrowed from Prof. John Holden, Cultural Fellow, King’s rankings/world- and influenced one another; but in the past these College London, Visiting Professor, University reputation-rankings- relationships often came about by chance, and of Leeds and Honorary Professor, University 2017-results-announced their characteristics and consequences were poorly of Hong Kong

The creative role of research ii The ways in which universities contribute to the on research questions as well as new mechanisms cultural sector are well documented – through the to communicate findings to a broader public. generation of relevant knowledge, as partners and The contribution that arts and culture can collaborators, through provision of publicly accessible make to problem solving and innovative spaces and, not least, through the education of creative thinking is in no way restricted to disciplines graduates (who also find employment in sectors way within the arts and humanities. Research beyond the creative industries). The contribution at Michigan State University found that creative of the cultural sector – its people and organisations – experiences are ‘significantly correlated with to universities is perhaps less well understood. Since producing patentable inventions and founding arriving at King’s five years ago, I’ve been privileged new companies’ and that lifelong participation to work with colleagues across the university to in the arts ‘yields the most significant impacts develop a distinctive approach to culture and cultural for innovators and entrepreneurs’. The university’s partnerships, embedding collaborations that have new Strategic Vision is underpinned by this been transformative in both education and research. approach to partnership and its potential to For students, these partnerships help to increase help King’s achieve its ambitions on the road creativity, flexibility and employability, developing to its 200th anniversary, in 2029. their networks and offering a chance to test learning in environments beyond King’s. For academics, Deborah Bull, Assistant Principal (London), art and artists can provoke imaginative perspectives King’s College London

During my time at Arts Council England we All well and good. But perhaps the research I value became a truly research-driven organisation, thanks above all recognises the serendipitous discoveries to an expanded unit in Manchester which informed that intriguing research projects, with no obvious all our policy initiatives. At the same time we saw initial utility, eventually yield. So, yes to research; the crossover between technology and creativity yes to funded research and please don’t make take flight...with games, Virtual Reality and other it solely utilitarian. embryonic sparks yet to be fully realised. To this I might add the diligent consumer research we Sir Peter Bazalgette, British television executive rely on in media to shape our programme offer. and author

The King’s College London survey of the creative report provides a platform for discussion as to how role of research shows the extent to which academic the huge body of existing research informs creative research is engaging with the cultural and creative practice and public understanding. sector and role the creative sector plays in maximising the impact of academic research. The opportunities Jenny Waldman CBE, Director, 14-18 NOW, and benefits of such collaboration are mutual. This Cultural Fellow at King’s

I am one of a number of academic scientists who in public engagement. I continue to carry out research have been working for several years to integrate in theoretical physics within the nuclear physics group conversations about science – by which I mean, at the University of Surrey and I also publish papers broadly, the scientific method of enquiry, as well in a new field of research called quantum biology. as the discoveries of modern science – into popular For me, the joys of fundamental research and of culture. I included my work in communicating enthusing the wider public about what I do are science through my writing and broadcasting into integral to each other. an impact case study for my physics department’s submission into the last REF exercise. As such Jim Al-Khalili OBE, Professor of Physics and I had to stress just how integral my research Professor of Public Engagement in Science, has been in enabling me to achieve what I have University of Surrey, Cultural Fellow at King’s

iii King’s College London In my work with King’s I have had first-hand theatre, the breadth and depth of new work emerging experience of the ground-breaking new ideas that at the interface between Higher Education and the can be catalysed through the collaboration of scholars, creative and cultural sector in the UK is innovative, artists and arts professionals. From a new vision for pioneering and vital. the future of Carnival, to the creation of a Museum of Homelessness, to enhanced communication between Hilary Carty, Director, Clore Leadership young people and clinicians through interactive forum Programme, Cultural Fellow at King’s

As somebody working in the creative industries, research doesn’t just give us a better understanding I’m acutely aware that we still rely far too much of what works in our world and why, it’s the window on anecdote and optimism rather than solid that allows the academic world to better understand research and analysis as a basis for thinking the needs, recognised and unrecognised, of the about the future. For a sector whose success creative economy. depends, almost by definition, on the generation of new knowledge and an understanding of new John Newbigin OBE, Chairman, and unexpected dynamics, that’s crazy. Robust Creative England

The roles of research and collaboration are a rich new audiences for their work. The higher education – and urgent – part of the cultural and educational sector in the UK is a brilliant crucible for forging new landscape in the UK today. Many artists and ideas and partnerships: in dance, music, literature, academics regularly come together to share their visual arts and more. knowledge, and to create multi-arts projects that are innovative, thought-provoking and entirely unique. Professor Joanna MacGregor OBE FRAM, Even better, their explorations often cross cultural, Head of Piano, Royal Academy of Music, intellectual and national boundaries, as well as create University of London, Cultural Fellow at King’s

The hope is that these growing creative dissemination of research, of new forms catalyzations, inter-disciplinary partnerships of academic output that will utterly explode and deep collaborations with culture will open traditional intellectual silos. up the academy to the broader public in ways that might truly alter our understanding of research Dr Gus Casely-Hayford, British curator, and education. We are entering an age of burgeoning cultural historian and Director of the National cross-disciplinary collaboration – of new creative Museum of African Art in Washington, mechanisms for the analysis, presentation, and Cultural Fellow at King’s

Making art is in many ways like research: it is and academics alike, providing challenge and a process of exploration, experiment, and discovery. stimulus. The opportunity to reflect on practice I look forward to closer collaboration between and incubate new ideas leads to unexpected outcomes creative practitioners and multi and interdisciplinary and innovation. researchers. Collaboration and exchange, based on mutual respect and trust, can enrich arts practitioners Sue Hoyle OBE, Cultural Fellow at King’s

The creative role of research iv ‘Madness & Modernity’, © Wellcome Collection

v King’s College London Contents

Executive summary 01

1  Introduction 04  1.1 Background and context 05 1.2 A pproach 06 1.3 Caveats 07

2 Research impact in the creative and cultural sector 08 2.1 From which research disciplines did impact originate? 09 2.2 What form does research impact take? 13 2.2.1 Influencing industry practice 13 2.2.2 Transforming public understanding 13 2.2.3 Audience development 17 2.2.4 Informing pedagogy and curricula 17 2.2.5 Social change 17 2.2.6 Policy development 19 2.2.7 Economic growth 19 2.2.8 Improving health and wellbeing 20 2.3 Research impact in the creative and cultural sector: key findings 21

3 Pathways to impact 22  3.1 What were the mechanisms through which research impact was achieved? 23 3.2 What type of cultural organisations did researchers collaborate with? 30 3.3 To what extent did researchers initiate collaborations with the creative and cultural sector? 32 3.4 How long did it take for research to achieve impact? 33 3.5 Pathways to impact: key findings 35

4 Evaluating and evidencing research impact 36  4.1 What types of evidence are used to evaluate research impact? 37 4.2 Evidencing reach 39 4.3 Evidencing significance 40 4.3.1 Industry status 41 4.3.2 Audience or user response 41 4.3.3 Informing discourse 41 4.3.4 Financial indicators 42 4.3.5 Testimonials 43 4.4 Evaluating and evidencing research impact: key findings 45

5 The creative role of research: final reflections 46

References 48  6.1 Secondary literature 48 6.2 REF 2014 case studies cited 48

Appendices 50 Appendix 1: Impact in REF 2014 50 Appendix 2: Coding in NVivo 50 Appendix 3: Topic model labels and associated words 50

The creative role of research vi ‘Performing Migration’ © Mark Loudon

vii King’s College London Executive summary

Executive summary

The Research Excellence Framework (REF) is a national assessment exercise that measures the quality of research undertaken across all UK universities. In 2014 this included, for the very first time, an enquiry into the impact of academic research beyond Higher Education (HE) in the form of brief narrative case studies.

Impact was defined by the Higher Education Headline findings Funding Council for England (HEFCE) as ‘an effect on, change or benefit to the economy, For the creative and cultural sector society, culture, public policy or services, health, • The creative and cultural sector plays the environment or quality of life, beyond a significant role in maximising the academia’. The two key criteria highlighted by impact of research in the UK. REF 2014 HEFCE for the assessment of impact were reach demonstrated that there was a degree of and significance. Though the definitions of these interaction with the sector appearing in a terms differed between each academic panel quarter of the total number of impact case tasked with scoring research impact (see Section studies available for analysis. 4.1), they broadly focus on the scale and diversity • Informing creative practice and transforming of beneficiaries, and the ways in which academic public understanding were the primary areas research has made a difference for an individual impacted through researcher involvement or community, policy or practice. in the creative and cultural sector in REF 2014. A total of 6,975 impact case studies were • The creative and cultural sector was linked submitted to the REF 2014 exercise, most to research from a wide range of academic of which are now available in the public domain disciplines. While there are strong disciplinary in a searchable online database. This database links between research in the arts and provides a rich source of information about humanities, considerable interdisciplinary the ways in which researchers have engaged activity also takes place between the sector with projects and programmes in the creative and all fields of academic research. and cultural sector, many of which had their • Creative and cultural sector collaboration with genesis long before impact was incorporated HE enabled knowledge exchange on a broad into research assessment practices. range of topics, from science and politics to This report seeks to understand the relationship human rights and social responsibility, which between all fields of research and their impact in turn facilitated new creative responses within the creative and cultural sector. Based on to issues of social and political significance. an analysis of 1,582 impact case studies from the above database that were identified in this study as For the HE sector being germane to the sector, the report illuminates • The impact of research in the arts and the different types of impact described, discusses humanities, the research discipline primarily the activities and partnerships that gave rise to associated with creative and cultural sector, those impacts, and offers an account of the forms scored more highly than any other discipline of evidence used to support the impact claims. in REF 2014, having received the highest It does not, however, seek to make judgements concentration of world-leading rankings (4*) about the quality of the REF impact projects for the reach and significance of its research. described, nor does it contain guidance on writing • HE partnerships and collaborations with impact case studies. Rather its goal is to better local and regional cultural organisations understand the longstanding ways in which were far outnumbered by projects describing academics and creative practitioners have forged work with a national or international partner. relationships, collaborated, borrowed from and This indicates the important role that national influenced one another. cultural organisations play in bringing research We hope that The Creative Role of Research will to wider audiences but also suggests there is inspire researchers from a wide range of academic an as yet untapped opportunity for universities disciplines to foster new opportunities to collaborate to develop more work with sector organisations with the creative and cultural sector and, at the based outside of the capital. same time, illuminate for artists and creative • Research impact on or through the creative professionals the manifold ways in which academic and cultural sector is seldom achieved through research can inform and develop their practice. a single, focused activity or mechanism: it is

The creative role of research 01 Executive summary

the product of various interdependent, yet often What form did research impact take in or through disparate activities spanning multiple areas. the creative and cultural sector? • Research impact was not evaluated by just • There were eight key areas in which researcher one yardstick but through a web of indicators. engagement with the creative and cultural sector While evaluation of reach was largely had an impact. Listed by the most prominent evidenced through quantitative data, type first, these include: evaluation of significance was more nuanced, –– influencing industry practice ranging from evidence that acted as an –– transforming public understanding endorsement or a pledge to think or act –– audience development differently in the future, to reflections on –– informing pedagogy and curricula longer-term change. However, across all –– social change evidence types, evaluation of research impact –– policy development was most often anchored in the immediate –– economic growth period after an event or activity, rather than –– improving health and wellbeing. over a longer term. • ‘Influencing industry practice’ within the creative and cultural sector was the primary area Using the REF alone to understand the nature in which research had an impact, with research of exchanges between universities and the creative collaboration and practice-led research making and cultural sector is limited to showing what a marked contribution to innovative practice. research has done for the creative and cultural • The creative and cultural sector played sector and not what the sector can do for research. an important role in ‘transforming public Developing a two-way dialogue may enhance understanding’ by translating research into understanding of the benefits of partnership impacts on opinions and behaviour, and to the development of research and the generation brokering wider audiences for academic of new knowledge. research (‘audience development’) through the translation of research findings into more Further breakdown of key findings widely accessible formats. The prevalence Which academic research disciplines have an impact and quality of such impacts suggests that on or through the creative and cultural sector? research collaborations are being utilised to • While research impact on and through the address issues of social and political significance creative and cultural sector had close ties by combining research with creative practice. with research in the arts and humanities, the sector nevertheless interacted with almost How did researchers engage with the creative all fields of academic study. 75 per cent of the and cultural sector? case studies that were identified in this study • The partnerships that researchers engaged as being relevant to the sector came from the in, be it on a local, national or international arts and humanities, with the remainder coming scale, were the product of both direct and from other fields. indirect engagement with the creative and • The academic disciplines with the greatest cultural sector: interaction with the creative and cultural sector –– 89 per cent of case studies described a in REF 2014 were: collaboration that researchers had initiated –– English language and literature (15 per cent directly or proactively contributed to. sample, 238 case studies) –– 67 per cent reported researcher involvement –– history (14 per cent sample, 222 case studies) in an advisory capacity or as a featured expert. –– art and design: history, practice and theory –– 18 per cent included examples of indirect (11 per cent, 176 case studies). forms of collaboration, with no contact between the researcher and the practitioner. Beyond the arts and humanities, other prominent disciplines were: –– geography, environmental studies and archaeology (5 per cent sample, 79 case studies) –– psychology, psychiatry and neuroscience (3 per cent sample, 41 case studies) –– physics (1 per cent sample, 23 case studies).

02 King’s College London Executive summary

What was the geographic spread of researcher What were the limitations of the evaluation methods engagement with creative and cultural and evidence used to describe impact? sector partners? • The narrative approach to describing reach • In REF 2014, researchers primarily engaged precluded an analysis of the data gathered with cultural organisations in the UK that have on anything but a thematic basis due to the a national or London remit. Of the regional inconsistency of the quantitative measures used. creative and cultural partners in the UK cited by • Information about reach offers little indication researchers, 79 per cent were based in England. of the demographics represented by quantitative • Just over a third of collaborations or partnerships measures; this is essential if we are to fully grasp with the sector were located outside of the the public benefit of research. The case studies UK, spanning six continents. These were also offer limited evidence of how researcher concentrated in Europe, with 47 per cent of engagement has contributed to wider economic international partner organisations situated activity within the sector: researchers seemed in one of 19 European countries. reluctant to use financial indicators if there was not a clear and causal link between the What were the activities or mechanisms that enabled research and income generation, even in cases research to have an impact? where wider economic activity was likely • 68 discrete activities or mechanisms (see Section to have occurred. 3.1) led to research impact on or through the • The evaluation of significance was predominantly creative and cultural sector. These research anchored in the immediate period after exposure, mechanisms were rarely cited in isolation, with typically communicating an endorsement an average of six mechanisms being presented or a change in opinion for the short term. per case study. • The most effective examples of change confirm • The most prevalent impact mechanism cited that, even within the constraints of the REF’s was public talks, which occurred in 43 per cent reporting guidelines, evaluation is not something of case studies analysed. that occurs at a set point at the end of a project, • Activities related to ‘music, performing but is a reflexive process that is more usefully and visual arts’, ‘film, TV, video, radio and set in motion at the beginning of a project, and photography’ and ‘museums, galleries and extends long beyond its end. libraries’ emerged as hot spots for sector engagement with research. Key areas identified for further investigation include: • An enquiry into how the strong, yet How did researchers evaluate and evidence impacts underutilised links between practice-led involving the creative and cultural sector? research in the arts and mental health services • 18 distinct evidence types (see Section 4.1) are helping to improve provision for mental were specifically used to describe or evaluate the health illness. presence and extent of research impact involving • A review of the ways in which activities the creative and cultural sector. in the creative and cultural sector help • Impact generated from creative projects was build communities. not evaluated by just one yardstick but through • An analysis of whether the relative deficit a web of indicators, with an average of five of impacts on policy, economy and health evidence types supporting each case study. were a product of barriers to working in • Evidence of significance was the most common these areas with and through the creative means of evaluation, underpinning the impact and cultural sector. narratives in 97 per cent of the case studies • A review of the ways in which engagement with in the sample; evidence of reach appeared research may play into a bigger impact strategy in 88 per cent of the case studies analysed. for creative and cultural sector organisations. • Evaluating research impact in the creative • An analysis of how frequently creative and and cultural sector was as much about validating cultural organisations draw upon academic and contextualising the activities that enabled research without direct collaboration with an impact as it was about supplying evidence the author of the research. that corroborated impact directly. While testimonials were the most common way in Answers to these questions fall beyond the remit which a change in opinion, practice or policy of the REF, but are critical for gaining a more was illustrated, other forms of evidence, such complete picture of how research informs work as indicators of reach, critical reviews or levels in the creative and cultural sector. of income, predominantly communicated the scale, quality or repute of the activities that enabled an impact.

The creative role of research 03 1. Introduction

–1– Introduction

‘Latin London: Improving the Visibility of Latin Americans in the UK’ © Cathy McIlwaine

04 King’s College London 1. Introduction

Researchers and creative practitioners have a long history of collaboration, yet it was not until 2014 that the impacts and fruits of these collaborations were formally assessed within Higher Education (HE), following the introduction of the ‘impact’ agenda to the UK government’s Research Excellence Framework (REF). This report uses the evidence gathered as part of this exercise to chart the current state of the interactions between HE and the creative and cultural sector, identifying opportunities to strengthen those links and more effectively measure their impact.

The first nationwide assessment of the ‘impact’ impacts involving the creative and cultural Departments in the of research beyond the academic community sector, and what were the limitations? arts and humanities had was launched in the UK in 2014 as part of the • What was the time lag between the originating the highest concentration government’s Research Excellence Framework research and impact, and at what stage in the of world-leading (REF). The REF is a national assessment exercise research lifecycle did impact occur? impact rankings, with that measures the quality of research from all this discipline seeing UK universities, the results of which inform the largest cluster of the allocation of government research funding. 1.1 Background and context departments awarded In 2014, for the very first time, this included 4* rankings for their an enquiry into the reach and significance The REF was designed to show accountability submissions to the of academic research beyond HE in the form for the use of taxpayers’ money for research and impact component of brief narrative impact case studies. to inform the allocation of government funds to of REF 2014. 6,975 impact case studies were submitted to research in a way that was proportionate to public REF 2014; all were subsequently made available benefit. Historically, this was achieved through in an online database1 commissioned by the peer review of outputs such as books or journal Higher Education Funding Council for England articles and assessing the institutional environment (HEFCE), with the exception of 296 case studies of in which the research had been cultivated; a confidential or sensitive nature. Far from being the however, in the 2014 exercise this was broadened residue of an administrative exercise, this repository to include an assessment of the impact beyond yields unprecedented and rich information about the academia (see Appendix 1). longstanding ways in which academics and creative Impact was defined in REF 2014 as ‘an effect practitioners have forged relationships, collaborated, on, change or benefit to the economy, society, borrowed from and inspired one another. culture, public policy or services, health, the The role of the creative and cultural sector environment or quality of life, beyond academia’.3 in maximising the impact of research was clearly This included, but was not limited to: ‘the apparent in the REF. Not only was the sector activity, attitude, awareness, behaviour, capacity, present in just under a quarter of all impact case opportunity, performance, policy, practice, process 1 impact.ref.ac.uk/ studies available for analysis, research from the or understanding; of an audience, beneficiary, CaseStudies arts and humanities, the discipline most closely community, constituency, organisation or 2 15 departments from associated with the creative and cultural sector, individuals; in any geographic location whether the arts and humanities performed particularly highly in terms of impact: locally, regionally, nationally or internationally’. received the top 4* ranking for impact, departments in the arts and humanities had the Impacts on research itself, or on advancing which indicates that the highest concentration of world-leading impact academic knowledge within the HE sector, were not case studies submitted rankings, seeing the largest cluster of departments eligible, neither were impacts on students, teaching were world-leading in terms of their originality, awarded 4* rankings for their submissions to the or other activities within the submitting institution. significance and rigour. impact component of REF 2014.2 Impact was reported as free-text case studies, The next highest was the With this in mind, our guiding questions typically four pages in length. These case studies social sciences with 12 departments, followed were as follows: were submitted into one of 36 disciplinary areas, by biological sciences • Which academic research disciplines have which were assessed by appointed experts in the and medicine with 11. an impact on or through the creative and field, working under the guidance of four main Just two departments 4 within physical sciences cultural sector? panels (see Table 1). Each case study was divided and engineering were • What form did research impact take? into five sections: awarded 4*. See results. • How did researchers engage with the creative 1. Summary of the impact ref.ac.uk/ and cultural sector? 2. Underpinning research 3 HEFCE, Assessment • What was the geographic spread of researcher 3. References to the research Framework and Guidance engagement with creative and cultural sector 4. Details of the impact on Submissions (2011): p. 48. Available at ref.ac.uk/ partners? 5. Sources to corroborate the impact. pubs/2011-02/. • What were the activities or mechanisms that enabled research to have an impact? Since the results were published in December 4 ref.ac.uk/ • How did researchers evaluate and evidence 2014, interest has been building in the body of case

The creative role of research 05 1. Introduction

Table 1: Panels and academic disciplines does the sector enable? What role do researchers Panel A: Biological Sciences and Medicine fulfil within the sector, and how do we measure Clinical Medicine the reach and significance of this activity? As the Public Health, Health Services and Primary Care first research assessment exercise in which impact Allied Health Professions, Dentistry, Nursing and beyond academia was measured, REF 2014, Pharmacy while not without its flaws, offers some of the Psychology, Psychiatry and Neuroscience most comprehensive insight ever gathered to Biological Sciences enable us to begin to answer these questions. Agriculture, Veterinary and Food Science Panel B: Physical Sciences and Engineering 1.2 Approach Earth Systems and Environmental Sciences Chemistry The selection of case studies for analysis in Physics this report was informed by previous research Mathematical Sciences undertaken in the Policy Institute at King’s Computer Science and Informatics College London, from which we identified Aeronautical, Mechanical, Chemical and 1,582 case studies that were germane to the Manufacturing Engineering creative and cultural sector. Electrical and Electronic Engineering, Metallurgy With more than six million words in the ‘Details and Materials of the impact’ sections alone, the Policy Institute Civil and Construction Engineering study used topic modelling to uncover hidden General Engineering themes – or ‘topics’ – that occur throughout the Panel C: Social Sciences 6,679 impact case studies available for analysis. Architecture, Built Environment and Planning 60 impact topics or areas where research influences Geography, Environmental Studies and Archaeology society, such as medical ethics, climate change, Economics and Econometrics clinical guidance, and women, gender and minorities, were identified. Business and Management Studies Drawing on the outcomes of this topic modelling Law exercise, the research sample for this report was Politics and International Studies selected by collating all 1,582 case studies tagged Social Work and Social Policy with topics relating to the creative and cultural Sociology sector. The selection of case studies was therefore Anthropology and Development Studies not based on the underpinning research or Education submitting department, but on themes described Sport and Exercise Sciences, Leisure and Tourism in the body of the case study that are relevant Panel D: Arts and Humanities to the sector. Area Studies Of the 60 topics identified in the first report, nine Modern Languages and Linguistics topics were selected as being relevant to the sector: • ‘Arts and culture’ (231 case studies) English Language and Literature • ‘Cultural and heritage preservation’ History (171 case studies) Classics • ‘Film and theatre’ (139 case studies) Philosophy • ‘Historical archives’ (285 case studies) Theology and Religious Studies • ‘Literature’ (166 case studies) Art and Design: History, Practice and Theory • ‘Media’ (630 case studies) Music, Drama, Dance and Performing Arts • ‘Museums and exhibitions’ (192 case studies) Communication, Cultural and Media Studies, • ‘Music, dance and performance’ Library and Information Management (134 case studies) • ‘Print media and publishing’ (253 case studies). studies submitted to the impact component of REF After closer reading, three of these topics were 2014 as research objects in their own right. Within renamed for the purpose of this report to better 5 National Co-ordinating the creative and cultural sector, for example, the reflect the content of the case studies that they Centre for Public Engagement, Share Museum University Partnership Initiative (MUPI) identify. In the remainder of the report, ‘Cultural Academy, Museum- published a report in 2016, in which the case and heritage preservation’ will be referred to University Partnerships in studies were used to survey the current state of as ‘Heritage and conservation’, ‘Literature’ as REF Impact Case Studies: A Review (2016). Available partnerships between museums and researchers in ‘Fiction, poetry and literature’, and ‘Arts and at publicengagement. the UK.5 Yet the bigger picture of research impact culture’ as ‘Visual arts’. ac.uk/sites/default/files/ in the creative and cultural sector has remained, In addition to analysing the entire sample publication/mupi_ref_ review.pdf until this report, unexplored. What types of impact quantitatively to understand the relationship

06 King’s College London 1. Introduction

‘Stonehenge Riverside Project’ © Mike Parker Pearson

between the discipline of the originating research 3. A small number of case studies relating to the and the impact area, over a quarter of the sample creative and cultural sector may not have been (417 case studies) was read and analysed in more picked up in the topic modelling, the purpose depth (see Appendix 2 for further information about of which is to identify high-level themes and the analytical ‘coding’ process).6 Besides capturing patterns in the data.8 Topic modelling is based on the earliest date cited under the ‘References to the probability and therefore subject to chance and research’, this qualitative analysis focused on the variation; as such, the output can differ slightly narrative information supplied under the ‘Details each time the analysis is run. What is captured of the impact’, in which we captured: also depends on how many topics are specified • Types of impact reported and how well suited this number is to the data. • Activities and mechanisms that enabled Accordingly, certain areas of the sector may not the impact be as well represented as others. • Sector partners 4. REF 2014 was the first time that impact had • Types of evidence used to evaluate impact. been included as a category in the assessment 6 We began with a sample of research; as a consequence, there was of 432 case studies. We analysed this section of the case study considerable uncertainty across the HE sector This included 48 case studies from each of submission form as a standalone statement, unless it about how impact would be assessed and what the nine impact topics required further contextualisation. The supporting would and would not count. This is likely to that were identified as information presented under ‘Sources to corroborate have led to risk aversion and to a curatorial being relevant to the sector via a process of the impact’ was not included in the analysis, as it approach by universities to their impact case topic modelling. 16 case tended to repeat what was already described in more study submissions. Many examples of research studies from each topic detail under ‘Details of the impact’, and frequently impact will have therefore gone unreported and were selected where it was identified as the made reference to sources that were not in the public as such are unavailable for analysis. most relevant topic in the domain, such as email exchanges or confidential 5. Once research has been released into the public case study’s narrative, data collected by partnering organisations. domain, it is only ever possible to capture a 16 where it was the second most relevant partial picture of how it is used to inform policy, and 16 where it was the practice or production beyond the university third. The sources were 1.3 Caveats walls, thereby limiting a fully comprehensive otherwise selected at random. During the understanding of research impact overall. analysis we also identified Using the REF case studies as material for analysis and excluded 15 false comes with a number of important caveats. These Further, we do not link case studies to their positives, leaving a final sample of 417 case limitations should not undermine the analysis but ranking by the expert panels. Since information studies. help to frame its interpretation. about the ranking of individual impact case 1. The case studies can be a problematic source for studies is not in the public domain, our criteria 7 See ref.ac.uk/pubs/ analysis as they were written for assessment, not for showcasing case studies was instead focused 8 For further information analysis. The impacts reported were therefore on those that were well evidenced, that reflected about the topic modelling selective, compiled according to a specific set of trends in reporting, or that showed unusual or underpinning our analysis, 7 see: King’s College London, requirements and guidelines that were published inspiring approaches to working with the sector. and Digital Science. to assist in preparing impact case studies. This What this report offers is not, therefore, tactical The Nature, Scale and means that some activities within the sector may approaches to perform well in the REF, nor does Beneficiaries of Research Impact: An Initial Analysis have gone unreported. it attempt to assess the quality of the case studies of Research Excellence 2. Impact was reported in a free-text format. As that underpin the analysis. Rather, we have valued Framework (REF) 2014 a result, much of the information captured is the introduction of the impact agenda to REF 2014 Impact Case Studies (2015). Available at kcl.ac.uk/ described inconsistently, especially quantitative as a unique opportunity to take stock of the ways sspp/policy-institute/ and geographic information. Comparisons of the in which academics and creative practitioners have publications/Analysis-of- scale or reach of the impact were therefore not worked together, long before it was incentivised REF-impact.pdf possible with the data available. by assessment practices.

The creative role of research 07 2. Research impact in the creative and cultural sector

–2– Research impact in the creative and cultural sector

‘Cultural Policy and Practice Exchange between Britain and Brazil’ © Ellie Kurttz

08 King’s College London 2. Research impact in the creative and cultural sector

This section identifies the nature of research impact in the creative and cultural sector, both in terms of the fields of research flowing into the sector and the types of impact generated from within the sector. Section 2.1 offers a quantitative analysis of all 1,582 case studies relating to the creative and cultural sector, tracing the research disciplines that had an impact on or through work in the sector. Section 2.2 identifies the types of impact claimed, presenting a qualitative analysis of a smaller sample of 417 case studies, supported by illustrative examples. The case studies that are featured are intended to reflect common, innovative and unusual ways in which researchers have achieved impact; they were not selected on the basis of their performance in the REF, nor are they necessarily indicative of how impact was claimed across the sample as a whole.

2.1 From which research disciplines yet outside of the arts and humanities, areas of We found that the did impact originate? research that achieved impact via the media were creative and cultural psychology, psychiatry and neuroscience (24 sector is linked to The research disciplines that underpinned impacts case studies), physics (20 case studies), politics research from a wide involving the creative and cultural sector were and international studies (20 case studies) and range of academic inherently multidisciplinary. We found that the sociology (13 case studies), among several others. disciplines. sector is linked to research from a wide range • Some common impact pathways originating of academic disciplines, with 75 per cent of from research disciplines outside the arts and impacts originating from research in the arts and humanities linked case studies from geography, humanities and the remaining 25 per cent coming environmental studies and archaeology to from diverse disciplinary areas. impacts within ‘Heritage and conservation’ Figure 1 shows that each impact topic was (55 case studies) and ‘Museums and exhibitions’ associated with research from at least two (13 case studies). academic panels, with six of the nine topics linked • Among the less common impact pathways, to research from all four. there are intriguing examples of research Drilling down further to a disciplinary level, reaching unexpected areas of the sector, such we found that: as work from the biological sciences feeding • Just one of the 36 disciplines listed in Table 1 into the topic ‘Visual arts’ (1 example), or (see page 06), civil and construction engineering, from computer science and informatics into was absent in our sample. the topic ‘Film and theatre’ (1 example). • There were clear links between disciplinary areas within the arts and humanities and related This interdisciplinarity is reflected on a case study areas of the creative and cultural sector, such level in the alluvial diagram in Figure 2. The as research from history having an impact connections drawn in Figure 2 reveal associations within ‘Historical archives’ (122 case studies), between certain arts and humanities disciplines and or art and design having an impact within the related areas of the sector, albeit embedded within ‘Visual arts’ (100 case studies). The impact topic a muddle of interdisciplinary activity. For example, ‘Media’ spanned the most academic disciplines. a thick yellow line connects the research disciplines Within the arts and humanities, history (100 of music, drama, dance and performing arts on case studies) had the greatest impact on media; the left axis with the impact topic ‘Music, dance

Figure 1: Representation of academic panels per topic

Visual Arts Biological Sciences Music, Dance and Performance and Medicine Media Physical Sciences and Engineering Museums and Exhibitions Social Sciences Historical Archives Arts and Humanities Fiction, Poetry and Literature Heritage and Conservation Film and Theatre Print Media and Publishing 020406080 100 % case studies tagged as topic

The creative role of research 09 2. Research impact in the creative and cultural sector 2. Research impact in the creative and cultural sector

Figure 2: Alluvial diagram connecting academic discipline and impact topic For each case study, up to three lines connect the originating academic discipline with the three most relevant topics. The number of lines flowing out of each point on the axes therefore reflects the number of times it occurs across the sample. The colour coding is generated by the panel grouping of the academic discipline.

10 King’s College London The creative role of research 11 2. Research impact in the creative and cultural sector

Figure 3: Representation of arts and humanities subjects in creative and cultural sector impact topics

Theology and Religious Studies % in sample Area Studies % not in sample Philosophy Communication, Cultural and Media Studies, Library and Information Management Modern Languages and Linguistics Art and Design: History, Practice and Theory History English Language and Literature Music, Drama, Dance and Performing Arts Classics 020406080 100 % total case studies submitted

and performance’ on the right, indicating that this linked to the corresponding topic ‘Fiction, poetry pairing occurred in over half (97 out of 172) of and literature’ (117 out of 238), this field the case studies in our sample from this academic of research also impacted upon: discipline. But while there was a strong connection • ‘Media’ (87 case studies, 37 per cent) between research in this field and related activity • ‘Public engagement’ (69 case studies, 29 per cent) within the sector, research originating from music, • ‘Historical archives’ (49 case studies, 21 per cent) drama, dance and performing arts also had an • ‘Community and local government’ impact in many other areas: (20 case studies, 8 per cent) • The ‘Visual arts’ had a strong connection • ‘Print media and publishing’ (19 case studies, to research from music, drama, dance and 8 per cent). performing arts (44 case studies, 26 per cent of the case studies submitted in this category), as did There were areas of research in the arts and ‘Media’ (36 case studies, 21 per cent), ‘Film and humanities that were not, however, well theatre’ (32 case studies, 19 per cent) and ‘Public represented in our sample. As shown in Figure 3, engagement’ (32 case studies, 19 per cent). Theology and religious studies, and Area studies • Research in music, drama, dance and performing were the least well represented. The remaining arts also impacted beyond the creative and disciplines from the arts and humanities, by cultural sector, the most prominent topics being contrast, had an average representation within our ‘Community and local government’ (16 case sample of 75 per cent, Philosophy being the lowest studies, 9 per cent of the case studies submitted at 51 per cent and Classics the highest at 92 per in this category), ‘Schools and education’ (7 case cent. Case studies from under-represented research studies, 4 per cent) and ‘Informing government disciplines would benefit from further investigation policy’ (6 case studies, 3 per cent). to identify hot spots for their impact beyond the • Of the 134 case studies that impacted on ‘Music, creative and cultural sector. dance and performance’ practice, 28 per cent Figure 2 also captures the flow of research from similarly originated from research within disciplines beyond the arts and humanities into indirectly related fields: English language and the sector. This is visible in the combination of literature was most prominent (12 case studies, red, green and blue lines flowing into the bottom 9 per cent of case studies captured under this nine topics in the right-hand axis: research from impact topic), followed by modern languages biological sciences and medicine is linked to and linguistics (6 case studies, 4 per cent), all but three of the impact topics relating to the art and design (6 case studies, 4 per cent), creative and cultural sector; physical sciences and and education (4 case studies, 3 per cent). engineering is linked with all but one; and social sciences intersects with all nine impact topics. This pattern was also repeated in several other This suggests that while there are strong disciplines. While just under half of the case disciplinary links between research in the arts studies from English language and literature were and humanities and related areas of the sector,

12 King’s College London 2. Research impact in the creative and cultural sector

considerable interdisciplinary activity is also 2.2.1 Influencing industry practice taking place between the sector and all fields Influencing industry practice was the most of academic research. common type of impact to appear in our sample, The disciplinary nuances displayed in Figure 2 featuring in 249 case studies (60 per cent of the add a new interpretation to the wider analysis of case studies analysed). Two-thirds of these spoke research impact. The previous analysis of research to influencing a creative approach or enhancing impact across all case studies submitted to REF the professional development of a practitioner. 2014, from which our topic analysis was derived, Yet other forms of impact on the sector included suggested strong interdisciplinarity, with many informing programming, commissioning or disciplines leading to a variety of diverse impact acquisitions, inspiring an artwork or project, topics.9 However when, in this study, we focused bringing an innovation to the sector, or creating in on the creative and cultural sector, we found opportunities for growth – be it establishing that this interdisciplinary activity coexisted with networks or spaces for performance, or boosting strong disciplinary links between the academic the profile of an individual. field and the area of the sector. It would be These claims were common among researchers instructive to see the extent to which this is active as practitioners in their own right, mirrored in other sectors to better understand underpinning just over two-thirds of all practice- the interdisciplinary origins of impact. led case studies, which made up one-fifth of the sample overall. A typical example of a practice-led case study can be found in Case Study 1, which 2.2 What form does research features the work of Lois Weaver, a performance impact take? artist, writer, director and activist from Queen Mary University of London. As a practitioner The types of impact described by researchers and mentor, Weaver has developed new spaces took various forms. Whereas some reported for the production and presentation of live influencing practicewithin the creative and performance, and supported the expansion cultural sector, others discussed the ways in which of live art programming among curators.11 9 King’s College London, their work had come to inform policy, education Physical and computer sciences also had and Digital Science. or public understanding through the sector; some a strong presence in the provision of services to The Nature, Scale and Beneficiaries of Research researchers described transforming the outlook the sector. For example, in 2004 researchers from Impact (2015): 38–39. of an individual, whereas others appealed broadly the Centre for Numerical Modelling and Process to economic growth or impacts in healthcare, Analysis at the University of Greenwich helped 10 This sample included 48 case studies from each conservation or politics. to facilitate the restoration of the Cutty Sark, of the nine impact topics We captured all of the impact claims made producing a digital model that predicted how the relevant to the sector. in our qualitative sample of 417 case studies,10 intricate elements of the ship’s complex structure 16 case studies were selected from each topic subsequently grouping them into eight main types would respond to variations in temperature, where it was identified (see Figure 4). Our criteria for coding these impact pressure, vibration and humidity over time as the most relevant types was deliberately broad and driven by the (see Case Study 2).12 theme in the case study’s narrative, 16 where it was content of the case studies. The impact types the second most relevant, captured were also not mutually exclusive: a single 2.2.2 Transforming public understanding and 16 where it was the case study tended to include a range of impact Transforming public understanding was the third. The sources were otherwise selected at claims, with an average of three claims cited per second most common type of impact, occurring random. During the case study. in 219 case studies (53 per cent of the sample analysis we also identified and excluded 15 false positives, leaving a final sample of 417 case Figure 4: Impact types studies. Influencing industry practice 11 ‘Transforming Publics and Participation through Transforming public understanding Performance’, REF 2014 Case Study. Available Audience development at impact.ref.ac.uk/ Informing pedagogy and curricula CaseStudies/CaseStudy. aspx?Id=19055 Social change

12 ‘ Applying Computational Policy development Reliability Engineering to the Conservation Economic growth of Maritime Heritage Improving health and wellbeing Structures’, REF 2014 Case Study. Available at impact. 050 100 150 200 250 ref.ac.uk/CaseStudies/ No. case studies CaseStudy.aspx?Id=3920

The creative role of research 13 2. Research impact in the creative and cultural sector

analysed). These forms of public education did accounting for a fifth of all case studies in our not just include knowledge exchange on issues sample that described an impact in this area. related to arts and culture, which occurred in just In some instances, the role of the creative and 80 case studies that described a change to public cultural sector was to translate complex ideas into understanding, but included the dissemination a more digestible format. For example, Rob Kesseler, of information and generation of discussion about an artist from the University of the Arts London, 13 additional themes. As shown in Figure 5, collaborated with scientists from the Royal these ranged from the transfer of knowledge Botanic Gardens in Kew to present the ‘intricate about history, science or local heritage to tackling structure, forms and functions of botanical issues of identity, gender and sexuality, human subjects’ through microscopic representations rights and social equality. of plant material of fruits, seeds and pollen. The activities that enabled such impacts on The aim of this work was to increase public public education ranged from public talks and understanding and support for the conservation conference papers to the dissemination of findings work undertaken at the gardens13, making through broadcasting and partaking in creative accessible knowledge previously available only projects. Practice-led research had an interesting to a small scientific community. Through being presence in transforming public understanding, exhibited at Kew, the work reached an estimated

Case Study 1: Transforming publics and participation through performance

The work of Lois Weaver (Queen Mary University up new spaces for the production and presentation of London) combines practice as an artist, curator of live performance, and actively supported and activist. Her work has emphasised greater other curators in the expansion of live art social engagement in live art practice, encouraging programming. As part of East End Collaborations, diverse and often marginalised communities to an organisation that Weaver established from 1999 make meaningful contributions to discussions of to 2009, she has experimented with new formats urgent social issues. Central to this body of work for such platforms through leading a professional is the question ‘how can popular theatrical forms, development seminar and instigating critical sometimes deeply conventional, be redeployed to conversations with practitioners. discover new gender possibilities, to challenge the Weaver’s impact as a performance artist traditions of theatrical practice, and, above all, is reflected in her influence on 12 artists, who to engage audiences in active conversation about have emulated or built on her work, or sought her their own lives?’ guidance as a mentor or director. Her approach Weaver has influenced the practice of emerging has also influenced curatorial strategies in the and established artists as well as curation and expansion of live art programming in select programming practices pertaining to live art. institutions, and in the development of new Through workshops and mentoring, she has artist-led spaces and initiatives, and artist supported a range of artists. She has also opened development programmes.

Case Study 2: Applying computational reliability engineering to the conservation of maritime heritage structures

In 2004, the Cutty Sark Trust approached the was dismantled there was a guarantee that the Centre for Numerical Modelling and Process safest way to dismantle each component of its Analysis (CNMPA) at the University of Greenwich structure had been taken. The modelling was also to help restore and conserve its famous maritime integral to designing a new support structure in heritage site. the reassembly, which saw the ship raised above Drawing on the CNMPA’s background in ground level and opened as a museum space. using computational reliability engineering to The research minimised the risk of conserving assess the performance of complex structures, a major national heritage site, saving an estimated researchers at the centre produced a digital finite £500,000. The model has since been adapted for element model of the Cutty Sark that factored use as a maintenance tool to help monitor the in its age, the composite composition of the hull, corrosion rate of the Cutty Sark, and has been 13 ‘Professor Rob Kesseler’s which is made up of iron and timber elements, utilised in the restoration of the Medway Queen, Collaboration with the Royal Botanic Gardens, and the material loss caused by corrosion. The ensuring that the rivets and hull plates meet Kew’, REF 2014 Case Study. model predicted structural behaviour during contemporary safety requirements to enable Available at impact.ref. the conservation process, so that when the ship the ship to set sail once again. ac.uk/CaseStudies/ CaseStudy.aspx?Id=40349

14 King’s College London 2. Research impact in the creative and cultural sector

Figure 5: Themes in Transforming Public Understanding

Arts and culture History Human rights, social equality Health, disability, wellbeing Identity, gender and sexuality Local heritage Religion Environment, climate change Politics Science Economics Family Language Law 0 10 20 30 40 50 60 70 80 No. case studies

audience of 105,000, and found a wider audience piece of performance art designed to encourage still online: featured as part of a Kew-produced reflection on the ethical and social challenges documentary, Kesseler’s images have been of climate change. The piece situated a small viewed by c. 9,500 people on YouTube, and audience in a shipping container, faced with a map an estimated audience of 1.3 million through of the local area and the scenario of an impending their reproduction in a TED talk delivered climate crisis. Audience feedback suggested that by Jonathan Drori. Collections of these prints through being able to shape the narrative by voting were also published in book form as Pollen, Seeds on key decision points, the experience enriched and Fruit, with a distribution of c. 25-30,000 their confidence in realising and addressing the per title, and prominent media coverage in the implications of climate change in their everyday Guardian, Telegraph and Time Magazine. lives.14 Such transformations of opinion were Others used creative mediums to encourage also reported on an international scale in case reflection on pressing societal issues.3rd Ring studies such as The Act of Killing, a documentary Out, a postdoctoral project undertaken by that exposed the 1965–66 genocide in Indonesia, Zoe Svendsen from the English Faculty at the leading to global recognition of the human rights University of Cambridge, was an immersive violations enacted (see Case Study 3).15

14 ‘Rehearsal as Effective Theatrical Method for Performance: Responding to Climate Change’, REF 2014 Case Study. Available at impact.ref.ac.uk/ CaseStudies/CaseStudy. aspx?Id=14678

15 ‘The Act of Killing’, REF 2014 Case Study. Available at impact.ref.ac.uk/ CaseStudies/CaseStudy. aspx?Id=20542 ‘3rd Ring Out’ © Simon Daw

The creative role of research 15 2. Research impact in the creative and cultural sector

Case Study 3: The Act of Killing

Michael Uwemedimo’s research on the role of as points of reflection and departure for further re-enactment in documentary practice fed into stylised re-enactments. his work as a producer on the critically acclaimed The release of the film prompted a historical documentary The Act of Killing (2012). re-examination of the 1965–66 killings at a The film exposed the Indonesian genocide scale previously inconceivable. The film gained of 1965–66 through a process of historical recognition for the gross violation of human re-enactment and genre re-stagings based rights in Indonesia, and positioned the genocide on primary interviews with those who had at the centre of global discussions about perpetrated or survived the crimes. The film historical human rights, initiating discussion about ‘intervenes in the country’s history of genocide by similarly hidden histories of violence, corruption reframing these historical events through the lens and genocide in countries such as Spain, the of genre and memory’, as the primary interview Philippines, Turkey and Denmark. Within Indonesia, material increasingly becomes fictionalised. 500 clandestine screenings and the distribution of This material not only exposed the human rights over 1000 free copies of the DVD across 29 of the violations, but powerfully captured on film the country’s 33 provinces meant that citizens within changed self-awareness of those who perpetrated the country were also engaged, circumventing the the crimes, whose reactions to screenings of censorship that would otherwise have prevented the fictionalised re-enactments were included the film from screening within the country.

Case Study 4: Stonehenge and its landscape; changing perceptions, informing the next generation and benefiting the local economy

‘Stonehenge Riverside Project’ © Mike Parker Pearson

The Stonehenge Riverside Project ran from 2003 Local volunteers and around 1,000 students to 2010, led by Mike Parker Pearson (University from universities across the UK and Europe were of Sheffield, now at UCL) with co-directors from trained in archaeological techniques at the many Bournemouth, Bristol, Manchester, Southampton sites excavated during the course of the project. and UCL, investigating the original purpose Many of these students are now employed as of the monument and its surrounding landscape. professional archaeologists. Local volunteers The connection of Stonehenge via the River also developed skills in archaeological work. Avon to nearby timber monuments at Durrington After gaining an intimate knowledge of their Walls led to the discovery of a settlement local heritage, many went on to staff a volunteer there. These were not primitive astronomical outreach team responsible for delivering tours observatories but places where people of the site to some 6,600 visitors. The project gathered to celebrate at different points generated a variety of further impacts, including of the year. The importance of the Avon transforming public perception of the famous linking the settlement at Durrington Walls monument, increasing tourism to the site and with Stonehenge itself was reinforced by the surrounding area, and generating new information discovery of Bluestonehenge, a riverside site that has informed the content of guidebooks and at the end of the Stonehenge Avenue. a complete redesign of the visitor centre.

16 King’s College London 2. Research impact in the creative and cultural sector

2.2.3 Audience development • influencing the design of curricula and Building and diversifying audiences was a further examination (52 case studies, 12 per cent way in which research made an impact within of case studies analysed) the sector, featuring in 186 case studies (45 per • improving learning and development cent of sample). Over two-thirds of these case (51 case studies, 12 per cent) studies reported on increasing engagement with • informing pedagogical approaches within the the general public. This was primarily framed as education sector (35 case studies, 9 per cent). broadly growing audiences for the creative and cultural sector or, more specifically, encouraging For example, the conceptual framework for participation in local heritage and regional art teaching creative writing developed at Teesside forms; however, a handful of anomalies also University fed into a national review of A-level existed which included increasing engagement specifications, the recommendations of which with science (nine case studies) and politics (two were subsequently taken up by the examining case studies) through creative or cultural projects. boards AQA and Edexcel. This resulted in Audience development did not just mean the introduction of creative writing to HE and getting more diverse audiences through the door, A-level curricula.20 16 ‘Stonehenge and its but included creative solutions to the problem Not all examples of shaping education, however, Landscape; Changing of how to engage people in a sustainable way. related to national frameworks. Others featured Perceptions, Informing the Next Generation and This was common in the heritage sector, with education programmes tailored to the arts, or Benefitting the Local initiatives such as the Stonehenge Riverside Project models designed to help facilitate learning among Economy’, REF 2014 Case encouraging volunteers from the local community a specific stakeholder group. For example,Sounds Study. Available at impact. ref.ac.uk/CaseStudies/ to become involved in their local history through of Intent, a project led by Adam Ockelford at CaseStudy.aspx?Id=12235 participation in excavations (see Case Study 4).16 the University of Roehampton, redefined music Similar engagement initiatives also surfaced provision for students with learning difficulties 17 ‘A Scottish Self-Portrait: The Northern Lights within documentary film. Nick Higgins through a developmentally based curriculum Documentary Project’, (University of Edinburgh), a practice-led framework and supporting resources specially REF 2014 Case Study. researcher with a background in using designed for these students.21 As summarised Available at impact.ref. ac.uk/CaseStudies/ documentary film to reflect the views in Case Study 5, the project targeted a subset CaseStudy.aspx?Id=24011 of marginalised social groups, turned to of schools and third sector organisations in the UK

18 ‘The Old Bailey Online: crowdsourcing audio-visual material to produce that work specifically with children with learning Democratising access to an inclusive documentary portrait of the Scottish difficulties. Its effect on pedagogy is already social history’, REF 2014 nation in Northern Lights (2012). By offering free apparent, with adoption ranging from the uptake Case Study. Available at impact.ref.ac.uk/ workshops and resources on how to self-shoot, of the supporting resources to the complete revision CaseStudies/CaseStudy. the project not only widened participation of music curricula within certain organisations. aspx?Id=24011 in documentary making among unlikely areas

19 Ca se studies describing of the community, it enriched the self-perception 2.2.5 Social change impact on HE were only of groups that are typically overlooked Enacting social change featured in 88 case included if the impact was or marginalised, and altered the perception studies (21 per cent of case studies analysed). not on the host institution, 17 as research impacts of the role of those groups in society. This included impacts on individuals, such within an academic’s Other examples of audience development as forms of empowerment and self-reflection own university were not included increasing public access to artworks and (33 case studies, 29 per cent), family cohesion counted in REF 2014. archival documents, with digital scholarship being (6 case studies, 5 per cent) and lifelong learning 20 ‘Developing the Teaching a key area of focus. A prominent example of this is (3 case studies, 3 per cent), as well as impacts of Creative Writing’, REF the Old Bailey Online, a digital humanities resource on wider societal issues, such as breaking down 2014 Case Study. Available at impact.ref.ac.uk/ established by Tim Hitchcock (University of stereotypes and representing marginalised CaseStudies/CaseStudy. Hertfordshire, now University of Sussex) and communities (24 case studies, 21 per cent), building aspx?Id=30191 Robert Shoemaker (University of Sheffield) that cross-cultural empathy (37 case studies, 33 per cent) 21 ‘Sounds of Intent’, REF offers online access to 197,745 historic trial reports. and social regeneration (5 case studies, 4 per cent). 2014 Case Study. Available The site has seen creative use in teaching, is a key Creative writing projects played an important at impact.ref.ac.uk/ resource for members of the public undertaking role in mobilising social change. Red Dust Road, CaseStudies/CaseStudy. aspx?Id=20469 genealogical research on family trees, and has the memoir of Jackie Kay (Newcastle University), enriched the content of media productions, notably and associated engagement activities have raised 22 ‘Red Dust Road: New being the subject of the BBC Radio 4 series Voices awareness of the complex identity issues around Forms of Memoir and the 18 Enrichment and Extension from the Old Bailey. race, adoption and belonging, empowering others of Public Discourse on to feel understood as adoptees or racial minorities Family, Identity, Belonging 2.2.4 Informing pedagogy and curricula themselves (see Case Study 6).22 and Adoption’, REF 2014 Case Study. Available Formal approaches to informing education, from Forms of social change were also prominent at impact.ref.ac.uk/ primary education to HE level,19 were described in participatory arts activities. Adam Benjamin, CaseStudies/CaseStudy. in just over a quarter of the case studies analysed. a practitioner–scholar from Plymouth University, aspx?Id=21725 This included: introduced integrated dance practice, in which

The creative role of research 17 2. Research impact in the creative and cultural sector

able-bodied and disabled dancers perform participated as dancers. As one participant wrote, together, to the Adugna Community Dance ‘when I move in the space I feel free; I feel very Company in Addis Ababa, Ethiopia. After proud ... in the street, when people are watching mentoring and working closely on choreography me I [think it is] because they probably saw me on with the company’s artistic directors, the resulting TV dancing. So, I don’t think they are watching performances challenged perceptions of disability me because I’m disabled.’23 in a culture that typically ‘shuns disabled people’. Impacts on social change seemed to be a In the words of one audience member: ‘three natural by-product of work in the creative and thousand people ... will leave the auditorium with cultural sector, frequently supplementing other a completely different picture about the role of impact types described above. Case studies such the disabled ... Those messages will be passed as the Northern Lights Documentary Project, cited on’. Such transformations also touched those who above as an example of audience development,

Case Study 5: Sounds of Intent

Sounds of Intent (2001–11) illuminated the lack third of special schools in the UK25, which use of understanding of the musical development of the Sounds of Intent material to inform their music students with learning difficulties, particularly those curricula and assessment. There has also been with severe or profound intellectual impairment, uptake in organisations offering music provision and in turn the inconsistency of the music education for children with learning difficulties such as Live that was provided for them. Led by Adam Ockelford Music Now, Jessie’s Fund and Drake Music, all of at the University of Roehampton, the research which use the programme to evaluate the impact addressed the need for a developmentally of their activities, and Ofsted refers to Sounds based music curriculum framework to map of Intent as an example of good practice in music- student development, attainment and progress educational assessment. in a systematic way. Following the launch of the project’s website in 2012, which contains resources ‘Sounds of Intent’ for teaching and learning, assessment and tracking © Adam Ockelford progress, the site has seen profound engagement, with over 2.5 million unique visitors and over 370,000 resources downloaded.24 Sounds of Intent was initially targeted at music coordinators in the 500 or so special schools in England for children with learning difficulties, and sought to raise awareness through organising conferences and seminars for schools, music hubs and education providers. Already the project has influenced practice in around a

Case Study 6: Red Dust Road

Jackie Kay’s 2010 memoir Red Dust Road the book addresses, with one adult adoptee writing

presents an account of her life growing up that the memoir made her ‘feel more understood 23 ‘Adugna Community black in Glasgow as the adopted daughter and seen than almost any other’; another reported Dance Company: of white parents, and follows her search for that the book had inspired the search for their own Empowering Disabled Dancers and Changing her birth parents. In this and her other works birth parents. The issues of race and belonging Attitudes toward of poetry and fiction, Kay investigates the also resonated with other audiences. This is Disabled People through complexity of identity, introducing storytelling reflected in the success of the author’s outreach Contemporary Dance in Ethiopia’, REF 2014 Case within the genre of memoir to interweave pluralist among hard-to-reach stakeholder groups. After Study. Available at impact. perspectives from her own life alongside those delivering readings in prisons, Kay has received ref.ac.uk/CaseStudies/ of others around her. Her books have animated responses from inmates showing strong self- CaseStudy.aspx?Id=4681 public discourse and policy debate around the identification with the themes of the memoir, with 24 As at June 2017 this had issues of adoption and identity, receiving broad comments such as ‘She understands, that Jackie, increased to 7,000,000 coverage in the media as well as with major she really knows’. Kay subsequently wrote a set unique visitors and over 1,000,000 downloads of adoption support networks. of poems for the Scottish Refugee Council’s Stop material. Testimonials from readers of Red Dust Road Destitution campaign, inspired by her work with describe identifying with the broader issues that asylum seekers in Scotland. 25 Two-thirds of special schools as at June 2017.

18 King’s College London 2. Research impact in the creative and cultural sector

also reported giving voice to marginalised • 11 per cent described an indirect influence communities. By using everyday technologies by appealing to citations in policy documents to crowdsource footage, the project enabled (10 case studies) anyone to submit a video of ‘their Scotland’.26 • 13 per cent presented endorsements from This empowered minority groups to take agency policymakers, without any link to a specific in the ways in which they are represented in policy (12 case studies) society, to ‘alter the self-perceptions of minority • 16 per cent reported on occasions where groups in Scotland, and also the mainstream’s policymakers had been exposed to the research, perception of these groups’. be it in a formal presentation or discussion (15 case studies) 2.2.6 Policy development • Four per cent described occasions where their Impact on policy development appeared in research had helped an organisation to deliver 81 case studies (19 per cent of case studies on an existing policy goal (4 case studies). analysed). This ranged from actively devising policy or determining the allocation of funding Impact on policy was most pronounced in non- to addressing established policy goals, be it within governmental organisations, which accounted for a small organisation or a government department, two-thirds of the case studies that spoke to the committee or working group. adoption of research outcomes (12 case studies). Of the case studies that described an impact At government or international policy level, on policy: the majority of case studies simply described • 43 per cent (40 case studies) discussed work exposure to the research (12 case studies) without with government officials and international elaborating on how this evolved into a change policymakers, including representatives from in policy or legislation. the European Union and the United Nations • 27 per cent (25 case studies) impacted on 2.2.7 Economic growth national organisations such as Arts Council Examples of creating economic growth England, English Heritage and Creative surfaced at a comparable rate to examples Scotland, or organisations operating outside of policy development, appearing in 63 case of the creative and cultural sector, such studies (15 per cent of case studies analysed). as the NHS and Ofsted These ranged from macro statements • 21 per cent (20 case studies) described about boosting the economy, be it through a change of policy within an individual increasing tourism, employment or GDP, organisation or company to direct commercial benefits for an • 9 per cent (8 case studies) described organisation, such as boosting income, policy development within local and efficiency, supporting business development regional councils. or assisting in product development. Just over half of the economic impacts reported Over two-thirds of these impacts related to related to income generation for an individual cultural policy. Within this, heritage was the most company or organisation (54 case studies). dominant area of the sector. For example, shortly 81 per cent of these case studies spoke directly after opening in 2006, the University of Glasgow’s to the generation of income, be it through Centre for Battlefield Archaeology was invited ticket sales or the revenue generated by the by Historic Scotland to produce an inventory of sale of a creative output (37 case studies), Scottish battlefield sites. This inventory was used or through assisting in securing grant funding 26 ‘A Scottish Self-Portrait: The Northern Lights by the Scottish Government in 2009 as the basis for a sector partner (4 case studies). Spin Documentary Project’, for the Scottish Historic Environmental Policy outs, contrary to their prevalence within science REF 2014 Case Study. (SHEP) to protect the nation’s historic battlefields. and medicine,28 had a minor presence within Available at impact.ref. ac.uk/CaseStudies/ 39 of these battlefields are now registered as sites the sample, featuring in just four per cent of CaseStudy.aspx?Id=24011 of national importance.27 Other areas of policy examples of income generation for an individual impact beyond the sector were in public policy company (2 case studies). 27 ‘ Commemoration and Protection of Battlefields (11 case studies, 13 per cent of impacts on policy), More indirect examples of commercial in Policy and Practice’, education (7 case studies, 8 per cent), health impact for a company or organisation included REF 2014 Case Study. (6 case studies, 7 per cent) and science (3 case increasing visitor numbers (6 case studies) Available at impact.ref. ac.uk/CaseStudies/ studies, 4 per cent). and improving efficiency (4 case studies). For CaseStudy.aspx?Id=23304 Only a third of the case studies that described example, in producing an edition of Heinrich an impact on policy (31 case studies) reported Böll’s complete works with the Deutscher 28 King’s College London and Digital Science. on the adoption of a recommendation derived Taschenbuch Verlag, Frank Finley (University The Nature, Scale and from their research. Otherwise: of Leeds) not only helped to generate publicity, Beneficiaries of Research • 23 per cent described influencing policy through marketing and sales opportunities for the publisher, Impact (2015): pp. 52–54. acting as a consultant or advisor (21 case studies) but as the first complete digital edition of Böll’s

The creative role of research 19 2. Research impact in the creative and cultural sector

‘Adugna Community Dance Company’ © Rosa Verhoeve ‘Adugna Community Dance Company’ © Rosa Verhoeve

works, the edition increased efficiency in the work with small groups to the national rollout printing and licensing processes for subsequent of recommendations derived from the research. editions of his texts. For example, the Military Writing Network Beyond the individual company level, founded in 2009 by Siobhan Campbell we identified just one case study that (formerly of Kingston University, now at described economic growth on a national The Open University) explored how creative level. Broader economic impacts were instead writing practice can help veteran soldiers, focused on local communities. Six case studies sailors and airmen to cope with combat stress. claimed impact through increasing spend In addition to providing opportunities for in local businesses, and four through creating veterans to publish their poetry and stories, jobs; however, the most common examples the network has developed pedagogy and of economic growth related to tourism workshop practices that encourage engagement (21 case studies), just under half of the references with creative writing to help alleviate common to which stemmed from work in the heritage signs of combat stress, such as avoiding unfamiliar sector. For example, based on a body of prior situations and rebuilding the loss of self-confidence research into the English artist JMW Turner, through fostering creativity.30 David Hill (University of Leeds) was invited Since REF 2014, the understandings of to be a development consultant to the Discover writing practice as enabling cultural resilience Turner’s Yorkshire project, which launched that emerged in Campbell’s work with Combat in 2010. Integrating his research into the Stress UK led to an expansion of the field of topographical aspects of Turner’s work with investigation: experiments with GPS technology, Hill’s contribution identified viewing points and ‘Having joined the Open University, I developed locations depicted in Turner’s drawings of the expressive writing pedagogy in life-writing for Yorkshire landscape, totalling 70 sites. Since work with victims of sexual violence via the human the launch of the trail, Yorkshire’s tourism rights team of INMAA, Kirkuk in a project jointly economy has seen a 6 per cent year-on-year convened by the Foreign and Commonwealth increase in visitor numbers, with 1.25 million Office (FCO) and Beyond Borders. Consolidation 29 ‘Discover Turner’s visitors estimated to have seen the interpretation of the thinking led to The Expressive Life-Writing Yorkshire: Public- Oriented Research and boards during the year after the trail was Handbook, now used by the FCO’s Countering Commercialisation’, REF launched. This also resulted in an increase Violent Extremism (CVE) programme and 2014 Case Study. Available in spend in the local area, with visitors in a ‘training the trainers’ scheme where the at impact.ref.ac.uk/ CaseStudies/CaseStudy. to locations on the Turner Trails spending Fellows of the Gulf Reconstruction initiative were aspx?Id=6393 an average £119 per head during 2010–11, trained in the uses of writing workshops in enabling compared to £106 the previous year.29 cultural story-telling and supporting resilience. 30 ‘The Military Writing Network: Creative From Jan-August 2017, I worked with VOS Writing, Life Writing and 2.2.8 Improving health and wellbeing (Veteran’s Outreach Support), with a view to Trauma’, REF 2014 Case Only seven per cent of the sample reported impacts exploring how creative writing pedagogies can Study. Available at impact. ref.ac.uk/CaseStudies/ on health and wellbeing through the creative be effective both in supporting mental wellness CaseStudy.aspx?Id=40629 and cultural sector (29 case studies), just over for peer mentors and also in providing these two-thirds of which specifically related to mentors with new ways of interacting with 31 Siobhan Campbell in email correspondence. mental health. These initiatives ranged from their mentees.’31

20 King’s College London 2. Research impact in the creative and cultural sector

Another case study described the implementation The connections between the creative and of findings from research at Anglia Ruskin cultural sector and physical health instead tended University into the provision of music therapy to be confined to raising public awareness around for dementia patients, following a programme health risks and medication. For example, in of joint research with Methodist Homes for the 2008 biomedical research from the University of Aged (MHA), a nationwide care provider for older Sheffield on the transport and molecular basis of people. Already the methods are transforming human sperm was developed into a film,The Great care, as one MHA manager writes: ‘We have seen Sperm Race. Through a wide distribution network some astonishing changes in our care homes when and considerable coverage in the press, the film people with dementia receive music therapy – they built debate and public understanding around become so much more responsive and joyful, and issues relating to reproductive medicine.33 In such it seems to have a wonderful calming effect on examples, the impact was in raising awareness of people who experience anxiety, as a lot of people issues related to health and wellbeing, rather than with dementia do’.32 a direct impact on the provision of health services.

2.3 Research impact in the creative and cultural sector: key findings

Researcher engagement with the creative and on influencing industry practice and cultural sector produced an assortment of varied transforming public understanding, which impacts that were complex and difficult to distil differs significantly from research impacts into a neat taxonomy. We found that impact types in other sectors, particularly medicine and may overlap, act independently of one another science, where impacts on health and policy or be interdependent; and that impact can be development rank more highly. fortuitous, surfacing after a body of research has • The creative and cultural sector played an already been published, just as it can be intimately important role in translating research into worked into the conception of a project. impacts on opinions and behaviour, and Yet despite this heterogeneity, we can, brokered wider audiences for academic however, draw the following conclusions: research through translating findings into • Research impact in the creative and cultural more widely accessible formats. The prevalence sector did not solely stem from research in the and quality of this translational role suggests arts and humanities, but drew from almost all that research collaborations are not only fields of academic research. helping to drive innovation within the creative • There were eight main areas in which and cultural sector, but are also being utilised 32 ‘Benefitting the Residents researcher engagement involving the creative to address issues of social and political of Methodist Homes for and cultural sector had an impact: significance by combining research with the Aged through Music –– influencing industry practice creative practice. Therapy’, REF 2014 Case Study. Available at impact. –– transforming public understanding ref.ac.uk/CaseStudies/ –– audience development Key areas identified for further investigation CaseStudy.aspx?Id=43781 –– informing pedagogy and curricula include: 33 ‘The Great Sperm Race –– social change • How the strong, yet underutilised links – Encouraging Public –– policy development between practice-led research and mental Understanding of Human –– economic growth health services are helping to improve Reproduction’, REF 2014 Case Study. Available –– improving health and wellbeing. provision for mental health illness. at impact.ref.ac.uk/ • The highest-ranking impacts of research • How activities in the creative and cultural CaseStudies/CaseStudy. in the creative and cultural sector were sector help build a sense of community. aspx?Id=12225

The creative role of research 21 3. Pathways to impact

–3– Pathways to impact

‘Madness & Modernity’ © Wellcome Collection

22 King’s College London 3. Pathways to impact

Generating impact from research does not follow a single pathway from the university to the creative and cultural sector. Multiple fields of research inform impact across all sectors, frequently leading to activity in areas that are weakly associated with the originating academic discipline. In this section, we analyse pathways to impact, reveal their diversity and reiterate the importance of maintaining an open approach towards identifying and describing them.

Previous research has shown that 3,709 unique In total, we identified 68 impact mechanisms Any one case study pathways were reported across the 6,679 (see Table 2). These were rarely described in could include several uncensored case studies submitted to REF isolation: two per cent of case studies reported impact mechanisms, 2014, meaning that ‘any attempt to define a on just one type of mechanism; up to 17 different forming a profile standard route to research impact could be mechanism types were referenced within a single of interconnected counterproductive’.34 These ‘pathways’ are made case study; and an average of six types were activities. up of the distinct activities, efforts, initiatives and mentioned per case study across the sample. mechanisms that come together to create ‘an effect Moreover, in just under a quarter of case studies on, change or benefit to the economy, society, analysed, engagement with the creative and culture, public policy or services, health, the cultural sector appeared alongside activities in environment or quality of life, beyond academia’.35 other sectors, including health, computing, law, This was especially true of research impact in policy and education. the creative and cultural sector, where we found The most frequently cited impact mechanism a total of 2,602 pathways in a sample of just 417 was public talks (present in 43 per cent of case studies. In a single case study, it was common case studies). These tended to be for a general to find impact pathways crossing multiple sector audience, including pre-screening talks, lecture boundaries: collaborations with curators might recitals and question and answer events. lead to outreach in schools; workshops with Exhibitions and installations were the next most practitioners may prompt invitations to act common mechanism (38 per cent), followed by as a consultant on related projects, or to brief radio broadcasts (30 per cent), workshops and cultural policymakers. There was no set point masterclasses (26 per cent), and TV programmes at which these pathways began, and no one form (24 per cent). As shown in Figure 6, when grouped of collaboration that enabled them. While some into broader sector areas, hot spots for research were the product of collaborative partnerships, activity in the sector accordingly fell within others reported on research being picked up by the ‘Music, performing and visual arts’ (71 per cent of sector, sometimes years after a large body of work sample), ‘Film, TV, video, radio and photography’ had already been published. What the following (58 per cent of sample), and ‘Museums, galleries analysis therefore shows is that there is no and libraries’ (56 per cent of sample). definitive pathway to impact within the creative These mechanisms featured in impact narratives 34 King’s College London and Digital Science. and cultural sector. Impact accumulates through in a variety of ways. Any one case study could The Nature, Scale and a variety of interconnected strands of activity; it include several impact mechanisms, forming Beneficiaries of Research can emerge at any point in the research process, a profile of interconnected activities, typically Impact: p. 71. and may be the product of both direct and indirect centred around one or two more dominant 35 HEFCE, Assessment forms of engagement. mechanisms. Whereas one mechanism may be Framework and Guidance Based on a qualitative analysis of the 417 case the central project, it was common for other on Submissions (2011): p. 48. studies in our sample, in this section we survey supplementary activities to be presented alongside the range of impact activities that researchers it, or to feature as follow-on projects. 36 ‘Madness and Modernity: reported on (‘impact mechanisms’); the types of For example, an exhibition mounted at the Mental Illness, the Visual Arts and Architecture institutions they engaged with and the nature of Wellcome Collection entitled Madness and in Fin de Siècle Vienna these collaborations (‘sector partners’); and we trace Modernity: Mental Illness and the Visual Arts – Popular Exhibitions in the point in the research lifecycle at which impact in Vienna 1900 drew on the work of an AHRC- London and Vienna’, REF 2014 Case Study. Available occurred (‘impact timescales’). funded project connecting psychiatry, visual at impact.ref.ac.uk/ arts and architecture.36 The underpinning CaseStudies/CaseStudy. research stemmed from art and design, yet the aspx?Id=18058 and ‘Madness and Modernity: 3.1 What were the mechanisms through impact mostly occurred in the museum sector, Mental Illness, Psychiatry which research impact was achieved? engaging three further impact mechanisms in and the Visual Arts in addition to the exhibition (see Figure 7). The Vienna 1900’, REF 2014 Case Study. Available The mechanisms or activities that enabled impact in exhibition’s success in encouraging reflection on at impact.ref.ac.uk/ the creative and cultural sector ranged from creative how, historically, society has dealt with mental CaseStudies/CaseStudy. projects in theatre, music or the visual arts to work illness then led to invitations to produce further aspx?Id=4678 in cultural policy and arts education. exhibitions, documentaries and a report that

The creative role of research 23 3. Pathways to impact

Table 2: Impact mechanisms ordered by frequency cited Impact mechanism Case studies % Sample Impact mechanism Case studies % Sample Public talk 200 42.5 Oral history project 22 4.7 Exhibition, installation 178 37.8 Fiction 22 4.7 Radio broadcast 139 29.5 Data set, database 20 4.3 Workshop, masterclass 123 26.1 Tour 20 4.3 Tv programme 115 24.4 Internship, mentoring 19 4.0 Conference, lecture series 109 23.1 Policy address 18 3.8 Website 108 22.9 Sound recording 18 3.8 Press coverage 101 21.4 Festival 17 3.6 Report, guidelines 84 17.8 Guidebook 15 3.2 Academic book 81 17.2 Programme or liner notes 13 2.8 Newspaper 74 15.7 Poetry 11 2.3 Lecture, conference paper 67 14.2 Residency 10 2.1 Video 65 13.8 Map 7 1.5 Teaching, learning resource 64 13.6 Game 6 1.3 Schools outreach 63 13.4 Press release, press conference 6 1.3 Film 63 13.4 Clinical or health services work 5 1.1 Catalogue, exhibition text 59 12.5 Programming 5 1.1 Industry magazine 54 11.5 Product design 5 1.1 Academic journal 50 10.6 Photography 5 1.1 Software 48 10.2 Expert witness 4 0.9 Curriculum design 48 10.2 Criticism 4 0.9 Heritage site 48 10.2 Dance (performance) 4 0.9 Physical or digital archive 45 9.6 Conservation tool 3 0.6 Mass-market factual 40 8.5 Choreography 3 0.6 Blog 37 7.9 Commission, inquiry 2 0.4 Podcast 35 7.4 Lobbying 2 0.4 Advisory board, steering group 33 7.0 Architecture 2 0.4 Theatre staging, performance art 32 6.8 Graphic design 2 0.4 Music composition, score 31 6.6 Pharmaceutical product development 1 0.2 Social media 30 6.4 Screening tool development 1 0.2 Concert, sound installation 29 6.1 Research as evidence at trial 1 0.2 Artwork (visual arts) 27 5.7 Crafts 1 0.2 Play, theatrical work 23 4.9 Fashion design 1 0.2 Reading 23 4.9 Mailing list 1 0.2

Figure 6: Distribution of mechanisms by sector group

Music, Performing and Visual Arts Film, TV, Video, Radio, Photography Museums, Galleries and Libraries Education Publishing Advertising and Marketing Policy IT, Software and Computer Services Architecture, Design, Crafts Health Legal 050 100 150 200 250 300 No. case studies

24 King’s College London 3. Pathways to impact

What our analysis Figure 7: Impact pathways – ‘Madness and modernity’ overwhelmingly revealed was not a series of ACADEMIC DISCIPLINE IMPACT MECHANISMS singular activities but a network of Exhibition Report mechanisms that work together: even if multiple Catalogue mechanisms were Art and Design mentioned in a case study, the narrative was often Video Exhibition centred around one more dominant activity. Press coverage Documentary

informed an application to the UNESCO world products and the onset of breast cancer. Through heritage list. This project therefore did not just pursuing coverage in broadcast and print media, constitute one pathway but engaged a variety Darbre was able to raise awareness among a wider of interdependent impact mechanisms. public, which in turn supported later successes Across the 417 case studies analysed, we in informing policy recommendations:37 identified 2,602 different impact pathways, starting from the originating research, going on to ‘When I started all this research, it was definitely the impact mechanism, and through to the sector not an accepted thing for academics to engage with in which the impact took place; 1,577 of these the public media but in the intervening years times pathways were unique. This is illustrated in the have changed. I have no doubt that the media has alluvial diagram presented in Figure 8. played a massive role in generating the impact on the The combination of coloured and grey blocks public without which most of the issue of constituents on almost all points on the left and right axes of personal care products would simply have been reveal that the links between originating discipline brushed under the carpet. Furthermore, the media and the sectors in which researchers were active involvement has enabled members of the general did not follow a simple pathway. This is as true public to write to me with their own stories which has for scholars working in disciplines such as physics alerted me to some very significant avenues in need or architecture as it is for historians and linguists; of research (not least the link between aluminium- it is as true for case studies with research impacts based antiperspirant salts and breast cysts).’ 38 in health or education as it is for impacts in 37 ‘Parabens in Personal heritage, museums or the visual arts. In all of these However, our analysis did not reveal a series Care Products: Providing Marketing Tools for instances, we see pathways passing through all of singular activities but a network of mechanisms Industry, Informing seven categories listed in the key. that work together: the functionality of these European Policy and Moreover, if we break down ‘Other’ into its networks of mechanisms is best illustrated Raising Public Awareness’, REF 2014 Case Study. respective sector areas – health, computing, law, on a case study level by Case Study 7, which Available at impact.ref. policy and education – we found that any one describes a UNESCO heritage route devised ac.uk/CaseStudies/ case study may mention mechanisms from up to in Turkey. 39 The route was the focus of the CaseStudy.aspx?Id=37245 eight of the 11 broader sector categories, with an case study; however, it was presented alongside 38 Professor Philippa D. average of four per case study. This demonstrates other impact mechanisms such as delivering talks, Darbre, citation from that impact is seldom achieved through a single, producing a guidebook, giving interviews and email correspondence. focused mechanism: it is the product of various a consultancy role for an exhibition in London. 39 ‘T he Horse and East-West interdependent, yet often disparate activities that Yet in building international interest in the route History: Generating span multiple areas of the creative and cultural and encouraging engagement with Ottoman Cultural and Economic Benefits in Turkey through sector, and frequently encompass those beyond history, each activity made a vital contribution a UNESCO Cultural Route, the sector. to the success and sustainability of the route. the Evliya Çelebi Way’, REF The role that these mechanisms played in Comparable examples were found throughout 2014 Case Study. Available at impact.ref.ac.uk/ mobilising research impact varied. Broadcast the case studies analysed. In Case Study 8, CaseStudies/CaseStudy. media was one of the most prevalent areas through the campaign to engage policymakers and gain aspx?Id=930 which researchers were able to mobilise impact, press coverage alongside production of the opera 40 ‘Anya 17’, REF 2014 Case particularly from disciplines beyond the arts and Anya 17 helped to achieve the work’s goal to Study. Available at impact. humanities. For example, research led by Prof. expose the enduring problem of slavery and ref.ac.uk/CaseStudies/ Philippa D. Darbre at the University of Reading sex trafficking in the UK.40 There are also many CaseStudy.aspx?Id=20912 discovered links between parabens in personal care examples where a successful project became the

The creative role of research 25 3. Pathways to impact

‘Evliya Çelebi Way’ © Mehmet Cam

King’s College London 3. Pathways to impact 3. Pathways to impact

Figure 8: Alluvial diagram linking academic discipline, impact mechanism and sector Each individual pathway is represented by a line. The mechanism types in the centre column are grouped into sector categories, from which the colour coding of the diagram is derived; this grouping adheres to the sector framework defined by the Department for Culture, Media & Sport (DCMS) in the most recent Creative Industries Economic Estimates: Statistical Release.41

41 See Department for Culture, Media & Sport, ‘Creative Industries Economic Estimates: Statistical Release’ (2016). Available at gov.uk/government/ uploads/system/uploads/attachment_data/file/523024/Creative_ Industries_Economic_Estimates_January_2016_Updated_201605.pdf

26 King’s College London The creative role of research 3. Pathways to impact

catalyst for further related activities in the sector. This linearity has been attributed to the Ian Kershaw’s research for his 1998 biography design of the case study template used in REF of Adolf Hitler led to a series of follow-on projects 2014. In an analysis of the impact case studies that transformed discourse on the Third Reich. in the health sector, Greenhalgh and Faye have As summarised in Case Study 9, his work with noted how the case study template encouraged documentary filmmakers in the late 1990s had researchers to represent impact as something with a ripple effect that came to inform A-level and a forward trajectory, originating with a specific university syllabi, and also shaped public opinion piece of research and ending with impact.43 The through media coverage, high profile lectures limitations of this linear approach were also noted and exhibitions.42 in the independent review of the 2014 assessment The tendency to represent impact as the exercise, which recommended that ‘a richer product of an interconnected and varied network picture of the impact of research could be of mechanisms was an aspect of the reporting developed which encompasses the research process in REF 2014 that was well served by the expertise, facilities and networks of an individual, free-text case study format, capturing not only group or institution that underpin or lead to the primary mechanism but the various other the eventual impact of research’.44 Moreover, activities that supplemented it to boost its reach. the extent to which this activity was circular Yet there are limits to what the REF 2014 or generative, with non-academic collaborations can tell us about pathways to impact. While leading to new findings or new research questions, researchers generally reported on several was seldom reported. complementary mechanisms, these were This linearity highlights one of the limits of largely presented as linear and causal. using the REF as a source for understanding the

Case Study 7: The Evliya Çelebi way

Donna Landry, a researcher from the School of along the route and advised them on Ottoman English at the University of Kent, played a central history and Evliya, as well as meeting with riders role in establishing the Evliya Çelebi Way, the first at the end of their journey. These rides have all equestrian UNESCO Cultural Route in Turkey. The been with Ercihan Dilari, a founding member of the route was a cross-institutional collaboration,45 project team, and his horses. Moreover, in addition tracing the first part of the journey to Mecca made to giving interviews on Turkish radio stations and the BBC, Landry co-authored a guidebook to the 42 ‘Working towards the by Evliya Çelebi in 1671, and was co-produced with Führer: Shaping Public the local community following sponsorship by local route, gave various public talks, participated Understanding of businesses and the endorsement of the mayors in interviews for online interest groups, wrote Nazi Power’, REF 2014 for popular magazines and assisted in planning Case Study. Available of Bursa and Kütahya, important towns along the at impact.ref.ac.uk/ route. Landry’s research role was supported by a forthcoming Turkish documentary. These CaseStudies/CaseStudy. a Leverhulme Study Abroad Fellowship. activities led to a consultancy role in an exhibition aspx?Id=12161

The route had impact for the local economy by in London dedicated to Evliya Çelebi’s travel 43 Greenhalgh, Trisha, and boosting tourism and raising interest in Ottoman writing, which later opened in Turkey, Athens Nick Fahy. ‘Research history nationally and internationally. But this and other locations around the globe. Impact in the Community- Based Health Services: success was not secured through the route alone: Partly owing to the success of the ride, An Analysis of 162 Case it relied on a number of supplementary activities UNESCO made Evliya Çelebi a ‘Man of the Year’ Studies from the 2014 pursued by Landry at the same time as devising the in 2011, and the UNESCO-approved route has UK Research Excellence Framework’, BMC route with colleagues in the UK and Turkey. Since brought sustainable tourism and commercial Medicine 13, no. 232 2010, Landry has accompanied commercial rides benefits to the region. (2015). doi: 10.1186/ s12916-015-0467-4 ‘Evliya Çelebi Way’ © Donna Landry ‘Evliya Çelebi Way’ © Gerald MacLean 44 Nicholas Stern, Building on Success and Learning from Experience: An Independent Review of the Research Excellence Framework (2016), pp. 22–23. Available at gov. uk/government/uploads/ system/uploads/ attachment_data/ file/541338/ind-16-9-ref- stern-review.pdf

45 See also impact.ref. ac.uk/CaseStudies/ CaseStudy.aspx?Id=39781

28 King’s College London 3. Pathways to impact

Case Study 8: Anya 17

Anya 17 was an opera produced by practice-led to Fight Human Trafficking (UN GIFT), which later researchers at the Royal Northern College of became a key partner in the project. Music. The work tackled themes of sex trafficking Press coverage amplified the messages and slavery in the UK, with the view to increasing underpinning the opera, bringing the discussion the profile of these issues among policymakers of trafficking and slavery beyond the opera-going and the general public. public. This was supported by the availability The opera began development in 2010 and of filmed rehearsals and a showreel on YouTube received its first performance in 2012. The libretto as well as social media presence. The chairman was underpinned by insights from preliminary of the Friendship Foundation (UK–Romania) consultations with anti-trafficking organisations, commissioned a follow-on performance at which was subsequently extended by a campaign the Third International Symposium on Human to gain endorsements for the opera itself from Trafficking in Arad, Romania, and the opera politicians, policy makers and anti-trafficking received subsequent performances in Germany, organisations such as the UN Global Initiative America and Wales.

‘Anya 17’ © Ben Kaye ‘Anya 17’ © Ben Kaye

Case Study 9: Working towards the Führer: shaping public understanding of Nazi power

From his 1998 biography of Hitler to his 2011 Hitler managed to seize and maintain power. study The End, Ian Kershaw’s published writings Kershaw has collaborated with documentary have contributed to a change in how the Nazi makers throughout this period, producing films regime is understood, replacing interpretative that were celebrated with a BAFTA and an clichés of ‘absolute evil’ and ‘banality’ with International Documentary Award. Worksheets an informed explanation of the ways in which have subsequently been developed to support the use of his 1997 documentary The Nazis: A Warning from History for use in A-level syllabi, and his publications inform teaching and examination at HE and A-level. The status of this body of work in informing discourse has been recognised by the award of the Leipzig Book Prize for European Understanding in 2012. Kershaw continues to deliver prestigious public lectures, and in 2010 his research also informed the first major public exhibition on Hitler in Germany. Kershaw was a member of the advisory council and delivered the opening lecture.

The creative role of research 29 3. Pathways to impact

benefits of collaboration with the sector. Because, understand how these partnerships benefit for assessment purposes, researchers were asked research, we need to look beyond impact case to demonstrate impact on the sector and not vice studies. The REF provides a strong starting versa, voices from the sector are conspicuously point from which to base this enquiry, but absent as agents in the production of knowledge; more qualitative work needs to be undertaken instead they serve to endorse or validate the in collaboration with practitioners to appreciate research. The insights that the REF yields, then, how HE and the creative and cultural sector come from the perspective of what research has work together to generate mutual impacts. done for the sector and not what the sector can do for research. This observation echoes a caveat offered in the REF analysis undertaken by the 3.2 What type of cultural organisations Museum University Partnerships Initiative, in did researchers collaborate with? which the authors caution that ‘REF case studies have limitations when it comes to identifying 94 per cent of case studies referred to a partner and articulating the benefits of partnership work outside of HE. Organisations partnering with to museums … because they are written from universities ranged from local arts centres to 46 National Co-ordinating a university point of view’.46 national institutions, with the BBC, British Centre for Public Engagement, Share Our analysis of impact mechanisms has Library, British Museum and the V&A being Academy, Museum- revealed the varied and cross-disciplinary nature among the most prominent (see Figure 9). University Partnerships in of pathways to impact involving the creative While our sample was chosen on the basis REF Impact Case Studies: A Review (2016). and cultural sector. However, in order to better of case studies that mentioned the creative

Figure 9: Sector partners (sized by most frequently cited) British Library Imperial War Museum Glasgow Women’s Library British Film Institute Penguin Books Home O ce Edinburgh International Book Festival Yale Center for British Art National Library of Wales Public Broadcasting Service Aldeburgh Festival Royal Shakespeare Company Department for Culture, Media and Sport National Library of Scotland Scottish Government Barbican The National Archives Institute of Contemporary Arts Science Museum RTE Foreign and Commonwealth O ce UNESCO King's Place Tate Britain National Maritime Museum Library of Congress JISC NPR Tate Modern Tate Museum Group Gerye Museum CNN Smithsonian Ofsted National Museums of Liverpool Natural History Museum Edexcel National Portrait Gallery FCO English Heritage Museum of Modern Art, New York John Rylands Library Birmingham Museum and Art Gallery National Geographic ARTE V&A House of Commons Wellcome Collection Globe Theatre National Gallery Getty Bodleian Library Museum of London AQA Cadw NHS Royal Festival Hall Edinburgh Fringe STV Department for Education ABC Edinburgh Festival Southbank Centre National Theatre Historic Scotland History Channel Sky British Museum Bloomsbury Fitzwilliam Museum Library of Congress ITV Ashmolean Museum Creative Scotland BBC Whitechapel Gallery Channel 4 British Council Royal Institute of British Architects Wellcome Collection Whitworth Gallery

30 King’s College London 3. Pathways to impact

More qualitative Figure 10: Partner types cited, by location (size of bubble represents the number of references work needs to be to an organisation) undertaken in collaboration with practitioners to appreciate how HE and the creative and International cultural sector work together to generate mutual impacts. National

London

Within the UK, researchers primarily engaged Local with creative and cultural organisations operating nationally

or within London. sm

Law

Film

Policy

Social

Health

Museum

Festivals

Religious

Education Transport

Publishing

Technology

Art & design & Art

Theatre & dance & Theatre

ience & medicine & ience

Sc

Heritage & archives & Heritage

oadcasting & journali & oadcasting

Br

Music & performance spaces performance & Music

and cultural sector, within these case studies we In relation to partnerships taking place within also identified partnerships that extended beyond the UK, researchers primarily engaged with this sector, encompassing organisations and creative and cultural organisations operating authorities in the education, health, legal, policy, nationally (21 per cent of references to sector technology and transport sectors. Researchers partners) or in London (34 per cent of references reported on collaborations with exam boards to sector partners), accounting for over half of all and education authorities such as the Assessment references to sector partners located in the UK. and Qualifications Alliance (AQA) and Office The remaining 45 per cent involved partnerships for Standards in Education (Ofsted), and with with regional organisations across the UK. Most policymakers with such diverse remits as the of these regional partners were based in England; Department for Culture, Media & Sport (DCMS), those in Scotland, Wales and Northern Ireland Historic Scotland, UNESCO, the Home Office were represented in just 15 per cent, five per cent and the Ministry of Defence. and one per cent of cases, respectively. 660 different organisations from the creative As shown in Figure 10, keyword searches and cultural sector were cited across the sample, for each of these 660 organisations revealed in the following areas: that broadcasters were the most common type • art and design (22 per cent) of partner cited in our sample. Organisations • museums (19 per cent) within art and design were prominent • broadcasters and journalists (17 per cent) (14 per cent), with a clear geographic preference • heritage sites and archives (15 per cent) towards the capital compared with other • music organisations and performance spaces regions of the UK. It was only in relation to (9 per cent) partnerships with heritage and archives, and • theatre and dance companies (8 per cent) to a lesser extent museums, theatre and dance, • film (4 per cent) that similar levels of engagement with partners • publishing (3 per cent) in the regions as in London were identified, • festivals (3 per cent). suggesting an as yet untapped opportunity

The creative role of research 31 3. Pathways to impact

for universities to develop partnerships with • Direct engagement: 89 per cent of case studies creative and cultural sector organisations based described a collaboration that researchers had REF case studies in the regions. initiated directly or proactively contributed to. overwhelmingly trace Just over a third of sector partners were located • Indirect engagement: 67 per cent reported a line leading directly outside of the UK, spanning six continents. This researcher involvement in an advisory capacity from the research to was concentrated around Europe, with 47 per or as a featured expert. the sector. Opening cent of these organisations situated in one of 19 • No personal engagement: 18 per cent included a dialogue about European countries, most prominently France examples of sector engagement with research, engagement with and Germany. The remaining spread of sector where no contact between the researcher and academic research partners was across North America (31 per cent) the practitioner had taken place. with the creative and Asia (17 per cent), with select partners in and cultural sector Australia, South America and Africa, which Each of the above types of engagement frequently may fill such gaps altogether comprised the remaining five per cent. overlap in a single case study, with 65 per cent of in our understanding These anomalies were made up of organisations in our sample describing engagement falling into two of research impact, Australia, Cuba, Mexico, Venezuela, Egypt, South or three of the above types. There was, however, a showing engagement Africa and Tanzania. As Figure 10 illustrates, the tendency not to report impacts where the researcher to be much more of majority of international partnerships were with had no personal involvement with the activities a two-way exchange. broadcasters and journalists, or within museums that enabled them. Crucially, of the 18 per cent that and heritage organisations. described indirect engagement, just six case studies (1 per cent of sample) reported exclusively on an impact with no personal engagement. 3.3 To what extent did researchers These anomalies nevertheless reveal important initiate collaborations with the creative pathways through which research was utilised and cultural sector? by practitioners. Case Study 10 offers an account of how the work of a historian was used as All of the partnerships analysed in our sample, foundational material for a novel; however, the be it on a local, national or international scale, author did not directly consult the researcher reflected one of the following types of engagement when writing the novel, despite explicitly citing with the sector: the research as a source of inspiration.47 A similar

Case Study 10: From scholarly historical research to prize-winning popular fiction

Timon Screech, a researcher at SOAS, has under-studied period to be brought to life’. published extensively in under-researched The novel has sold over 200,000 copies, and areas of Japanese art, history and culture, was awarded the Commonwealth Prize for with a specific focus on the Edo period. Literature in 2011. His work caught the attention of the writer David Mitchell, who drew on Screech’s publications as the historical basis for his 2010 novel The Thousand Autumns of Jacob de Zoet. The novel cites historical details and anecdotes contained only in Screech’s publications, drawing extensively on the monographs The Shogun’s Painted Culture 47 ‘From Scholarly Historical Research to Prize-Winning (2000) and The Lens within the Heart (2002) Popular Fiction – The to furnish the novel with historical accuracy. Thousand Autumns of The origin of this influence was noted by Mitchell, Jacob de Zoet’, REF 2014 Case Study. Available who described Screech as possessing ‘so deep at impact.ref.ac.uk/ a knowledge of the Tokugawa period – its arts, CaseStudies/CaseStudy. economics, politics, religion[,] trade, popular aspx?Id=42782 culture, its view of itself – and writes about this 48 ‘Latin London: Improving milieu so accessibly, and with such authority the Visibility of Latin and verve’. Americans in the UK’, REF 2014 Case Study. Yet Mitchell and Screech have never met. Available at impact.ref. In the words of Screech, what is noteworthy ac.uk/CaseStudies/ about this exchange is ‘its illustration of how CaseStudy.aspx?Id=17838 research can inspire and inform a much-lauded 49 H EFCE, Assessment work of literary expression, enriching it with © British Museum Framework and Guidance historically accurate detail and enabling an on Submissions (2011), p. 27.

32 King’s College London 3. Pathways to impact

example appears in relation to the No Longer just as multifaceted as the activities that result Figure 11: Date of Invisible report (2011), which identified the size from these partnerships? Our analysis suggests originating research and socio-economic status of the Latin American that, historically, new collaborations ranged from community in London for the first time. The the purely fortuitous to the strategic. It will be 1973 report not only helped in gaining recognition interesting to revisit this finding in future iterations 1 1974 for Latin Americans as an ethnic group among of the REF, as impact agendas are increasingly 0 policymakers and third sector organisations; its embedded in academic workflows in response to the 1975 wider role in empowering this community also new reporting requirements. Moreover, given that 2 1976 inspired a play highlighting the experiences of impact case studies are written from the perspective 0 1977 Latin American migrants in the UK, Juana in of the researcher, would research engagement be 0 a Million. The play went on to win a Fringe First described differently if written from the perspective 1978 award in 2012 at the Edinburgh Festival, yet it of the sector? While the case studies are a fruitful 0 1979 was only after the play had been staged that the starting point for this enquiry, they overwhelmingly 1 1980 researcher became involved, featuring in a Q&A trace a line leading directly from the research to the 48 4 at the Southwark Playhouse in 2013. sector. Opening a dialogue about engagement with 1981 It seems plausible that there are countless other academic research with the creative and cultural 2 1982 examples of research being put to work in the sector may fill such gaps in our understanding of 1 cultural sector, without any personal contact research impact, showing engagement to be much 1983 between a researcher and a practitioner, and more of a two-way exchange. 1 1984 that these were simply not reported in REF 2 1985 2014. One reason for this may be that researchers were unaware of such indirect impacts, as they 3.4 How long did it take for research 3 1986 are seldom cited by creative and cultural sector to achieve impact? 3 1987 practitioners. Yet it is also equally plausible 1 1988 that, given the REF determines allocation of REF 2014 set a clear timeframe in which impact government funding, researchers who did describe could be reported. The Assessment Framework and 1989 0 engagement did so with a view to demonstrating Guidance on Submissions49 released by HEFCE 4 1990 proactive pursuit of impact in the sector. in 2011 stipulated that the impact was required 1991 3 Researchers tended mostly to describe being to have taken place between 2008 and 2013. 5 reactive to opportunities arising from the sector The research that underpinned this was not 1992 when it came to media and cultural policy, permitted to predate 1993, albeit with some 1993 7 where emphasis was placed on advisory roles disciplinary exceptions where this was extended 30 1994 or invitations to appear as an invited speaker. due to the time lag between research and impact. In the case of TV and radio production, these We tracked the date of originating research, 9 1995 collaborations largely appear to have been tagging the earliest date cited in the section of the 17 1996 fortuitous, based on a researcher’s reputation case study describing the ‘Underpinning research’ 1997 23 as an expert in the field. Moreover, these or in the citations listed under ‘References to the 10 invitations were frequently aligned with trending research’. What we found was a relatively even 1998 news stories or contributions to programmes spread of research undertaken across the timeframe 13 1999 that had already been commissioned. Just 13 per specified by HEFCE, with peaks in 1993 at the cent of all references to engagement with media start of the underpinning research period, and 21 2000 companies described a project that the researcher across the three years immediately preceding the 22 2001 devised or took a central role in producing, impact assessment period (2008–2013) (see Figure 19 2002 be it in collaboration with a media company 11). The narrative descriptions of the underpinning 19 or independently. research also included a handful of outliers dating 2003 While REF 2014 succeeded in capturing a rich back to 1973. 18 2004 and broad legacy of partnerships with the creative There was no set point in the research cycle 25 2005 and cultural sector, what was reported was skewed at which the journey to impact commenced since 33 towards proving a causal link between a specific the length of time between initiating the research 2006 piece of research and the activity or mechanism and beginning to engage with the creative and 2007 30 that led to impact. As little data is available on cultural sector varied from one case study to the 29 2008 such indirect pathways to impact, our analysis next: exactly half of the case studies analysed is admittedly distorted by the omission of ways described an impact that took place after a piece of 23 2009 in which research can – and surely often does autonomous research had already been completed. 20 2010 – inform work in the sector without any direct On the one hand, these might describe impactful contact between academics and practitioners. projects in which the researcher had proactively 2011 11 Our analysis of sector partners also raised built on previously completed research; on the 4 2012 many questions that beg further investigation. other, they may describe a fortuitous moment 1 Are there established ways in which researchers in which the research was independently picked and practitioners form collaborations, or is this up, or thrust into the public eye as a result of its

The creative role of research 33 3. Pathways to impact

‘Madness and Modernity © Wellcome Collection

34 King’s College London 3. Pathways to impact

relevance to current events. 30 per cent of the work or influences, therefore limiting insight remaining case studies described impact as being into how impact evolves over time? intrinsic to the project from the outset and 20 per • Given the assessment purpose of REF case cent related to ongoing practice-led research. studies, did authors choose to include their Practice-led research was cited relatively evenly best publications rather than those that best across the timeframe displayed in Figure 11, demonstrate the trajectory of the underpinning accounting for many of the anomalies that predate research and its impact?50 the assessment period, whereas impact subsequent Invariably, the decisions that each researcher made to research accounts for the peak at the beginning on the above three points will have affected the of the assessment period in 1993. Proportionally, reported date of originating research. Moreover, impact that had been built into the underpinning the majority of case studies captured only short- research process reported the quickest return, with term impacts. While the data gathered suggests the originating research most commonly dating that the impact of research on or enabled by between 2005 and 2008. the creative and cultural sector happened more These observations about impact timescales quickly than research impact in other sectors (the should, however, be read with the following time lag for research impact in the health sector, caveats in mind: for example, is typically estimated at 17 years),51 • Does the ‘Underpinning research’ relate to a further investigation with additional parameters specific project, or is it a small part of a larger is needed since, at present, the REF 2014 case research profile? studies do not yield enough information to draw • To what extent do the time constraints conclusions about the longevity of impact in the stipulated in the REF exclude or distort earlier creative and cultural sector.

3.5 Pathways to impact: key findings

• REF 2014 demonstrated a significant • The partnerships that researchers engaged appetite for working with the creative and in, be it on a local, national or international cultural sector across Higher Education, scale, were the product of both direct and with pathways to impact including multiple indirect engagement. interdependent mechanisms and activities • Impact may be fortuitous, surfacing after a enacted via an array of partnerships and body of research has already been published, collaboration types. or it can be intimately worked into the • Impact is seldom enabled by just one conception of a project. The latter, however, activity or mechanism. Only two per cent appeared to provide the fastest route to of case studies reported on a single impact impact, typically returning an impact in just mechanism; instead, an average of six 5–8 years. mechanisms were cited per case study • The case study format encouraged authors across our research sample. to trace impact from originating research • The most common impact mechanism was to activity in the sector, leading many to public talks, which occurred in 43 per cent report on a causal, research-centric pathway of the case studies analysed. to impact in the sector. It is likely that • ‘Music, performing and visual arts’, ‘Film, TV, this focus on forward trajectories within video, radio and photography’ and ‘Museums, a prescribed timeframe distorted the complete galleries and libraries’ surfaced as hot spots for picture of research impact in the creative and 50 See King’s College London and Digital Science, sector engagement with research. cultural sector. The Nature, Scale and • Researchers mostly engaged with organisations • More work is needed to link findings to the Beneficiaries of Research in the UK that have a national or London perspectives of the creative and cultural Impact (2015) for further discussion of this point. remit. Of the regional partners, most were sector. Does engagement with researchers, based in England. for example, play into a larger impact 51 Zo ë Slote Morris, Steven • Just over a third of sector partners were strategy for these creative and cultural sector Wooding, and Jonathan Grant, ‘The Answer is located outside of the UK, spanning six professionals? How often is academic research 17 Years, What is the continents. These were concentrated in drawn on without direct collaboration? Question: Understanding Europe, with 47 per cent of international Answers to these questions fall beyond the Time Lags in Translational Research’, Journal of the partner organisations situated in one of remit of the REF, yet are critical for gaining Royal Society of Medicine 19 European countries, most prominently a more complete picture of the ways in which 104, no. 12 (2011): pp. France and Germany. research informs work in the sector. 510–520.

The creative role of research 35 4. Evaluating and evidencing research impact

–4– Evaluating and evidencing research impact

‘Discover Turner’s Yorkshire’ © Welcome to Yorkshire

36 King’s College London 4. Evaluating and evidencing research impact

Strategies for evidencing impact were deliberately left open in REF 2014. Authors of case studies were required to provide evidence to support their claim for impact; yet the ways in which evidence of the impact itself was captured and described were at the discretion of the individual, with no single approach or set of metrics prescribed. Based on a close reading of our qualitative research sample, this section captures the different types of evidence used to evaluate research impact resulting from work involving the creative and cultural sector, and identifies their nuances, strengths and weaknesses, and untapped potential.

The guidelines for submissions to REF 2014 4.1 What types of evidence are used contained examples of suitable evidence types, to evaluate research impact? tailored to the idiosyncrasies of each academic panel tasked with scoring impact case studies,52 The evidence types identified in our sample and according to the broad criteria of ‘reach’ ranged from quantitative data illustrating reach and ‘significance’. However, the way in which or income to qualitative personal narratives that these two parameters were to be interpreted and described growth and empowerment. In a single benchmarked was decided by the expert panels, narrative, a researcher may describe broadcasting and tailored according to the nature of research their findings to millions alongside testimonials in each discipline.53 How to evidence impact was, that convey profound impact on the life of an accordingly, one of the biggest unknowns in the individual; they may describe the cultivation exercise – both for case study authors and the of a loyal audience through community outreach, members of the expert panels tasked with assessing while validating this with endorsements such as the submissions. industry-leading awards or favourable reviews What follows is an analysis of the evaluation in national newspapers. methods Higher Education Institutions (HEIs) and These evidence types naturally fell into the two academics intuitively used to assess their impact. criteria highlighted for assessment in REF 2014: Yet it should not be read as a guide on how to reach and significance. As shown in Table 3, the score highly in the REF. We do not make value definition of reach and significance published by judgements about the strength of the evidence, HEFCE differed between each academic panel, but read each case study as a claim for impact, albeit broadly focused around the scale and critiquing how the evidence was deployed and diversity of beneficiaries, and the way in which what it contributed to the impact claim. The research made a difference for an individual or extent to which the information itself is accurate community, on policy or practice. It was also or authoritative was not assessed. emphasised in the Assessment Framework and

Table 3: Definition of ‘reach’ and ‘significance’ in REF 201454

Reach Significance Biological ‘The spread or breadth of influence or ‘The intensity of the influence or effect.’ Science and effect on the relevant constituencies.’ Medicine

52 See HEFCE, Assessment Physical ‘Reach is the extent and breadth ‘Significance is the degree to which the impact has Framework and Guidance Sciences and of the beneficiaries of the impact.’ enabled, enriched, influenced, informed or changed the on Submissions (2011). Engineering products, services, performance, practices, policies or understanding of commerce, industry or other 53 See HEFCE, Research organisations, governments, communities or individuals.’ Excellence Framework Impact Pilot Exercise: Social ‘Reach will be understood in terms ‘Significance will be understood in terms of the Findings of the Expert Sciences of the extent and diversity of the degree to which the impact has enriched, influenced, Panels (2010). Available at communities, environments, individuals, informed or changed policies, opportunities, ref.ac.uk/pubs/refimpact pilotexercisefindingsofthe organisations or any other beneficiaries perspectives or practices of communities, individuals expertpanels/ that have benefited or been affected.’ or organisations.’

54 HEFCE, Panel Criteria Arts and ‘The extent and/or diversity of the ‘The degree to which the impact enriched, influenced, and Working Methods Humanities organisations, communities and/ informed or changed the policies, practices, (2012): pp. 34, 54, 74, 93. or individuals who have benefited understanding or awareness of organisations, Available at ref.ac.uk/ from the impact.’ communities or individuals.’ pubs/2012-01/

The creative role of research 37 4. Evaluating and evidencing research impact

Guidance on Submissions55 that reach did not Table 4: Evidence types for reach and significance specifically relate to a geographic scale, and that (ordered by frequency) Impact generated no preference would be made between local, from creative projects national or international impacts. No. case % case is not evaluated by Each type of evidence that we identified within studies studies in just one yardstick sample the ‘Details of the impact’56 is listed in Table 4. but through a web of ‘Details of the impact’ is the section in which the Reach 365 88% indicators, each adding evidence for the impact happening is described, Number of attendees, 288 69% a new, crucial layer to using whichever set of indicators, metrics or participants, users the claim for impact. descriptions the researcher deemed appropriate. Distribution, network reach 191 46% We identified 18 evidence types that were Hits, downloads, shares, likes 165 40% specifically used to describe research impact in the creative and cultural sector. Number of events, reissues 106 25% Multiple evidence types were typically used Significance 403 97% in conjunction to evaluate impact. In our analysis, Individual testimonial 278 67% an average of five evidence types appeared per Review 133 32% case study, with 85 per cent of case studies offering evidence of both reach and significance. Of the Subsequent commission 120 29% remainder, 12 per cent presented significance or collaboration as the sole evidence type and just three per cent Press coverage 114 27% exclusively supplied evidence of reach. Award 97 24% As listed in Table 4, evidence of reach took Extent of discussion online 83 20% one of four forms: the ‘number of attendees, participants or users’; ‘distribution or network Feedback form 72 17% reach’; ‘hits, downloads, shares or likes’; or the Grant/government funding 68 16% number of people implicitly reached through for sector partner ‘repeated events or reissues’. All four approaches Recommendation or rating 62 15% to evaluating reach related to the scale of the Citation 61 15% dissemination, primarily through quantitative data describing the size of a demographic Research used in policy, 51 12% or audience, be it reached in person, online report, syllabus or via an indirect network. Amount of income generated 43 10% Evaluation of significance was, however, more Employment and 23 6% nuanced. We identified over three times as many economic growth indicators of significance than those that described Work procured for 15 4% reach. These ranged from personal testimonials exhibitions or collections from audience members, practitioners or policy makers to the amount of income generated. Contrary to reach, which mostly described the were most prevalent, appearing in 95 per cent of scale of the dissemination, the 14 indicators of the case studies supplying evidence of significance. significance were communicated as numbers, text, By contrast, pledges to ‘think or act differently in quotes or statistics. They also conveyed a broader the future’ and demonstrable changes in opinion, 55 See HEFCE, Assessment range of information. We found that each piece practice or behaviour occurred in just 31 per cent Framework and Guidance of evidence, regardless of type, may constitute: and 42 per cent of case studies, respectively. on Submissions (2011): p. 45. 1. an endorsement Quotes from testimonials and reviews were 2. a pledge to think or act differently in the future frequently woven into the prose of a case study 56 Unless it required further 3. an actual change in opinion, practice or behaviour. numerous times to illustrate significance in the contextualisation, we analysed this section of words of the beneficiary. Information about the the submission form as Evidence of impact that pledged or confirmed a ‘number of attendees, participants or users’ also a standalone statement. change in opinion, practice or behaviour tended received a high number of repeat references, The supporting information under to substantiate the impact itself, whereas evidence and was often included to give a sense of scale ‘Corroborating evidence’ that acted as an endorsement spoke more to the to the reach of each impact mechanism referenced tended to repeat what quality of the activity or project that enabled within a case study. This indicates that, similar was already described in more detail under an impact. to the ecosystem of impact mechanisms described ‘Details of the impact’, Considered in isolation, these strands of in Section 3.1, impact generated from creative and often pointed out information may not, then, always evidence impact projects is not evaluated by just one yardstick but to sources such as email exchanges or confidential in and of themselves: over half of the case studies through a web of indicators, each adding a new, data collected by included evidence of significance that fell into at crucial layer to the claim for impact. partnering organisations, least two of the above three categories, with 13 In the following sections, we drill down further and hence were not in the public domain. per cent including all three. Forms of endorsement to understand how these types of evidence are

38 King’s College London 4. Evaluating and evidencing research impact

used by researchers to evaluate the impact of described the scale of the impact mechanism or their work. We outline how this information is activity, be it listing the number of people present presented, what it conveys about impact, and the at an event or the numbers of impressions58 made areas in which evaluation could be strengthened. via promotional literature or downloads of an online resource. The statistics gathered in REF 2014 about the 4.2 Evidencing reach number of people engaging with research were, however, limited. Although evidence of reach Evidence of reach typically measured provided meaningful context about the impact the number of people engaged by research. mechanisms on an individual case study level, it The most common indicator was the ‘number is impossible to draw comparisons across multiple of attendees, participants or users’, occurring case studies due to the inconsistency of the metrics in 69 per cent of case studies in the sample. used. With distribution, for example, how can a This was most commonly cited in relation case study that cites the ‘estimated UK audience’ to ‘Music, performing and visual arts’ and the be distinguished from another that presents precise ‘Museum, gallery and library’ sectors. Only hits from a platform such as iPlayer? We have read five per cent of all references to audience and case studies that alluded to the average weekly user statistics presented reach in indefinite terms, audience of a television network as well as those simply stating that an event had been ‘well that drill down into the number of people engaged attended’ or achieved a ‘capacity audience’. by a single episode; we found case studies that The remaining 95 per cent instead cited statistics gave this number in isolation, and others that such as the exact numbers of tickets sold, or the benchmarked it against the typical audience number of registered users or attendees. share; and we found numerous examples where Numerical data also prevailed in descriptions these statistics were provided without any of ‘distribution or network reach’, with 93 per cent reference to the provenance of the data. Given of all references to such data measuring the size such inconsistent measures, any attempt to of the demographic. This encompassed broadcast benchmark reach on a national level would networks, the print run of a magazine or book, or be inconclusive, if not futile. the number of subscribers connected to research A further limitation surfaces in terms of via social media. Evidence of distribution was most who was reached. Of the case studies that cited common among media and publishing projects, information about the ‘number of attendees, where audience share, sales distribution or the participants or users’, just 16 per cent described circulation of a magazine all gave a sense of scale the demographic. Of this 16 per cent, most offered to the network engaged. cursory information such as country names or ‘Hits, likes, downloads and shares’ were the qualifiers, such as ‘school groups’. More detailed third most common indicator of reach, with 40 information tended to be provided only if it related per cent of case studies in the sample describing to a target stakeholder group, such as a particular the size of the community engaged online. age group or ethnicity. Allusions to the ‘number of events or reissues’ When referring to a more heterogeneous also signalled reach through illustrating a greater group, such as the ‘general public’, representations number of opportunities for exposure, such as of gender, socio-economic status and race were repeat performances of a new work, extended seldom mentioned. The same also surprisingly runs of exhibitions or theatre productions, occurred with new media. Just one per cent of all rebroadcasts, or new editions of published works. metrics relating to ‘hits, downloads, shares and Information about reach typically contextualised likes’ failed to state a tangible number for reach; 57 ‘The Antikythera the impact activity or mechanism, affording yet only 11 per cent spoke in detail about who Mechanism: Seeing Inside a sense of scale to its dissemination. There were they reached. Given the availability of information a Two-Thousand-Year-Old Computer’, REF 2014 Case a small number of exceptions to this rule, where about location, age, gender and dwell time through Study. Available at impact. increases in the number of people engaged were tools such as Google Analytics, this suggests ref.ac.uk/CaseStudies/ used to identify audience development. For a failure to dig deeper into exactly who the CaseStudy.aspx?Id=3465 example, a case study describing the discovery of beneficiaries ‘beyond academia’ were. 58 An impression (in the structure and functions of the world’s earliest Notwithstanding the above caveats, the the context of online known astronomical calculator, now housed in evaluation of reach in our sample reflected the advertising) refers to the number of times an the National Archaeological Museum in Athens, impressive scope of the creative and cultural advert is viewed once by illustrated growth by demonstrating an 86 per cent sector in brokering a wider audience for academic a visitor or displayed on increase in visitor numbers compared to the same research and also highlighted the scale of the a website. Whether the ad is clicked is not taken into period the previous year, at a time when ‘other importance of research to the sector. A more account. Each time an ad museums in Greece … suffered a general fall in nuanced understanding of the demographics is displayed, it is counted numbers’.57 Yet such uses of reach data were in a behind these numbers is essential if we are as one impression. notable minority. For the most part, ‘reach’ purely to fully grasp the public benefit of research.

The creative role of research 39 4. Evaluating and evidencing research impact

Moreover, while reach was largely presented 4.3 Evidencing significance as a quantitative measure, a qualitative layer Interweaving statistics of information about the type of engagement it Evidence of significance took many forms. describing the scale of described also appeared vital. Little distinction can Whereas some evidence types demonstrated reach with qualitative be made between direct and indirect beneficiaries approval from the creative and cultural sector, information about the when considering reach in purely statistical terms. be it expressed publicly in reviews or privately in ways in which people For example, whereas distribution networks such testimonials from individual practitioners, others engage with research as TV or radio networks may yield significant reflected public opinion gathered through feedback helps to guard against dissemination in quantitative terms, potentially forms, or voiced indirectly in discussion online using metrics that distort broadcasting research findings to millions, the after an event. the quality of engagement nature of the audience’s engagement is likely to be Overwhelmingly, we found that illustrations with research beyond indirect. By contrast, forms of direct engagement, of significance, regardless of the evidence type academia. which typically yield much lower reach in used, were a form of endorsement: the information numerical terms, may have a more enduring tended not to corroborate the impact claim impact. Community arts projects, for example, directly, but, like reach, imparted descriptive may only touch a small number of people, but information about the activities or ‘mechanisms’ through prolonged exposure to the research that enabled impact. 91 per cent of case studies these stakeholders are more likely either to reflect used evidence of significance to endorse the impact on, or enact a change based on their experience. activity in this way. Yet 57 per cent of case studies It is therefore important not to define reach too presented additional evidence of significance narrowly. Interweaving statistics describing the that went beyond endorsement and testified scale of reach with qualitative information about the to a change in opinion, practice or behaviour: ways in which people engage with research helps to • 30 per cent of case studies analysed included guard against using metrics that distort the quality evidence of change that captured an immediate of engagement with research beyond academia. response to the research, describing transformed

Figure 12: References to significance evidence types and their described impact

Long-term change

Pledge to think or act di erently

Endorsement

g

n

e

Award

Review

Citation

Grant funding Grant

Feedback form Feedback

or collaboration or

Press coverage Press

Employment and Employment

Research used in used Research

economic growth economic

Income generated Income

Work procured for procured Work

hibition or collection or hibition

Individual testimonial Individual

report, policy, syllabus policy, report,

ex

ecommendation, ratin ecommendation,

Subsequent commissio Subsequent

R Extent of discussion onlin discussion of Extent

Audience/user feedback Financial indicators Industry status Informing discourse Testimonials

40 King’s College London 4. Evaluating and evidencing research impact

perspectives, enhanced knowledge, or stating of it (in contrast to case studies that had an impact how the beneficiary planned to approach tasks within the sciences, which typically do not count or situations differently. awards or fellowships as evidence of impact). This • 41 per cent included references to evidence that reflects an instinct to show the perceived value went further to document actioned, longer-term of a piece or body of research by the creative and change, be it implementing a policy or practice, cultural sector rather than evidencing a substantive or making an enduring change to their lifestyle, change as a result of it. opinions or wellbeing. 4.3.2 Audience or user response Certain types of evidence seemed more suited to Responses from audience members, consumers documenting change than others. As the matrix of or service users also primarily served as a form evidence types in Figure 12 illustrates, individual of endorsement. Recommendations or ratings, for testimonials, feedback forms and the use of example, were typically expressed in conjunction research in reports and syllabi were more likely to with a specific output or activity. These pieces evidence change than the conferment of an award, of evidence may endorse an event or exhibition recommendations or ratings, or details of the by virtue of being featured as a critics’ choice; acquisitions of creative output(s) for collections or the significance of a book may be shown by its exhibitions. As outlined in the following sections, presence on bestseller lists, or through customer the latter served more to contextualise the claim reviews posted on Amazon. All but one reference for impact by offering evidence of its status within to recommendations and ratings constituted the sector, rather than evidencing change via the an endorsement, with just one case study using direct beneficiaries of a project, who witnessed it a rating to describe a change in practice. or helped to make it happen. The scale and nature of online discussions In the following section, we discuss how generated by research were also presented as different forms of evidence were used to evaluate an indicator of audience or user response. 65 significance. We start with those forms that per cent of references to discussion on social more frequently conveyed an endorsement and media or blogs solely indicated the scale of end with forms that were more inclined towards this activity (for example, ‘x many comments’, documenting change. ‘y many blog posts’), whereas the rest included a summary of the discussion or provided quotes. 4.3.1 Industry status Of these, just 13 per cent pledged to think or act References to awards (featured in 24 per cent differently in the future and two per cent reflected of case studies) were the most overt evidence a longer-term change. Overwhelmingly, the of an endorsement, the range and prestige evidence either endorsed the research or revealed of which present a compelling account of how more about the ripples of discussion immediately academic research has led to industry-leading following an event, saying as much about the reach work. This includes references to films awarded of the research as it did about its significance. or shortlisted for Oscars, BAFTAs and Emmy It was only in formal data gathered via feedback Awards, literature awarded the Man Booker Prize, forms and surveys that responses from users or and plays celebrated with a Fringe First Award. audience members communicated change. Just Endorsement from the sector also manifested 35 per cent of references to data gathered via in less formal measures. Subsequent commissions feedback forms presented an endorsement or or collaborations were alluded to in 29 per cent described the potential for change, with statements of case studies to illustrate a growing reputation such as ‘x per cent of respondents said they wanted in the sector. This might include researchers to learn more’. Of the remaining 65 per cent, just being invited to enter into a new collaboration over half pledged to act differently in the future, or to undertake a new commission based on the and the remaining 32 per cent spoke to a longer- success of preceding projects; other indicators term change. also included invitations to serve on an advisory board, to act as a consultant or to judge awards. 4.3.3 Informing discourse Purchases or acquisitions of creative output(s) 83 per cent of references to reviews in newspapers, (4 per cent) served a similar role, in which magazines or broadcast media contained attributed work procured for exhibitions or collections quotes. Many of these quotes expressed strong was used as an indicator of its industry standing, approval, reflecting how the work of a researcher, mostly occurring in practice-led research in especially those engaging in practice-led research, areas such as the visual arts. Only a quarter was received on a par with industry professionals. of references to acquisitions originated from The substance of these quotes predominantly research-led projects. endorsed these activities, with just three case Crucially, all of these measures contextualise studies (0.7 per cent of sample) describing an impact rather than serving as direct evidence enduring impact on practice.

The creative role of research 41 4. Evaluating and evidencing research impact

‘Performing Migration’ © Mark Loudon

Press coverage was also used as an indicator Other areas of discourse where substantial of research gaining traction beyond academia, evidence of change was captured were indicated though just 12 per cent of references to press through formal citations in articles and academic coverage described the substance of the coverage. journals, or the use of research in a policy, report The remaining 88 per cent simply noted the or syllabus. An instructive example occurs in a newspaper title or reported ‘extensive coverage in case study from the Institute of Education (IOE), the press’, providing indicative examples of select whose scrutiny of music services offered by local titles. Again, such evidence could be interpreted authorities prompted increased investment in as evidencing reach as much as it does significance. music education by the national government. Some anomalies do, however, exist where The influence of these studies was illustrated press coverage is presented as an indicator through references to new schemes for music of change. For example, a team of researchers education launched by Labour, and in the actions at the Brighton and Sussex Medical School, subsequently taken by the government to address led by Prof. Somnath Mukhopadhyay, presented issues highlighted by the IOE reports.60 changes in journalistic discourse to illustrate how they had changed attitudes towards the benefits 4.3.4 Financial indicators of personalised asthma treatment for children.59 The creative and cultural sector is increasingly 59 ‘ Personalising Asthma The media coverage that ensued from a press recognised as being an important area of the Care for Children’, REF 2014 Case Study. Available conference in 2009 initially focused, for example, UK economy, with annual economic estimates at impact.ref.ac.uk/ on the inhaler failing children, reflecting a narrow published by the DCMS61 consistently reporting CaseStudies/CaseStudy. interpretation of the research findings rather revenue and employment above the national aspx?Id=39753 than communicating the benefits of personalised average. Yet in REF 2014, evidence of income 60 ‘Music in Schools: healthcare: ‘Asthma inhaler may not work for generation was used surprisingly infrequently Boosting Achievement, some children, study shows’ (Guardian); ‘Asthma to evaluate the impact of research within the Enthusiasm and Participation’, REF 2014 inhaler failing children’ (BBC News). Further press sector, appearing in just 28 per cent of the case Case Study. Available coverage following a second press release four studies analysed. at impact.ref.ac.uk/ years later revealed a more holistic interpretation The financial indicators used by researchers CaseStudies/CaseStudy. aspx?Id=44311 of the research. The writer for the Guardian, in fell into one of three broad types: particular, described the research as ‘a wonderful 1. grant or government funding for a sector partner 61 See DCMS, ‘The Creative example of stratified or personalised medicine 2. amount of income generated Industries Economic Estimates’ (2016). working its way into practice’. 3. evidence of employment and economic growth.

42 King’s College London 4. Evaluating and evidencing research impact

Almost half of the references to financial indicators is an intrinsic part of an academic career, and related to securing grant funding for a sector is a skill that can usefully be brought to the sector. partner. This only includes the instances where But more importantly, it is an activity that is the funding was an outcome of a project or unambiguously attributable: if a researcher has collaboration, or directly supported activity within contributed to a funding bid, a clear, causal link the sector: any references to the funding that can be made between the effort and the funding enabled the originating research, or that supported acquired. Income generated via commercial further autonomous research were not included. activities may also be linked to tangible projects, In some cases, references to grant funding products or artworks to which a researcher has evidenced the impact indirectly, with the award directly contributed. However, where the causality of follow-on funding testifying to the long-term of wider economic impacts was more ambiguous, significance of work in the sector. For example, researchers appeared reluctant to use financial a case study describing a successful World Heritage indicators to link these impacts with their bid to protect an important fossil site in Yunnan underpinning research. The limited data reported Province (China) reported subsequent government about employment and economic growth thus funding of 78 million yuan to help protect the tended to be in the context of work in a small site and build a natural history museum.62 More community, such as developing a heritage site common, however, were examples where a where measurements such as a boost to tourism researcher made a direct contribution to securing in the local area have a more demonstrable and funding that enabled further activities in the sector lasting link to the project at hand. (either in collaboration with, or as a consultant This reluctance to measure wider economic to a sector partner). Such references to funding impacts was also identified in theMuseum- occurred in 16 per cent of the sample, including University Partnerships in REF Impact Case Studies grants from the Heritage Lottery Fund (HLF); the report, which found ‘limited evidence of the Arts and Humanities Research Council (AHRC); partnerships securing economic returns for the the Esmée Fairbairn Foundation; Arts Council museums and the regions in which they are based’.63 England (ACE); and the Wellcome Trust, among others. These funds predominantly supported 4.3.5 Testimonials activities in the museum and heritage sector. The most widespread and direct evidence of The amount of income generated was a less impact came in the form of testimonials from common indicator of significance, featuring in just an individual, which underpinned 67 per cent of 10 per cent of the sample. Yet in contrast to grant case studies in the sample. Most were supported funding, income generation overall was evidenced with an attributed quote, with just five per cent of across all areas of the creative and cultural sector, testimonials simply using phrases such as ‘positive from theatre, visual arts, heritage, film, libraries feedback’ or pointing to statements held on file. and archives, through to museums, music, crafts, Moreover, 42 per cent of these testimonials went dance and broadcast media. Over two-thirds of beyond an endorsement: one-third captured an 62 ‘UNESCO World Natural these references described the income generated in immediate response to the research, while the Heritage Status for the quantitative terms, such as descriptions of turnover remaining two-thirds described how exposure to Chengjiang (Cambrian) or income from the sales of tickets or merchandise; the research had led to a change in their practice, Fossil Site, Yunnan Province, China’, REF 2014 only 31 per cent of case studies appealed to opinions or behaviour. Case Study. Available the level of income generated but provided no These statements came from a variety at impact.ref.ac.uk/ supporting figures for income generation at all. of stakeholders, including: CaseStudies/CaseStudy. aspx?Id=18052 Evidence of employment and economic growth • collaborators (32 per cent of testimonials)

The regional Chinese was a surprising outlier, featuring in just six per • consumers, service users and audience Government in Yuxi City, cent of the case studies analysed. The use of this members (30 per cent) Yunnan, who control evidence type was also remarkably sector specific, • peers and practitioners (28 per cent) and administer this UNESCO site, continue with over half of these references occurring in • policy makers (9 per cent) to make infrastructure narratives where the main impact was in the • funders (1 per cent). expenditure in and museum or heritage sector. The evidence cited around the site for scientific, educational and ranged from job creation in a start-up to generating In statements from collaborators, peers and tourism purposes (email employment across the industry as a whole; practitioners, the substance of a testimonial correspondence with and from boosting tourism and spend in local typically endorsed the work, identifying its Derek Siveter). businesses to raising GDP. This data tended to be successes, promise or significance. Yet some 63 National Co-ordinating attributed to a specific event or project, derived testimonials also offered an account of how Centre for Public from independent reports or economic estimates practitioners had subsequently embraced new Engagement, Share Academy, Museum- produced by government or local councils. creative practices, workflows or curatorial University Partnerships in This dominance of grant funding as a financial directions after exposure to academic research. REF Impact Case Studies: indicator raises important questions about how A similar spectrum is evident for policymakers: A Review (2016). impact was reported. Writing funding proposals whereas some endorsed the research as a model

The creative role of research 43 4. Evaluating and evidencing research impact

‘Cultural Policy and Practice Exchanges between Britain and Brazil’ © Ratão Diniz

of best practice (24 per cent of testimonials exhibition and talk delivered at the Aberystwyth from policymakers), others went on record to Arts Centre by a survivor of the Kinder transport state how findings had been implemented in movement, one respondent stated that it ‘made policy decisions, how research had informed the me consider the different types of immigration allocation of funds, or shaped national reforms and that they [sic] are a lot of immigrants that (49 per cent testimonials from policymakers). don’t fit the stereotypes’.65 For example, practice-led work on Brazilian arts An exception comes with participatory research, practices in Britain by Paul Heritage at Queen which is unique in having ‘more opportunities for Mary University of London caught the attention multiple and unhurried data collection moments’.66 of the then Executive Director of Arts Council In these instances, we found testimonials that 64 ‘Cultural Policy and Practice Exchanges England, Moira Sinclair, who stated that ‘I and spoke of transformed perspectives, empowerment, between Britain and other senior colleagues have been actively engaged breaking down of stereotypes, self-reflection and Brazil’, REF 2014 Case in debate, in reading your papers, in exchange the building of identity. For example, a case study Study. Available at impact. ref.ac.uk/CaseStudies/ with this programme. … Partly as a result of this associated with the Performing Migration project, CaseStudy.aspx?Id=19052 engagement, we designed and launched a new led by Lena Simic at Liverpool Hope University, 10 year programme, Creative People and Places presented a testimonial from a journalist exiled from 65 ‘ The Kindertransport to Britain 1938/39: New which seeks to engage communities in the UK in Gambia, who after seeking asylum in the UK was Perspectives’, REF 2014 new and radically different approaches to develop invited to join Simic’s participatory theatre project Case Study. Available inspiring and sustainable arts programmes’.64 How to Become British. His testimonial expressed at impact.ref.ac.uk/ CaseStudies/CaseStudy. Testimonials from funders, service users and how he had made a transition from a place of distress aspx?Id=44022 audience members, by contrast, were more inclined and confusion to a point in which he felt empowered towards endorsements. With funders, this often to state his own story proudly through performance, 66 John Knell, et al. ‘Home: Quality Metrics Research constituted a positive evaluation of a project within reflecting how ‘this invitation not only gave me & Development Report.’ the funding period. Those from service users and courage to speak out about my ordeal in an open November 2015, p. 64. audience members also seldom reflected on the forum, but it also gave me opportunity to meet other 67 ‘P erforming Migration’, enduring resonance of an encounter with research. people, may be, in a similar situation and, as a result REF 2014 Case Study. More common statements relating to change contributed a lot in making me who am I today’.67 Available at impact.ref. are those that attested to the immediate response And in her work with a prisoner re-entry ac.uk/CaseStudies/ CaseStudy.aspx?Id=41441 to an event. For example, after attending an programme in Ohio, Nicola McCartney,

44 King’s College London 4. Evaluating and evidencing research impact

a practice-led researcher at the University of of how academic research has touched the lives Evaluating research Edinburgh, similarly translated work in Applied of those living through human rights or mental impact in the creative Theatre to encourage formerly incarcerated health issues. and cultural sector in women re-entering civilian life to develop more These examples resonate with a shift towards REF 2014 was as much constructive responses to situations of conflict, more flexible evaluation strategies among about validating and using the principles of dramatic narrative to help participatory arts researchers, with a recent paper contextualising the people better interpret and address their own life by Kate Wakeling and Jonathan Clark calling activities that enabled circumstances. One respondent reflected: ‘She’s for the use of more qualitative evidence that an impact as it was about a life-saver, really, because she can talk to you and provides ‘a stronger platform for participants’ supplying evidence that get you to say things that I would normally keep own reflections and which in turn celebrate the corroborated impact inside’. 68 In such testimonials, a clear sense is given complexity and nuance of such interactions’.69 directly.

4.4 Evaluating and evidencing research impact: key findings

Evaluation is not Evaluating research impact in the creative occurring in just 31 per cent and 42 per cent something that occurs and cultural sector in REF 2014 was as of case studies, respectively. at a set point at the much about validating and contextualising • Quotes from testimonials and reviews were end of a project, but is the activities that enabled an impact as it was often woven into the prose of a case study a reflexive process that about supplying evidence that corroborated numerous times to illustrate significance is more usefully set in impact directly. The open format of the case in the words of the beneficiary. motion at the beginning study approach in REF 2014 accommodated of a project and extends this spectrum of direct and indirect evidence There were, however, limitations to the types long beyond its end. types, combining strands of information of evidence used: about reach and significance to present • The narrative approach to describing reach a more holistic description of impact and precluded an analysis of the data gathered on its supporting activities: anything but a case study by case study basis, • 18 evidence types were identified that were due to the inconsistency of the quantitative specifically used to describe research impact measures used. in the creative and cultural sector. • Information about reach offered little • An average of five types appeared in each indication of the demographics represented case study, with 85 per cent of case studies by quantitative measures; this is essential if we presenting evidence of both reach and are to grasp fully the public benefit of research. significance. • The case studies yield limited evidence of how • Evidence of significance was the most researcher engagement has contributed prevalent, underpinning impact narratives in to wider economic activity within the sector. 97 per cent of the case studies in the sample; • The evaluation of significance was evidence of reach appeared in 88 per cent of predominantly anchored in the the case studies analysed. immediate period after exposure, • Evidence of reach was largely communicated typically communicating an endorsement 68 ‘Life as Story: The Applied as quantitative data that described the or a change in opinion for the short term. Theatre Practice of Nicola scale of the impact mechanism, whereas • Overall, little was reported about the longer- McCartney’, REF 2014 the information conveyed by evidence term impact of the activities and mechanisms Case Study. Available at impact.ref.ac.uk/ of significance was more nuanced, ranging described. This need to support longer-term CaseStudies/CaseStudy. from endorsements and pledges to think monitoring of impact was reflected in the aspx?Id=24015 or act differently in the future to reflections recommendations put forward in Lord Stern’s 69 Kate Wakeling, and on longer-term change. independent review of REF 2014, in which Jonathan Clark, ‘Beyond • Forms of endorsement tended not to it was observed that ‘realising the full impact Health and Well-Being: corroborate the impact claim directly, but of research is a continuing and sometimes Transformation, Memory 70 and the Virtual in Older instead provided descriptive information long-lived process’. Such monitoring of People's Music and about the activities or mechanisms that impact can, however, only be fostered by a Dance’, International enabled impact. 91 per cent of the case studies culture in which investment in evaluation takes Journal of Ageing and Later Life (2015): 7–34. analysed used evidence of significance to place across longer time scales. As the most endorse impact activity in this way. effective examples of change show, evaluation 70 Nicholas Stern, Building • Evidence of significance that communicated is not something that occurs at a set point at the on Success and Learning from Experience: An a pledge to think or act differently in the end of a project, but is a reflexive process that Independent Review of future or a longer-term change in opinion, is more usefully set in motion at the beginning the Research Excellence practice or behaviour was less common, of a project and extends long beyond its end. Framework (2016), p. 23.

The creative role of research 45 5. The creative role of research: final reflections

–5– The creative role of research: final reflections

‘Sounds of Intent’ © Adam Ockelford

46 King’s College London 5. The creative role of research: final reflections

This report has identified the breadth and diversity of ways in which research has flowed into the creative and cultural sector, and highlighted the opportunities to further strengthen these exchanges and more effectively describe their impact.

While our analysis focused on research that causal link between research and impact Work on creative had a primary impact on practice within the reflected in the REF impact submission form, projects supported creative and cultural sector, it was equally and the messy, circular and often fortuitous knowledge exchange on apparent that work on creative and cultural pathways to impact that surfaced in our granular a wide range of topics, projects supported knowledge exchange on analysis of the narrative reporting; presenting from science and politics a wide range of topics, from science and politics impact as having a forward trajectory within to human rights and to human rights and social responsibility. These a prescribed timeframe distorts this reality. Yet social responsibility. impacts did not just originate from research notwithstanding such caveats, REF 2014 is an in the arts and humanities, although strong unprecedented starting point for understanding links in these areas were evident, but stemmed the ways in which researchers and creative from almost every disciplinary area within HE. practitioners have collaborated, and continue This study also found that impact was seldom to work together. There is a clear achieved through a single, focused activity, since The introduction of ‘impact’ to the assessment paradox between each case study included various interdependent of publicly-funded research in the UK, in spite of the simplistic, causal – yet often disparate – pathways to impact, its limitations, has positively incentivised academic link between research creating a network of mechanisms that typically collaboration with the creative and cultural sector, and impact reflected supplemented a central project rather than each and opened up many exciting opportunities for in the REF impact being impactful in its own right. practitioners to engage with new knowledge and submission form, and What the impact assessment in the REF could expertise in an unprecedented way. It has also the messy, circular not, however, capture was the extent to which gained recognition for the quality and social and often fortuitous the creative and cultural sector draws on research importance of practice-led research, which pathways to impact without any direct contact with the researcher. historically has tended to slip through the cracks that surfaced in our The reporting instead tended to be focused on of research assessment practices in UK HE. analysis. proving a proactive and causal link from a piece Gathering impact data on a national scale has of research and the impact claimed. Moreover, also helped to identify sectors and sub-sectors the extent to which research itself evolved through that are not yet fully benefiting from research, the process of collaborative working went largely and where the provision of training and unreported, as did any acknowledgement of networking opportunities in untapped areas may the new knowledge generated through these be beneficial. This study identified broadcast exchanges. These omissions highlight the areas media and cultural policy as two key areas where where impact case studies from REF 2014 are researchers have tended not to take a proactive limited research subjects, since researchers were role in initiating and delivering projects, and not required to report on the reciprocal benefits found that impact projects with local and regional to their work, and are unlikely to be aware of cultural organisations were far outnumbered the full extent to which their work has impacted by projects describing work with a national on the sector indirectly. or international partner. The analysis of time lag between embarking Finally, though we, the authors of this on research and achieving impact was also limited report, have sought to identify frameworks for by the data available. Though this suggests that understanding research impact in the creative research impacts involving the creative and and cultural sector, we are conscious of the risk cultural sector do not have such lengthy time of impact assessment criteria becoming too lags compared to other sectors such as health, prescriptive. The current diversity of research this is based on evidence that tends to capture impact pathways is something to be celebrated only the immediate response to an event, offering and supported rather than curtailed by a desire to little or no evidence of the longevity of the impact. find accurate metrics. We therefore welcome the This lack of long-term perspective was true recommendation proposed by Lord Stern in his across all evidence types, which more commonly independent review of the REF to further broaden endorsed the activity or reflected the potential for the definition of impact. In this way, we hope impact, rather than documenting actual examples that researchers from a wide range of academic of change. disciplines will be further encouraged to foster These limitations already highlight some of opportunities to work with the creative and cultural the issues that should be acknowledged when sector, and that sector professionals will in turn be considering and communicating research impact. inspired by the vast array of ways in which research There is a clear paradox between the simplistic, can inform and develop their policies and practices.

The creative role of research 47 6. References References

6.1 Secondary literature

Department for Culture, Media & Sport. HEFCE. Research Excellence Framework REF 2014 Impact Case Studies. Available ‘Creative Industries Economic Estimates’ Impact Pilot Exercise: Findings of the Expert at impact.ref.ac.uk/CaseStudies/ [last (2016). Available at gov.uk/government/ Panels (2010). Available at ref.ac.uk/pubs/ accessed 7 January 2017]. uploads/system/uploads/attachment_ refimpactpilotexercisefindingsoftheexpert data/file/523024/Creative_Industries_ panels/ Slote Morris, Z., Wooding, S., Grant, J. Economic_Estimates_January_2016_ ‘The Answer is 17 Years, What is the Updated_201605.pdf King’s College London and Digital Question: Understanding Time Lags Science. The Nature, Scale and in Translational Research’. Journal of Greenhalgh, T., Fahy, N. (2015). Beneficiaries of Research Impact: An Initial the Royal Society of Medicine 104, no. ‘Research Impact in the Community- Analysis of Research Excellence Framework 12 (2011): 510–520. doi: 10.1258/ Based Health Services: An Analysis (REF) 2014 Impact Case Studies (2015). jrsm.2011.110180 of 162 Case Studies from the 2014 UK Available at kcl.ac.uk/sspp/policy- Research Excellence Framework’, BMC institute/publications/Analysis-of- Stern, Nicholas. Building on Success and Medicine 13, no. 232 (2015). doi: 10.1186/ REF-impact.pdf Learning from Experience: An Independent s12916-015-0467-4 Review of the Research Excellence Knell, J., Bunting, C., Gilmore, A., Framework (2016). Available at gov.uk/ HEFCE. Assessment Framework and Florack, F., Arvanitis, K., Merriman, government/uploads/system/uploads/ Guidance on Submissions (2011). Available N. Home: Quality Metrics Research & attachment_data/file/541338/ind-16-9- at ref.ac.uk/pubs/2011-02/ Development Report (2015). ref-stern-review.pdf

HEFCE. Panel Criteria and Working National Coordinating Centre for Public Wakeling, K., Clark, J. ‘Beyond Health Methods (2012). Available at ref.ac.uk/ Engagement, Share Academy. (2016). and Well-Being: Transformation, pubs/2012-01/ Museum-University Partnerships in REF Memory and the Virtual in Older People’s Impact Case Studies: A Review. Available Music and Dance’. International Journal HEFCE. REF 2014: The Results (2014). at publicengagement.ac.uk/sites/default/ of Ageing and Later Life (2015): 7–34. Available at ref.ac.uk/pubs/201401/ files/publication/mupi_ref_review.pdf

6.2 REF case studies cited

‘A Scottish Self-Portrait: The Northern ‘Benefitting the Residents of Methodist ‘Discover Turner’s Yorkshire: Lights Documentary Project’, REF 2014 Homes for the Aged through Music Public-Oriented Research and Case Study. Available at impact.ref.ac.uk/ Therapy’, REF 2014 Case Study. Available Commercialisation’, REF 2014 Case CaseStudies/CaseStudy.aspx?Id=24011 at impact.ref.ac.uk/CaseStudies/ Study. Available at impact.ref.ac.uk/ CaseStudy.aspx?Id=43781 CaseStudies/CaseStudy.aspx?Id=6393 ‘Adugna Community Dance Company: Empowering Disabled Dancers and ‘Commemoration and Protection ‘From Scholarly Historical Research Changing Attitudes toward Disabled of Battlefields in Policy and Practice’, to Prize-Winning Popular Fiction – The People through Contemporary Dance in REF 2014 Case Study. Available Thousand Autumns of Jacob de Zoet’, REF Ethiopia’, REF 2014 Case Study. Available at impact.ref.ac.uk/CaseStudies/ 2014 Case Study. Available at impact. at impact.ref.ac.uk/CaseStudies/ CaseStudy.aspx?Id=23304 ref.ac.uk/CaseStudies/CaseStudy. CaseStudy.aspx?Id=4681 aspx?Id=42782 ‘Cultural Policy and Practice Exchanges ‘Anya 17’, REF 2014 Case Study. Available between Britain and Brazil’, REF 2014 ‘Language Policy, Diversity and Usage’, at impact.ref.ac.uk/CaseStudies/ Case Study. Available at impact.ref.ac.uk/ REF 2014 Case Study. Available at impact. CaseStudy.aspx?Id=20912 CaseStudies/CaseStudy.aspx?Id=19052 ref.ac.uk/CaseStudies/CaseStudy. aspx?Id=26214 ‘Applying Computational Reliability ‘Developing the Teaching of Creative Engineering to the Conservation of Writing’, REF 2014 Case Study. Available ‘Latin London: Improving the Visibility Maritime Heritage Structures’, REF 2014 at impact.ref.ac.uk/CaseStudies/ of Latin Americans in the UK’, REF 2014 Case Study. Available at impact.ref.ac.uk/ CaseStudy.aspx?Id=30191 Case Study. Available at impact.ref.ac.uk/ CaseStudies/CaseStudy.aspx?Id=3920 CaseStudies/CaseStudy.aspx?Id=17838

48 King’s College London 6. References

‘Life as Story: The Applied Theatre ‘Rehearsal as Effective Theatrical ‘The Old Bailey Online: Democratising Practice of Nicola McCartney’, REF 2014 Method for Performance: Responding Access to Social History’, REF 2014 Case Study. Available at impact.ref.ac.uk/ to Climate Change’, REF 2014 Case Case Study. Available at impact.ref.ac.uk/ CaseStudies/CaseStudy.aspx?Id=24015 Study. Available at impact.ref.ac.uk/ CaseStudies/CaseStudy.aspx?Id=24011 CaseStudies/CaseStudy.aspx?Id=14678 ‘Madness and Modernity: Mental Illness, ‘Transforming Publics and Participation Psychiatry and the Visual Arts in Vienna ‘Sounds of Intent’, REF 2014 Case through Performance’, REF 2014 Case 1900’, REF 2014 Case Study. Available at Study. Available at impact.ref.ac.uk/ Study. Available at impact.ref.ac.uk/ impact.ref.ac.uk/CaseStudies/CaseStudy. CaseStudies/CaseStudy.aspx?Id=20469 CaseStudies/CaseStudy.aspx?Id=19055 aspx?Id=4678 ‘Stonehenge and its Landscape; ‘UNESCO World Natural Heritage ‘Madness and Modernity: Mental Illness, Changing Perceptions, Informing Status for the Chengjiang (Cambrian) the Visual Arts and Architecture in Fin the Next Generation and Benefitting Fossil Site, Yunnan Province, China’, de Siècle Vienna – Popular Exhibitions the Local Economy’, REF 2014 Case REF 2014 Case Study. Available at impact. in London and Vienna’, REF 2014 Case Study. Available at impact.ref.ac.uk/ ref.ac.uk/CaseStudies/CaseStudy. Study. Available at impact.ref.ac.uk/ CaseStudies/CaseStudy.aspx?Id=12235 aspx?Id=18052 CaseStudies/CaseStudy.aspx?Id=18058 ‘The Act of Killing’, REF 2014 Case ‘Working towards the Führer: Shaping ‘Music in Schools: Boosting Study. Available at impact.ref.ac.uk/ Public Understanding of Nazi Power’, Achievement, Enthusiasm and CaseStudies/CaseStudy.aspx?Id=20542 REF 2014 Case Study. Available at impact. Participation’, REF 2014 Case Study. ref.ac.uk/CaseStudies/CaseStudy. Available at impact.ref.ac.uk/ ‘The Antikythera Mechanism: aspx?Id=12161 CaseStudies/CaseStudy.aspx?Id=44311 Seeing Inside a Two-Thousand- Year-Old Computer’, REF 2014 Case ‘Parabens in Personal Care Products: Study. Available at impact.ref.ac.uk/ Providing Marketing Tools for Industry, CaseStudies/CaseStudy.aspx?Id=3465 Informing European Policy and Raising Public Awareness’, REF 2014 Case ‘The Great Sperm Race – Encouraging Study. Available at impact.ref.ac.uk/ Public Understanding of Human CaseStudies/CaseStudy.aspx?Id=37245 Reproduction’, REF 2014 Case Study. Available at impact.ref.ac.uk/ ‘Performing Migration’, REF 2014 Case CaseStudies/CaseStudy.aspx?Id=12225 Study. Available at impact.ref.ac.uk/ CaseStudies/CaseStudy.aspx?Id=41441 ‘The Horse and East-West History: Generating Cultural and Economic ‘Personalising Asthma Care for Children’, Benefits in Turkey through a UNESCO REF 2014 Case Study. Available at impact. Cultural Route, the Evliya Çelebi Way’, ref.ac.uk/CaseStudies/CaseStudy. REF 2014 Case Study. Available at impact. aspx?Id=39753 ref.ac.uk/CaseStudies/CaseStudy. aspx?Id=930 ‘Professor Rob Kesseler’s Collaboration with the Royal Botanic Gardens, Kew’, ‘The Kindertransport to Britain 1938/39: REF 2014 Case Study. Available at impact. New Perspectives’, REF 2014 Case ref.ac.uk/CaseStudies/CaseStudy. Study. Available at impact.ref.ac.uk/ aspx?Id=40349 CaseStudies/CaseStudy.aspx?Id=44022

‘Red Dust Road: New Forms of Memoir ‘The Military Writing Network: Creative and the Enrichment and Extension of Writing, Life Writing and Trauma’, REF Public Discourse on Family, Identity, 2014 Case Study. Available at impact. Belonging and Adoption’, REF 2014 Case ref.ac.uk/CaseStudies/CaseStudy. Study. Available at impact.ref.ac.uk/ aspx?Id=40629 CaseStudies/CaseStudy.aspx?Id=21725

The creative role of research 49 Appendices Appendices

Appendix 1: Impact in REF 2014

Research assessment in England has ie a form outlining the research strategy, Arts and Humanities. occurred on a near-quinquennial basis current staff and research students, Impact was defined in the REF as since 1986. The Research Excellence levels of income, infrastructure and ‘any effect on, change or benefit to the Framework (REF) is the new system facilities, and degree of collaboration economy, society, culture, public policy or for assessing the quality of research and contribution to the discipline or services, health, the environment or quality in UK Higher Education Institutions research base, weighted at 15 per cent of life, beyond academia’ (REF, 2011). (HEIs). The results inform allocation • the wider impact of research ie in the An impact case study, which formed the of research funding to HEIs, providing form of narrative case studies, weighted basis of the previous work and the analysis accountability for public investment in at 20 per cent. conducted in this report, is a short four- research and contributing to reputational page document with five sections: excellence of HEIs. HEIs made submissions to 36 Units 1. summary of the impact In REF 2014, HEIs were assessed by: of Assessment (UOAs), which were 2. a description of the underpinning • the quality of research outputs, ie a list evaluated by expert panels working under research of publications, weighted at 65 per cent the guidance of four panels: Biological 3. references to the research of the total assessment Sciences and Medicine, Physical Sciences 4. details of the impact • the vitality of the research environment, and Engineering, Social Sciences, and 5. sources to corroborate the impact.

Appendix 2: Coding in NVivo

The qualitative analysis underpinning capturing only the information presented case studies. This generated the main our findings was based on a sample of under the ‘Details of the impact’, with pillars of analysis and some top-level 417 case studies. We began with a sample the exception of the date of originating categories. Sub-categories underneath of 432 case studies, which included 48 research, for which we captured the these headings were then generated case studies from each of the nine impact earliest date cited in the ‘Description while reading, driven by the content topics identified as relevant to the sector of the underpinning research’ or the of the case studies. We began capturing – 16 where the topic was identified as ‘References to the research’: this information in a granular way, the most relevant to the case study’s • impact type later consolidating or subcategorising narrative, 16 where it was the second • sector partners and funders similar nodes as patterns emerged. most relevant, and 16 where it • research engagement These groupings and their labels was the third. All of these case studies • impact mechanism or activity further evolved through consultation were read and analysed in NVivo, • beneficiaries among the three authors at four stages software for qualitative data analysis. • measurement type throughout the analysis process, with a During the analysis we identified and • geographical reach comprehensive review after the analysis excluded 15 false positives, leaving • point of impact of all 417 case studies was complete. a final sample of 417 case studies. • sector We did not make value judgements NVivo enables the bespoke generation • date of originating research. about the strength of the evidence, of nodes to code and analyse passages of but sought to capture objectively the text to identify trends and relationships Coding was an iterative process. To information supplied by the researcher in large datasets. We tagged and collated identify the above themes for analysis, without querying whether it was the following types of information, we undertook a pilot analysis of 20 accurate or authoritative.

Appendix 3: Topic model labels and associated words

Arts and culture – art artist work cultur Historical archives – histori archiv public Museums and exhibitions – exhibit museum creativ project public audienc exhibit histor project librari heritag cultur materi visitor art galleri collect curat displai public Cultural and heritage preservation – Literature – book read poetri write literari Music, dance and performance – music heritag archaeolog site visitor histor writer publish literatur translat perform danc work sound audienc concert museum project cultur tourism record festiv Media – public bbc media radio programm Film and theatre – film theatr perform plai interview time broadcast articl Print media and publishing – univers book audienc product festiv screen director intern translat world publish de public uk

50 King’s College London Appendices

‘Discover Turner’s Yorkshire’ © Welcome to Yorkshire

The creative role of research 51 The creative role of research

2017

Culture Cover: Professor Rob Kesseler’s collaboration with the Royal Botanic Gardens, Kew. © Calotis breviradiata. Burr rayed daisy fruit. Hand coloured micrograph. From Fruit, edible, inedible, incredible. Kesseler & Stuppy. Publ. Papadakis kcl.ac.uk/culture @CulturalKings Design: RF Design, www.rfportfolio.com Approved by [email protected], October 2017