Final report, 2016-07-01 User survey and impact assessment of the Norwegian social science institutes Tobias Fridholm, Tomas Åström, Emma Ärenman and Lena Johansson de Château www.technopolis-group.com User survey and impact assessment of the Norwegian social science institutes technopolis |group| 1 July 2016 Tobias Fridholm, Tomas Åström, Emma Ärenman and Lena Johansson de Château Table of Contents Summary ........................................................................................................................................................ 1 1 Introduction ............................................................................................................................................ 5 1.1 Evaluation of the Norwegian social science research institutes ................................................................... 5 1.2 Supporting documentation for the evaluation............................................................................................... 7 1.3 The user survey and impact assessment ........................................................................................................ 7 1.4 Definitions ....................................................................................................................................................... 8 1.5 Method and empirical data............................................................................................................................. 9 1.6 Report structure ............................................................................................................................................ 15 2 User survey ........................................................................................................................................... 16 2.1 Motives for collaboration .............................................................................................................................. 16 2.2 Competence ................................................................................................................................................... 19 2.3 Management.................................................................................................................................................. 26 2.4 Competitiveness ........................................................................................................................................... 30 3 Impact assessment ............................................................................................................................... 35 3.1 Overview of impact cases .............................................................................................................................. 35 3.2 Impact cases ................................................................................................................................................. 40 3.3 Impact from the users’ point of view ........................................................................................................... 48 3.4 Impact typology ............................................................................................................................................ 53 4 Conclusions ........................................................................................................................................... 56 4.1 Users’ assessment ......................................................................................................................................... 56 4.2 Impact assessment ........................................................................................................................................ 58 4.3 Concluding remarks ...................................................................................................................................... 59 Appendix A Impact case studies .................................................................................................................60 Appendix B Classification of impact cases ................................................................................................. 83 Appendix C Interview guide ........................................................................................................................ 86 Appendix D Survey questionnaires ............................................................................................................. 93 i Summary The assignment reported herein was tasked with investigating the views of the users of the Norwegian social science institutes, and with providing an assessment of the societal impact of the institutes’ R&D. For the purposes of the assignment, users include both clients buying commercial services from the institutes and partners collaborating with the institutes in publically co-funded R&D projects. The survey is part of the background material for an evaluation of the social science institutes that is conducted by a Scandinavian panel of experts appointed by the Research Council of Norway (RCN). The assignment was carried out by Technopolis Group between January and July 2016. The social science institutes The social science institute sector, which comprises 22 institutes, is quite diverse. The institutes encompasses topics such as labour and social policy, health, welfare, education, industry, development, regional development, foreign policy, international conflicts, and global development. Some institutes have a profile towards basic research while others mainly conduct applied research, and the types of problems and methodological approaches varies significantly between institutes. However, they all have in common that organisations in the public sector are the most important sources of funding – either through funding of R&D projects from RCN or the European Commission, or as direct clients. To rationally deal with an immense amount of empirical data, we have grouped the institutes into three groups based on an overall assessment of their main characteristics: Four internationally oriented institutes Nine regionally anchored institutes Nine institutes in welfare and society Method and empirical data User survey The user survey was conducted through a web survey and through interviews. The web survey generated 620 responses, corresponding to a response rate of 25 percent. Almost four in five survey respondents are partners, and two out of three are based in Norway. More than half of respondents represent higher education institutions (HEIs) and research institutes (other than the social science institutes), and slightly more than a quarter represent the national, regional or local public sector. The survey response rates differ quite a bit between institutes, from 59 percent down to a mere 9 percent. We have conducted 78 telephone interviews, but since some interviews – mainly with key users in the national public sector – concerned more than one institute, the interviews encompass a total of 96 user–institute relationships. Slightly more than half of the interviewees represent the national, regional or local public sector, and a mere 6 percent are foreign, plainly due to very few institutes having very many foreign users that are not academic research partners (with presumed limited insight into the Norwegian institutes as such). Impact assessment The impact assessment was predominantly based on 71 impact cases submitted by the institutes at RCN’s request. RCN asked for societal impact, defined as any impact except impact on other R&D and impact on the institute’s own organisation. The impact assessment was inspired by the British Research Excellence Framework from 2014 (REF2014). For 15 impact cases, we engaged in additional data collection through document studies and interviews to be able to complement and elaborate the cases further, as well as to verify the information provided by the institutes. As a complement to the impact cases, we also used the web survey to collect data on user impact from working with the institutes. User survey and impact assessment of the Norwegian social science institutes 1 User survey The users of the institutes agree that accessing expertise is the most important motive for collaboration. This concerns subject-specific as well as methodological expertise, and in the case of regionally anchored institutes, region-specific knowledge. A clear majority of users are highly satisfied with the scientific and methodological competence of the institutes. This is true for all institute groups and all individual institutes, with the highest ratings going to the internationally oriented institutes, while the regionally anchored institutes lag behind in this respect. Users generally give lower scores for relevance of services than for scientific expertise. One reason for this may be that most users are in a better position to assess relevance and usefulness than scientific expertise, and therefore have higher demands. However, some users expect R&D services with less academic emphasis than they presently receive, meaning more adapted to their specific needs. Users see several reasons why the R&D results from the institutes are not as useful as they could be. Some reason that the institutes to a certain extent lack staff with experience from working in for example public agencies or companies. Others users argue that (small) institutes should cooperate more with other actors to make up for their size- and capacity-related weaknesses. Overall, clients give lower ratings to the institutes’