Peer Review College Newsletter Summer 2013 Introduction CONTENTS The feature article in this issue looks at the results of the College Member General 1. EPSRC Peer Review Survey for 2013, which many of you were able to complete. We hope you will find College Member General the results interesting. We are also currently undertaking a refresh of the College Survey 2013 – page 1 membership, and there is an article explaining what this involves. We are expecting the updated College On-Line Training to be available very shortly and give some further 2. College refresh information about this. A list of new members appointed to the College since the last community nomination College Refresh is included. process – page 6 3. Update to College Online EPSRC Peer Review College Member General Survey Training – page 6 2013 4. Membership Matters – page 7 Since the last College Newsletter we approached all College members to take part in an on-line survey which covered a range of topics including the peer review process, 5. Editors End Piece – college management, and college communications. page 10 Thank you very much for your excellent response – of the 3800 College members contacted 1149 members completed the survey, which included 128 new members. For both new and existing members this is about 1/3rd of all members, which is an excellent response rate. We greatly appreciate you making the time to provide this information, which we will use to inform and improve our operational arrangements for the college and our engagement with College members. The information provided indicates (as shown in the charts below) a good spread of experience both in terms of length of service on the college and in terms of the numbers of reviews undertaken to date. In addition about half of all respondents have attended a prioritisation panel during their time on the college. Peer Review College Newsletter Summary The majority of College members were satisfied in general with their relationship with EPSRC, although some would like greater opportunities to engage with EPSRC, via an organised event or similar. There was very positive feedback on most aspects of college operation and about the guidance provided to support peer review activity, with the great majority of respondents very supportive of the arrangements in place. This gives us a lot of confidence in moving forward for the future. There was some very useful feedback concerning the Annual Activity Reports and, from new members, on the recently implemented process for appointing members to the college, which we will use to refine and improve these processes. Results in Detail Relationship with EPSRC. Almost 75% of members felt that their relationship had been sustained at a good level or had improved. For those who were less happy the overwhelming majority identified the loss of opportunity for engagement with EPSRC as their main cause of concern. This came in a variety of forms. Reduced opportunities for direct engagement with EPSRC staff, and for college members to meet together and interact, were particularly identified. We will investigate how we can improve interactions of this type, and will seek further feedback from members over the coming months as to the preferred methods to achieve this. A major concern, raised through this survey and also increasingly by direct contact with us, relates to the reducing opportunity to serve on prioritisation panels. Unfortunately there is little we can do about this. The reality is that we are operating fewer panels so need fewer panel members each year. We have prepared a separate article on this subject to provide some of the facts and figures, and to show that opportunity to serve on panels is still being provided on a fair and equitable basis. One major change that could have impacted adversely in this respect was the transfer of much grant processing activity into the RCUK Shared Services Centre (SSC), meaning that there was far less direct contact with EPSRC within the reviewing process. Responses to the survey show this has not been the experience. Only 11% of respondents felt that the involvement of SSC had weakened the process in that respect. Analysis of this shows that not all of this was in fact about the SSC, as a significant element actually referred to the increasing amount of automated processing and responses in the new grants processing system implemented at the same time. 2 Peer Review College Newsletter New Members For new College members, the motivation to become members arose for a variety of reasons, including the fact of being nominated by colleagues, community spirit, and a desire to influence the future. Most (over 95%) found the appointment process acceptable or better. 48 members provided specific comments suggesting the welcome process had felt rather impersonal and that they would have welcomed more information about the appointment process and greater guidance about what is expected of new College members. We have made changes to improve this process accordingly. Activity Reports Two thirds of respondents who were existing College members found the annual activity reports useful. The ability for members to compare themselves with their peers was identified as the most significant benefit, and a third of all those commenting indicated that all aspects of the current report should be retained. Generally there were few detailed comments, but one significant concern was with the accuracy of the data. Although we believe the data is correct, we do accept that it is quite complex and there have been problems with how it has been presented, causing questions about its validity. We have identified ways to improve the clarity of the data and we will use these in future reports. College Newsletter We were very pleased to find that two thirds of respondents not only read the college newsletter but found it interesting and/or useful. Based on comments we received we will in future include links to the newsletter, and to the College Members page on our website, as standard in all our emails to College members. Comments on the newsletter were wide and diverse with no really clear messages for improvement. We will however seek to continue to improve it and would welcome suggestions of topics you would find it useful or of interest for us to cover. Training 25% of existing members who responded had completed on-line training, although currently we have asked new members to wait until the new version of the training is available. College members who have completed the training made mainly positive comments. 3 Peer Review College Newsletter When asked why they had not done the on-line training, the members’ main replies were: a) Not aware of the on-line training b) Have done alternative training previously c) Do not need training, is an experienced reviewer Many members stated they would do the training if instructed to do so, or if the benefits were explained. Reviewing Both existing and new College members were asked about panel attendance, including what was provided in terms of guidance, paperwork, and EPSRC/SSC support. A more detailed report will be included in a future edition of the College Newsletter, but the vast majority of panel members found the Panel Member Guidance helpful and informative and the paperwork at the correct level, with over 90% in both groups finding the EPSRC support appropriate. Results from Existing College Members I feel that I am provided with appropriate support from EPSRC and/or the SSC when I attend a panel meeting 97.7% 520 2.3% 12 Were you aware of the help text available on Je-S? Yes - if so how useful 84.7% 798 was the help text?: No: 15.3% 144 On a scale of 1 to 5 how useful was the help text? 1 - not at all useful: 2.5% 20 2: 12.4% 99 3: 43.1% 344 4: 35.8% 286 5 - Extremely useful: 6.1% 49 Were you aware of the reviewer guidance on the EPSRC website? Yes - if so how useful 91.4% 861 was the guidance?: No: 8.6% 81 On a scale of 1 to 5 how useful was the guidance? 1 - not at all useful: 2.1% 18 2: 10.5% 90 3: 41.6% 358 4: 38.2% 329 5 - extremely useful: 7.7% 66 Is there another way you would prefer to access guidance on reviewing? Yes: 5.6% 47 No: 94.4% 799 4 Peer Review College Newsletter Conclusions We conducted this survey in recognition of the changes we have made to the College and Peer Review process which we have hoped have benefitted College members. In general the feedback has been very supportive and this encourages us to build on this foundation and progress with a program of continuous improvement. Issues which have been identified are mainly concerned with mutual engagement and the level of panel attendance available. We shall look to enhance EPSRC engagement with College members over the next 12 months and hope that you will be willing to help when we approach members on an individual basis. Panel Attendance Feedback from the college survey and in response to last year’s annual activity reports has highlighted growing concern about lack of opportunity to attend panels. Most people joining the college will have done so with a clear expectation that they will participate in prioritisation panel meetings, and experience of long standing members will be that this was a more regular part of college activity. Unfortunately, over the years the number of panels we operate has declined and the number of opportunities to attend panels has reduced accordingly. Over the last calendar year for example we only required 640 panel members all told.