D B F
20/21 February 2013 European Research Council Executive Agency Covent Garden, Place Rogier 16, Brussels Ground floor – AUDITORIUM (SDR2)
Identification of "frontier research" and "emerging research areas" in research proposals: A bibliometric approach
Final workshop of the DBF & ERACEP Coordination and Support Actions
The basic mission of European Research Council (ERC) is "...to stimulate scientific excellence by supporting and encouraging the very best, truly creative scientists, scholars and engineers to be adventurous and take risks in their research". Over the last three years ERC has funded two ambitious projects:
• Development and Verification of a Bibliometric model for the Identification of Frontier Research (DBF) • Emerging Research Areas and their Coverage by ERC-supported projects (ERACEP)
… that have looked at the ways in which bibliometrics could be used to measure the extent to which ERC is reaching this mission. An important prerequisite for such an approach is the availability of suitable bibliometrics techniques for identifying "emerging research topics" and "frontiers of research". These were developed by the CSA projects and they will be presented and demonstrated during the workshop. Their potential to monitor the ERC peer review process and to assist the selection of research proposals will also be discussed.
Agenda
Day 1 Identification of "frontier research" and "emerging research areas" through bibliometrics
Opening
09:30 – Welcome and Introduction: professor Reinhilde Veugelers, ERC Scientific Council, 10:15 Chair of the working group on ERC key performance indicators
Setting the scene: Scientometrics in ERC monitoring and evaluation framework; Theodore Papazoglou, ERCEA, Head of Support to the Scientific Council
Morning Chair: William Cannel; Imperial College London, former advisor of EC in support of session scientific excellence in Europe, managed the project of establishing ERC
10:15 – DBF: Capturing dimensions of frontier research with bibliometrics; Katy Whitelegg 11:00 AIT, Edgar Schiebel AIT, and Ivana Roche INIST-CNRS
This session will provide an insight into concepts and the implementation of the DBF project: definition of "frontier research" by the EC’s High Level Expert Group, the conceptualisation of five indicators for identifying frontier research: Timeliness, Innovativeness, Pasteresqueness, Interdisciplinarity, and Risk; as well as development of bibliometric techniques that can be used to measure each of them.
11:00- Coffee break 11:30
11:30 – ERACEP: Identifying Emerging Research Areas with bibliometrics; Wolfgang 12:15 Glänzel, KU Leuven - ECOOM
Most bibliometric analyses with regard to identifying "emerging research areas" are based on different citation methods. These different methods will be presented in parallel with ERACEP specific approach that was developed for capturing and mapping emerging research areas in ERC-funded research activities.
12:15 – Open floor discussion: 13:00 To what extent can "frontier" and "emerging" research be identified in research proposals using bibliometrics?
13:00– Lunch break: 14:00 Restaurant/Canteen: open on the 1st floor of the ERCEA building
2
Afternoon Chair: Jose Labastida; ERCEA, Head of Scientific management department session
14:00 – DBF: Measuring frontier research in the ERC research proposals: Effect on 14:30 selection outcome; Thomas Scherngell AIT
This session will show application of the five bibliometric indicators developed by DBF to predict ERC proposal evaluation outcome. ERC selection process will be simulated by econometric model based on 5 composite bibliometric indicators of frontier research and results will be scrutinized in comparison with the selection of the ERC peer review.
14:30 – ERACEP: Identifying emerging research areas in the ERC research proposals: 15:00 Effect on selection outcome; Thomas Reiss, ISI Fraunhofer
This session will show how ERC proposals can be matched against the emerging research areas identified by the bibliometric approach elaborated within ERACEP. Matching approach will be validated with qualitative expert based analyses. Results will be compared against ERC peer review selection outcome.
15:00– Open floor discussion: 15:30 How can bibliometrics techniques be used in funding decision process?
15:30– Coffee break 16:00
16:00 – Round table: Implications of DBF and ERACEP findings for the further 17:30 development of ERC peer-review selection process?
Facilitator: Katy Whitelegg, coordinator of the DBF project
The experts are expected to draw on their expertise and scrutinize the presented outcomes. The question what can bibliometrics and scientometrics offer a peer review process will be addressed. The issue of the use of bibliometric tools for pre- filtering of research proposals will be discussed.
17:30 End of day 1
19:30 WORKSHOP DINNER:
Brasserie Le Saint Germain; Place Rogier, 1 - 1210 Bruxelles
3
Day 2 Hands-on bibliometrics tools in support of research funding organisations
Morning Chair: Thomas Konig, scientific advisor to the ERC president Helga Nowotny session
09:30 – Demonstration session ERCEA: Insight into ERC research information system, 10:00 Alexis Michel MUGABUSHAKA, ERC Executive Agency – Support to the scientific council
This session will show the ERC efforts to account for the added-value of its funding activities by demonstrating ERC research information system conceived as an integrated platform which combines several tools to support the gathering, management and analysis of data on ERC submitted research proposals. Its potential for integration of new bibliometric tools will be pointed out.
09:30 – Demonstration session DBF: Bibliographic measurement of interdisciplinarity in 10:00 research proposal; Edgar Schiebel, AIT
This session looks in more detail at the interdisciplinarity indicator of the DBF project. It will show details on the method and tools that can be used in practice to measure the level of interdisciplinarity in research proposals.
10:00 – Demonstration session ERACEP: Bibliographic measurement of scientific novelty 10:30 in research proposal; Wolfgang Glänzel and Bart Thijs, KU Leuven - ECOOM
Based on emerging research topics identified with ERACEP specific bibliographic approach a practical example of estimating scientific novelty in ERC research proposals will be demonstrated for selected scientific fields. Advantages of the approach but also limitations will be pointed out.
11:00 – Coffee break 11:30
11:30 – Round table discussions: Challenges for implementation of DBF and ERACEP 13:00 developed bibliometric tools into a business process of research funding agency
Facilitator: Thomas Reiss, coordinator of the ERACEP project
This session will try to close the gap between theoretical potential of bibliometric techniques to assess research proposals on one side and application of these in day-to-day business of a research funding agency on the other. DBF and ERACEP project teams, experts in scientometrics and ERC scientific officers overseeing proposal evaluation process will step together to discuss the opportunities for implementation of demonstrated bibliometric techniques into a business process of ERC executive agency.
13:00 Wrap-up session:
Final comments and conclusion
13:30 End of workshop
4