The Funding and Management Performance Evaluation of NSFC
Yonghe Zheng Deputy Director, Bureau of Planning National Natural Science Foundation of China Beijing, Oct. 10, 2012 1. A Glance at China R&D and Performance Evaluation;
2. A Glance at NSFC Funding and Management;
3. The funding and management performance evaluation of NSFC. R&D Expenditures for US, EU, and Asia 1996-2007
Source: National Science Foundation 3 Average Annual Growth of R&D Expenditure for US, EU-27, Asia-8
Source: National Science Foundation 4 2011 Global R&D Expenditures
The world as a whole will spend over $1.4T on R&D in 2012 In 2011 China spent more on R&D than Japan, at $175B vs. $152B (PPP USD). In 2011, India’s investments in R&D were roughly equivalent to the UK’s, at around $38B annually. U.S. R&D spending was up 2.9% over prior year to $427B Some further research output data
Citable Documents
457,642
315,768
36,055
13,189
Chinese Taiwan
SCImago. (2007). SJR — SCImago Journal & Country Rank. Retrieved March 08, 2012, from http://www.scimagojr.com 6 Growth of R&D in China Development of Basic Research Channel The national evaluation effort in China and the performance evaluation of NSFC
Currently, it has become an urgent need for Chinese governments to establish an evaluation system within the public administration system in China. The whole – of - government performance evaluation system is establishing in China now, the performance evaluation for ministry in whole is needed in the budget application each year since 2011.
MoF is the leading body Ministry of Finance (MoF) is the leading body of the performance evaluation of public sectors in China. There are already some pilot activities of performance evaluation, which are carried out by line ministries or local governments in China. NSFC funding and management performance evaluation is the most successful case which obtains comments from Prime Minister Weng Jiabao and State Councilor Liu Yandong.
1. A Glance at China R&D and Performance Evaluation;
2. A Glance at NSFC Funding and Management;
3. The funding and management performance evaluation of NSFC. Mission
Support basic research
Identify and foster talented researchers
Strengthen international cooperation
Promote socioeconomic development
Total Amount and Relative Share of Basic Research Funding Budget: NSFC budget has been increased at an annual rate of over 20%. The total budget for 2012 is 15 billion CNY.
Proposals Received
No.ofProposals Received Project Evaluation Process
Applicants
Applicants’ Org Divisions
Mail Review
18~20% <30% NSFC Approval Panel Review Funding Scale Unit: RMB
1. General Project 800 K / 4 yrs 2. Key Project 3 million / 5 yrs 3. Young Scientist Fund 250 K / 3 yrs 4. Creative Research Group 6 million / 3 yrs 5. Less Developed Region 500 K / 4 yrs 6. Distinguished Young Scholars 2 million / 4 yrs 7. Major Intl. Joint Research Project 3 million / 5 yrs 1. A Glance at China R&D and Performance Evaluation;
2. A Glance at NSFC Funding and Management;
3. The funding and management performance evaluation of NSFC. The Funding and Management Performance Evaluation of NSFC
Challenges Objectives Approach Modility International Evaluation Committee Organization Dimensions Issues-Questions-Evidents Implimentation Report Releasing Use of Evaluation
Challenges
25 years history, the quality and accuracy of data achieved; Funding and Management System is complicated; This Comprehensive and Strategic Evaluation is the first practice in China; International Evaluation
The evaluation is commissioned by MoF and NSFC together. The State Council attached importance to this evaluation. The NSFC case is the most influential evaluation in public sectors in China up to now and represents good practice from China.
Dual Objectives of the Evaluation
■ Accountability Provide an independent assessment of the overall performance of NSFC’s funding and management during the past 25 years
■ Lesson learning Improve the performance of NSFC’s funding and management ; redefine NSFC’s strategic role within the NIS of China
22 What does the evaluation focus on and what does the evaluation not attempt to do
The evaluation does The evaluation does NOT
■Focus on NSFC’s funding Assess NSFC from the and management agency’s internal perspective performance over 25 years
■Focus on NSFC’s overall Assess individual projects performance as a whole supported by NSFC
23 Evaluation Approach and Principals
Modality: Domestic preparation followed by international evaluation Approach: Evidence-based Principals: Independent Objective Modality of the Evaluation
■The Evaluation adopts an approach of “domestic preparation followed by international evaluation”. ■Domestic Preparation NCSTE, an independent and professional evaluation organization, design the evaluation and prepare evidence needed, with the support of NSFC. ■ International Evaluation International Evaluation Committee (IEC) provide evaluation conclusions and recommendations based on the evidence prepared by NCSTE and its own investigations and international comparisons
25 Organization of the Evaluation ■The Leading Group (LG) Chairmen: Vice Minister of MoF, President of NSFC ■Working Group (WG) and Evaluation Office (EO) Responsibility: Supporting NCSTE, IEC
■The International Evaluation Committee (IEC) Responsibility: Making evaluation, Completing IEC report
■The National Center for S&T Evaluation (NCSTE) Responsibility: Designing and organizing evaluation activities, Completing Evidence Report ■NSFC Responsibility: Providing funding and management information and materials, attending the interview activities, case study
26 International Evaluation Committee Prof Richard N ZARE (Chair), Stanford University, Chemistry Prof HAN Qide (Vice Chair), Vice Chairman, National People’s Congress, Medicine Prof Ernst-Ludwig WINNACKER (Vice Chair), Human Frontier Science Programme , Biochemistry Prof Erik ARNOLD (Rapporteur), Technopolis Group; University of Twente, Research and Innovation policy Prof LU Yonglong (Rapporteur), Research Center for Eco-Environmental Sciences, Chinese Academy of Sciences, Environmental Science and Management Prof XUE Lan (Rapporteur), Tsinghua University, Science and Technology Policy and Management Prof Akito ARIMA, Chairperson, Japan Science Foundation, Nuclear physics Dr Richard A ANTHES, University Corporation for Atmospheric Research, Atmospheric Science Prof Anthony K CHEETHAM, University of Cambridge, Materials Science Prof MA Zhiming, Academy of Mathematics and Systems Science, Chinese Academy of Science, Mathematics Prof Andrew (FA) SMITH, University of Adelaide, Agriculture, Food and Wine Prof Jeannette M WING, Carnegie Mellon University, Computer and Information Sciences Prof XU Zhihong, Peking University, Life Sciences
International Evaluation Committee members with State Councilor Liu Yandong The scope of the Evaluation covers four dimensions:
(1) NSFC’s strategic positioning, (2) funding performance, (3) management performance, (4) impact on China’s S&T system. The delegation from NSFC discussesd Evaluation Plan with IEC Chair in Stanford，2009.5 IEC Chair discussed TOR with NSFC and NCSTE in Beijing, 2009.6
10 Key Issues in ToR
□Strategic Issue 1. NSFC’s strategic positioning in the national Positioning innovation system of China Issue 2. Funding strategy □Funding Issue 3. Contributions to ‘original innovation’ Performance Issue 4. Promoting the balanced, coordinated and sustainable development of scientific disciplines Issue 5. Fostering innovative talents Issue 6. Supporting the national demands and challenges □Management Issue 7. Modality of funding management performance Issue 8. Funding instruments Issue 9. Peer Review System □Impact Issue 10. Impact of NSFC’s funding
32 Issue 1 NSFC’s strategic positioning in the national innovation system of China
Main Evaluation Questions and Related Evidence
Q1 Has NSFC’s strategic positioning evolved in response to the significant changes in China during the past 25 years? How much relevant is NSFC’s strategic positioning to national science and technology policy and priority needs? Q2 What kinds of roles does NSFC play in the National Innovation System (NIS), especially in the basic research system during the 25 years? How does NSFC co-ordinate its activities with the relevant actors in China’s basic research funding system and in what way is NSFC’s funding linked with other national S&T plans? Q3 How has NSFC responded to the challenges of the NIS of China? Are there any missed opportunities in NSFC’s strategic positioning and does it need to be redefined in the future?
33 Issue 1 NSFC’s strategic positioning in the NIS of China
Main Evaluation Questions and Related Evidence
Q1 Has NSFC’s strategic positioning undergone any evolution in responding to significant changes in China during the past 25 years? How much relevant is NSFC’s strategic positioning to national policy and priority needs of China?
Evidence ■ Evolution of NSFC’s guiding principles, strategic goals and priorities to allocate funds ■ The relevance of the priority objectives of NSFC’s Strategic Plans with those of National Strategic Plans on basic research ■ Perspectives of government officials, scientists as well as present and former leaders of NSFC on this question. （Including: agreed viewpoints and controversial content）
34 Implementation of the Evaluation Phase Major Evaluation Activities Phase1 Inception and evaluation design 2010. 2－2010. 6 Phase2 Prepare evaluation evidence report 2010. 7－2010.12 The IEC first meeting in Beijing
Phase3 Complete the Evaluation Report 2011. 1－2011. 6 Phase4 Use and dissemination of the evaluation 2011年7－10月 results
35 IEC Evaluation Acitivities in Beijing, December, 2010 After International Evaluation, Chairs of IEC published Editorial in SCIENCE and released the Evaluation Report to public on the web. NSFC: Response and Dissemination
Take management response to the evaluation report at the draft stage Distribute the evaluation report to line ministries, the National People's Congress, the Chinese People's Political Consultative Conference Arranging dissemination events of the evaluation. The Use of Evaluation: NSFC Actions
NSFC hosted leading level and director level meeting for whole one week to discuss the evaluation report, and some recommendations were adopted: the project term of General Program changed from 3 years to 4 years; established Excellent Young Scientist Program; established Continue Program for YSP-GP; extended the scale of Less Developed Region Program; enhance the female funding in different level; enhance the internationl funding; ...... Thank you!