Transformative Research – An Exploration of Six Propositions By Bhavya Lal S.B. in Nuclear Engineering, June 1990, Massachusetts Institute of Technology S.M. in Nuclear Engineering, June 1990, Massachusetts Institute of Technology S.M. in Technology and Policy, June 1992, Massachusetts Institute of Technology A Dissertation Submitted to The Faculty of The Columbian College of Arts and Sciences of The George Washington University in partial fulfillment of the requirements for the degree of Doctor of Philosophy August 31, 2012 Dissertation directed by Nicholas Vonortas Professor of Economics and International Affairs The Columbian College of Arts and Sciences of The George Washington University certifies that Bhavya Lal has passed the Final Examination for the degree of Doctor of Philosophy as of July 3, 2012. This is the final and approved form of the dissertation. Transformative Research – An Exploration of Six Propositions Bhavya Lal Dissertation Research Committee: Nicholas Vonortas, Professor of Economics and International Affairs, Dissertation Director Kathryn Newcomer, Professor of Public Policy and Public Administration Committee Member Scott Pace, Professor of the Practice of International Affairs, Committee Member ii To Kinjal iii Acknowledgements My greatest gratitude is reserved for my advisor and dissertation chair Nicholas Vonortas for seeing the potential of this topic, and giving me the opportunity to work on it. I am grateful to him not only for the confidence he showed in me, and for the incalculable ways he stretched me intellectually, but also for the clear direction I got whenever I drifted too far off course. I thank my committee members Scott Pace and Kathryn Newcomer, for pushing me to defend every statement, and in so doing, sharpening my work and preparing me for any audience. Thanks also go to my readers David Bray and Valerie Schneider for their guidance and advice, and thorough readings of many versions of this document, even the early ghastly ones. I would like to acknowledge the contributions of my colleagues at the Science and Technology Policy Institute (STPI), especially Alyson Wilson, Stephanie Shipp, Mary Beth Hughes, Mario Nunez, Seth Jonas, Amy Marshall, Elizabeth Lee, Asha Balakrishnan and Vanessa Pena, who were alongside me in our many high-risk high-reward adventures, and who guided me in the completion of the dissertation. They were my thought partners in every way. Above all, they provided me with an intellectual home at STPI. Last but not least, I would like to thank President David Chu and the leadership of the Institute for Defense Analyses (IDA) and the Science and Technology Policy Institute (STPI), for not letting me slow the momentum of this work, and the IDA Scholar program, that allowed me to start and complete the doctoral program while still working full time and raising a family iv Abstract of Dissertation Transformative Research – An Exploration of Six Propositions U.S. federal programs that fund transformative research have proliferated in recent years, with both Congress and the Administration urging science agencies to fund more of it. However, there appears to be no firm understanding of the concept. Its definitions tend to be inspirational but vague. More importantly, there is no operational agreement on how to identify, fund, and evaluate such research. This dissertation attempted to bridge this knowledge gap, and examined transformative research using qualitative case study, quantitative analysis, and text mining methods. Building on two reviews, the literature in the interdisciplinary field of the science of science policy, and federal programs that support transformative research, I developed six propositions - three about those who conduct transformative research, and three about transformative research itself. The propositions were then explored using data from a transformative research program created at the National Institutes of Health. Findings from the exploratory analysis showed that projects supported by transformative research is neither more interdisciplinary, nor are its performers younger, more productive, or more distinctive in their track records. Moreover, while transformative research is perceived to be risky, riskiness of proposals does not appear strongly associated with transformative outcomes. The only distinguishing characteristic of transformative research is the level of disagreement among peers reviewing it. The findings make intuitive sense, and may not appear new or unexpected. However what makes them different is that they are supported by data, a feature that the current literature on transformative research lacks. The study has limitations, the principal ones being that the findings were based on basic research in biomedical sciences, and no attempt was made to generalize to other research fields. Two potential policy implications emerge assuming that the propositions are tested more broadly: to achieve transformative outcomes, research programs may be better served by funding high-quality researchers to conduct high-quality research, rather than trying to identify potentially transformative ideas. In addition, extra attention could be paid to research proposals with divergent peer rankings, as this divergence may suggest the kernel of a paradigm-shifting idea. v Table of Contents Dedication ............................................................................................................................................ iii Acknowledgements ............................................................................................................................. iv Abstract of Dissertation ...................................................................................................................... v Table of Contents ............................................................................................................................... vi List of Figures ...................................................................................................................................... ix List of Tables ....................................................................................................................................... xi 1.0 Introduction .................................................................................................................................. 1 1.1 Background ........................................................................................................................... 1 1.2 What is “Transformative Research”? ................................................................................ 2 1.3 Origin of the Concept in S&T Policy ............................................................................... 4 1.4 The Policy Challenge ........................................................................................................... 9 1.5 Research Questions and Overall Approach ................................................................... 10 2.0 Defining Transformative Research and Related Concepts .......................................... 12 2.1 Defining Transformative Research .................................................................................. 12 2.2 Key Characteristics ............................................................................................................. 20 2.3 Operationalizing Transformative Research .................................................................... 26 2.4 Operationalizing High-Risk Research ............................................................................. 39 2.5 Conclusion .......................................................................................................................... 41 3.0 Review of Programs Funding Potentially Transformative Research ........................ 43 3.1 Methodology ....................................................................................................................... 43 3.2 Overview of Programs ...................................................................................................... 46 3.3 Definitions of Terms of Interest ...................................................................................... 51 3.4 Program Origins ................................................................................................................. 53 vi 3.5 Areas of Funding ................................................................................................................ 54 3.6 Competitiveness of Programs – Funding Rates ............................................................ 55 3.7 Timeline and Renewals ...................................................................................................... 58 3.8 Selection Criteria ................................................................................................................ 60 3.9 Post-Award Support .......................................................................................................... 63 3.10 Types of Programs Found – By Program Goals ......................................................... 63 3.11 Types of Programs Found –By Management Strategies ............................................ 66 3.12 Types of Programs Found – by Approach to Research ............................................. 71 3.13 Conclusion ........................................................................................................................ 72 4.0 Introducing Propositions ......................................................................................................