Energy I-Corps Program: Year 1 Process and Impact Evaluation
Total Page:16
File Type:pdf, Size:1020Kb
Energy I-Corps Program: Year 1 Process and Impact Evaluation Final Report November 28, 2016 Gretchen Jordan, Ph.D. Albert Link, Ph.D. East Mountain IP ENERGY I-CORPS PROGRAM: YEAR 1 PROCESS AND IMPACT EVALUATION Acknowledgements This research project was initiated and directed by Jeff Dowd of the U.S. Department of Energy’s (DOE) Office of Energy Efficiency & Renewable Energy (EERE) under contract number DE-SOL-0007898. Our team of evaluators would like to thank Jeff for his support and guidance on this project. We would also like to thank the Energy I-Corps pilot managers and management teams at the participating labs: National Renewable Energy Laboratory (NREL, which served the Node Lab) and the Site Labs Argonne National Laboratory (ANL), Idaho National Laboratory (INL), Lawrence Berkeley National Laboratory (LBNL), Lawrence Livermore National Laboratory (LLNL), in partnership with Sandia National Laboratory-California (SNL), and Pacific Northwest National Laboratory (PNNL). These pilot managers participated in lengthy interviews that enabled us to develop a strong understanding of the pilot and its baseline conditions. We are especially grateful for the assistance provided by Matt Ringer and Jennifer Ramsey of NREL, who responded to our requests for clarification and who provided invaluable assistance in fielding the trainee and nonparticipant surveys. We are grateful that Matt Ringer invited us to attend the Energy I-Corps cohort 1 on-site training and a subsequent day-long meeting of the lab pilot managers to share experiences and discuss lessons learned. Attending the Energy I-Corps training, we conducted informal interviews with trainees and instructors. We would like to thank those attending the training for welcoming us. Instructors granted us additional time in subsequent interviews. We thank the trainees and the nonparticipants contacted for this research for the time they took to complete two surveys, one prior to and one after the training period. Everyone we spoke with was overwhelmingly friendly and cooperative. We are grateful to the peer reviewers that Jeff Dowd assembled for this research. They helped guide our detailed evaluation plan and reviewed our draft report. Their critiques and insights greatly improved the work. The peer review team was comprised of Dr. Donald Siegel, Dr. Irwin Feller, Dr. Brian Zuckerman, Dr. Maryann Feldman, and Dr. Lori Lewis. NMR Group, Inc., was the prime contractor of a team that also included Research Into Action, Inc., 360 Innovation, LLC (Dr. Gretchen Jordan), East Mountain IP (Dr. Robert Watson), and Dr. Albert Link. Research Into Action directed this evaluation and was the principal investigator and author of this report, benefitting from the guidance and review provided by the other team members. NMR conducted and reported on the baseline and follow-up surveys with trainees and comparison-group nonparticipants. Dr. Jordan was the principal author of the evaluation plan and the Energy I-Corps logic models. Dr. Watson provided expertise in issues concerning intellectual property. Dr. Link provided expertise in science and technology policy. Gretchen Jordan, Ph.D. i ENERGY I-CORPS PROGRAM: YEAR 1 PROCESS AND IMPACT EVALUATION Notice This document was prepared as an account of work sponsored by an agency of the United States government. Neither the United States government nor any agency thereof, nor any of their employees, makes any warranty, express or implied, or assumes any legal liability or responsibility for the accuracy, completeness, usefulness, or any information, apparatus, product, or process disclosed, or represents that its use would not infringe privately owned rights. Reference herein to any specific commercial product, process, or service by trade name, trademark, manufacturer, or otherwise does not necessarily constitute or imply its endorsement, recommendation, or favoring by the United States government or any agency thereof. The views and opinions of authors expressed herein do not necessarily state or reflect those of the United States government or any agency thereof. Gretchen Jordan, Ph.D. ii ENERGY I-CORPS PROGRAM: YEAR 1 PROCESS AND IMPACT EVALUATION Table of Contents ACKNOWLEDGEMENTS ........................................................................................................ i NOTICE ................................................................................................................. ii TABLE OF CONTENTS ........................................................................................................ iii ACRONYMS AND ABBREVIATIONS ...................................................................................... ix GLOSSARY ................................................................................................................. x EXECUTIVE SUMMARY ......................................................................................................... I PILOT DESCRIPTION ........................................................................................................ II KEY FINDINGS ............................................................................................................... III SECTION 1 INTRODUCTION ........................................................................................... 1 1.1 ENERGY I-CORPS PILOT OVERVIEW ....................................................................... 1 1.2 CONTEXT FOR THE ENERGY I-CORPS PILOT ........................................................... 4 1.3 ENERGY I-CORPS PILOT DESCRIPTION ................................................................... 6 SECTION 2 METHODS ................................................................................................ 15 2.1 ASSURING RESEARCH DESIGN QUALITY ............................................................... 15 2.2 STUDY LIMITATIONS ............................................................................................ 16 2.3 EARLY OUTCOME METRICS ................................................................................. 17 2.4 EVALUATION DATA SOURCES .............................................................................. 18 2.5 SOURCES USED TO ASSESS OUTCOME METRICS .................................................. 21 2.6 SOURCES USED TO DOCUMENT AND ASSESS PILOT PROCESSES ........................... 24 2.7 NONPARTICIPANT SURVEY ................................................................................... 25 2.8 NONPARTICIPANT SAMPLE SELECTION PROCEDURE .............................................. 27 SECTION 3 NEAR-TERM OUTCOME METRIC FINDINGS .................................................. 29 3.1 LEARNING .......................................................................................................... 29 3.2 COMMERCIALIZATION .......................................................................................... 32 3.3 INSTITUTIONAL SUPPORT ..................................................................................... 35 SECTION 4 TRAINEES KNOWLEDGE GAIN AND SATISFACTION ....................................... 38 4.1 TRAINEES’ KNOWLEDGE GAIN .............................................................................. 40 4.2 SHARED KNOWLEDGE ......................................................................................... 43 4.3 TEAMS POSITIONED FOR CONTINUED COMMERCIALIZATION ACTIVITIES ................... 44 4.4 TRAINEE SATISFACTION ...................................................................................... 45 Gretchen Jordan, Ph.D. iii ENERGY I-CORPS PROGRAM: YEAR 1 PROCESS AND IMPACT EVALUATION 4.5 TRAINEES UNDERTOOK CUSTOMER DISCOVERY AND OTHER COMMERCIALIZATION ACTIVITIES ......................................................................... 49 4.6 TRAINEES WORKLOAD AND FUNDING CHALLENGES ............................................... 50 4.7 TRAINEE RECOMMENDATIONS .............................................................................. 53 4.8 LESSONS LEARNED ............................................................................................. 58 SECTION 5 NONPARTICIPANT BASELINE AND KNOWLEDGE GAIN .................................. 59 5.1 BASELINE COMPARISON OF TRAINEES AND NONPARTICIPANTS ............................... 61 5.2 ENERGY I-CORPS INFLUENCE ON NONPARTICIPANTS ............................................. 64 5.3 TRAINEE AND NONPARTICIPANT KNOWLEDGE GAINS ............................................. 68 5.4 NONPARTICIPANT SUGGESTIONS TO IMPROVE LABS COMMERCIALIZATION SUPPORT .......................................................................................................... 71 5.5 NONPARTICIPANT RESPONSE TO THEIR ENERGY I-CORPS INVOLVEMENT ................ 72 5.6 SUMMARY OF ENERGY I-CORPS INFLUENCE SUGGESTED BY NONPARTICIPANT INVESTIGATION .................................................................................................. 73 SECTION 6 EARLY OUTCOMES AS DESCRIBED BY PARTICIPANTS .................................. 74 6.1 PARTICIPATING LABS ANTICIPATE WIDE-RANGING OUTCOMES ............................... 74 6.2 SOME PARTICIPATING LABS ANTICIPATE LICENSES, PARTNERSHIPS AND STARTUPS IN 2016 ............................................................................................ 76 6.3 PILOT PARTICIPANTS SEE SIGNS OF EARLY POSITIVE OUTCOMES .......................... 77 6.4 LESSONS LEARNED ............................................................................................. 83 SECTION 7 ENERGY I-CORPS INFRASTRUCTURE .........................................................