THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INNOVATION

A DISSERTATION

SUBMITTED TO THE DEPARTMENT OF MECHANICAL ENGINEERING

AND THE COMMITTEE ON GRADUATE STUDIES

OF STANFORD UNIVERSITY

IN PARTIAL FULFILLMENT OF THE REQUIREMENTS

FOR THE DEGREE OF

DOCTOR OF PHILOSOPHY

Tammy L. Carleton

September 2010

© 2011 by Tammy Lee Carleton. All Rights Reserved. Re-distributed by Stanford University under license with the author.

This dissertation is online at: http://purl.stanford.edu/mk388mb2729

ii I certify that I have read this dissertation and that, in my opinion, it is fully adequate in scope and quality as a dissertation for the degree of Doctor of Philosophy.

Larry Leifer, Primary Adviser

I certify that I have read this dissertation and that, in my opinion, it is fully adequate in scope and quality as a dissertation for the degree of Doctor of Philosophy.

Riitta Katila

I certify that I have read this dissertation and that, in my opinion, it is fully adequate in scope and quality as a dissertation for the degree of Doctor of Philosophy.

Chuck House

Approved for the Stanford University Committee on Graduate Studies. Patricia J. Gumport, Vice Provost Graduate Education

This signature page was generated electronically upon submission of this dissertation in electronic format. An original signed hard copy of the signature page is on file in University Archives.

iii THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION

ABSTRACT

Is a technological vision needed to drive radical or disruptive innovations? Few studies have discussed a possible relationship between the formation of a technological vision and the sustained creation of radical innovation. Even fewer have analyzed a relationship systematically, and finding a suitable data set has proven to be a challenge to scholars.

Since 1958, the U.S. Defense Advanced Research Projects Agency (DARPA) has sponsored high risk, high reward research and development that bridge the gap between fundamental discoveries and their military uses. The agency’s sole charter has been radical innovation, providing 52 years of sustained practice. This study draws on 59 interviews with DARPA personnel and funding recipients, whose time at the agency spans 45 of those years. Historical interviews and agency documents further validated the data set. Using grounded theory methodology, this study identifies the importance of vision in radical technological innovation, synthesizing prior studies from organizational innovation, technology management, visionary leadership, and industrial research and development. Four major findings are discussed, which address the timing and criteria for innovation visions, specific mechanisms for the formation of visions, the role of socialization on envisioning, and the corresponding governance model. These findings, supported by empirical evidence, add to the current understanding of technological visions and radical innovation research.

iv THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION

ACKNOWLEDGEMENTS

I would like to thank the following individuals, who have supported my research with extraordinary contributions: Chuck House, Riitta Katila, and especially Larry Leifer for being willing to think boldly in the spirit of DARPA Hard; John Cockayne for his generosity of spirit and funds; William Cockayne for his steadfast faith and provocation; and Jon and Suni Carleton for ensuring the final chapters were drafted.

v THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION

TABLE OF CONTENTS

Abstract...... iv List of Tables ...... viii List of Illustrations...... ix Chapter 1 Introduction ...... 1 Research Opportunity...... 2 Research Plan ...... 5 Research Contribution...... 7 Chapter 2 Literature Review ...... 9 Radical Technological Innovation...... 9 A Time for Discovery & Invention...... 12 Processes of Radical Innovation ...... 13 Organizational Sustainability...... 17 Visions of Technology ...... 17 Visionaries and Other Innovation Roles...... 21 Innovation and DARPA ...... 23 Organizational Structure ...... 24 Processes of Innovation...... 29 Measures of Program Success...... 35 Conclusion...... 36 Chapter 3 Methodology...... 37 Grounded Theory Methodology...... 37 Iterative Process of Data Collection and Analysis ...... 39 Theoretical Sampling...... 43 Method Limitations ...... 44 Conclusion...... 45 Chapter 4 Data Set...... 47 Data Sources...... 49 Subject Profile ...... 53 DARPA Personnel ...... 54 Funding Recipients...... 58 DARPA Office Affiliation ...... 59 Conclusion...... 60 Chapter 5 Analysis...... 62 Process Model of Radical Innovation...... 62 Stage 1: Recruitment...... 63 Stage 2: Vision Formulation ...... 63 Stage 3: Program Launch...... 64 Stage 4: Portfolio Management...... 64 Stage 5: Technology Transfer ...... 65 Vision and Recruitment...... 67 Recruiting Criteria and Evaluation...... 70 Channels for Recruiting Technology Visionaries...... 74 Vision Formulation...... 76 Sources of Technology Visions ...... 77

vi THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION

Criteria of a Radical Vision...... 78 Advancing Partial Visions of Technology ...... 81 Expert Workshops...... 83 Proof-of-Concepts...... 84 Similarities Between the Mechanisms ...... 85 Learning the Capacity for Envisioning...... 86 A Community of Informal Learning...... 87 An Entrepreneurial Spirit ...... 88 Converting Visions to Programs ...... 89 Conclusion...... 92 Chapter 6 Discussion ...... 94 A Process Model of Radical Innovation...... 95 New Dimensions of Vision ...... 96 Visions at the Program Level...... 97 Vision Quality ...... 99 Visionaries of Technology ...... 104 The Development of Partial Visions ...... 107 Learning Radical Innovation Through Socialization ...... 109 Internal Review of Radical Innovation Ideas ...... 111 Conclusion...... 113 Chapter 7 Conclusion ...... 114 Contribution to the Field ...... 115 Contribution to the Practice of Innovation ...... 115 Research Limitations...... 117 Directions for Future Research...... 118 Appendix A: DARPA Technology Offices...... 120 Appendix B: Federal Budget Categories ...... 122 Appendix C: Research Interviews Conducted ...... 124 DARPA Staff...... 124 Funding Recipients...... 125 Related Defense Roles...... 125 Other Researchers...... 125 Oral History (Interview) Transcripts...... 126 List of References...... 127

vii THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION

LIST OF TABLES

Page

Table 1: Uniqueness levels of a technological innovation ...... 11

Table 2: Twelve organizing elements of DARPA ...... 33

Table 3: Common processes and interview quotes per DARPA innovation stage...... 67

Table 4: A selection of DARPA program visions ...... 70

Table 5: Criteria for a radical vision at DARPA ...... 78

Table 6: The concept of DARPA Hard...... 81

Table 7: U.S. federal defense budget categories...... 123

viii THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION

LIST OF ILLUSTRATIONS

Page

Figure 1: Open innovation and network partners of DARPA ...... 25

Figure 2: Basic staff structure at DARPA ...... 26

Figure 3: Theory building and theory testing approaches to research...... 38

Figure 4: Iterative process of data collection and analysis ...... 44

Figure 5: Research interviews (n=59) by category...... 54

Figure 6: Different roles held by DARPA subjects (n=29) ...... 55

Figure 7: Distribution of roles (n=42) held by DARPA subjects over time...... 57

Figure 8: DARPA funding recipients (n=19) by organizational type...... 58

Figure 9: Distribution of funding recipients (n=19) based on primary DARPA engagements over time ...... 59

Figure 10: Subject affiliation (n=44) by DARPA office ...... 60

Figure 11: Process model of radical innovation at DARPA...... 63

Figure 12: Radical innovation starts with vision at DARPA...... 68

Figure 13: Rise of starting ages for new DARPA personnel...... 73

Figure 14: Formulation of partial visions of technology for radical innovation ...... 83

Figure 15: Transition between stages of Vision Formulation & Program Definition .. 90

Figure 16: Comparison between DARPA’s process model and the stage-gate model. 95

Figure 17: Visions at DARPA operate at the program level ...... 98

Figure 18: A sample 7-point scale for quantification of human performance variables103

Figure 19: A radical technological vision relies on one big idea and one visionary .. 104

Figure 20: Efforts preceding a complete vision of technology...... 107

ix THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION

CHAPTER 1

INTRODUCTION

• A computerized network of thinking centers operating remotely • An aircraft that cannot be detected by conventional radar • A navigation system that allows land, sea, and airborne users to determine their exact location in all weather conditions • A doctor able to operate virtually on an injured soldier miles away • A robotic ground vehicle that can operate autonomously • A rough-terrain robot that walks, runs, climbs and carries heavy loads • An artificial arm controlled by the human mind • A personal transportation vehicle that can drive and fly

The above ideas are all examples of radical technology visions, funded by the

U.S. Department Advanced Research Projects Agency (DARPA). When these engineering concepts were first imagined, they were at the forefront of technical knowledge, pushing the limits of previous inventions and, in many cases, requiring the creation of new technologies in order to be built. While scholars may refer to radical innovation by a variety of synonyms, such as disruptive (Christensen, 1997) or revolutionary (Dundon, 2002), all the terms share a core definition. Radical innovation is not incremental development; instead, it is a new technology or solution that creates an entirely new and often unexpected market, what industry might call “big R&D”.

Radical innovation is a popular topic in business and engineering literature, largely due to a lack of understanding about what to do and how to do it (e.g.,

Christensen, 1997; Leifer et al., 2000; Christensen & Raynor, 2003; O’Connor, Leifer,

Paulson, & Peters, 2008). Two conflicting beliefs can be found in this literature. First, radical innovation, like many business functions, is seen as a management process that requires specific tools, rules, and discipline. Second, applying traditional evaluation methods and criteria to radical innovation projects is inappropriate and

THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION counterproductive. This study of successful radical innovation is intended to help organizations be more deliberate in their pursuit of technology visions that will generate greater business value and change society’s beliefs about what is possible.

Research Opportunity

One of the principal obstacles in studying radical innovation is finding appropriate data sets (Van de Ven, Angles, & Poole, 2000; O’Connor et al., 2008). For example, O’Connor and colleagues identified a small sample of companies, whose management expressed a strategic intent to develop or evolve a sustained capability for radical innovation. The majority of the companies had recently begun the process, and after several years of study, many were still experimenting with different mechanisms to facilitate their efforts in radical innovation. Although the researchers posit a set of shared patterns of activity across these projects, their ability to describe a single process of radical innovation is limited by the small number of projects completed and by the distribution of successful projects across a variety of companies.

In the search for a larger pool of technology visions and a better ability to compare successful outcomes, this study looks to the Defense Advanced Research

Projects Agency, or DARPA, as providing a repeated, and repeatable, model of radical innovation. Founded in 1958 by the United States Department of Defense, DARPA specializes in radical innovation. The mission of the federal agency is to “maintain the technological superiority of the U.S. military and prevent technological surprise from harming our national security by sponsoring revolutionary, high-payoff research bridging the gap between fundamental discoveries and their military use” (DARPA, 2010b). The

2 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION agency’s name was changed from Advanced Research Projects Agency (ARPA) to

Defense Advanced Research Project Agency (DARPA), then back to ARPA, and in

1996, back to DARPA, purely for political expediency. For simplicity, DARPA is used throughout this report.

DARPA has a long history of finding and building big engineering visions. At

DARPA, the conversion rate of new technologies to market has consistently been higher than comparable private industry sectors over the agency’s long lifespan, and many technologies have transitioned to commercial products, according to studies from

Richardson, Bosma, Roosild, and Larriva (1999) and Richardson, Larriva, and Tennyson

(2001). Easily recognized examples include the foundation of today’s Internet known as

ARPANET, the global positioning satellite (GPS) system, and the da Vinci Surgical

System used for laparoscopic surgery in hospitals (Belfiore, 2009).

DARPA’s organizational structure supports the pursuit of big ideas (Bonvillian,

2006). The organization is independent from other more conventional military research organizations with approximately 100 technical personnel (the number changes as personnel are intentionally rotated through), who directly manage a $3.25 billion R&D budget (DARPA, 2009; M. Peterson, personal communication, June 10, 2010). DARPA’s primary staff consists of program managers (PMs) hired on a short-term basis, typically

3-4 years, to find, fund, and foster new engineering ideas based on technology visions that each one of them is expected to develop. While the PMs develop the overarching visions, they do not bring the visions to life themselves. Instead, the PMs fund teams at universities, industry, and government labs. With the combination of an achievable technology vision and funding, these teams are induced to think and act like innovators

3 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION for an intense period in time, designing and building functioning prototypes that extend today’s knowledge. This practice reflects a model of open innovation (Chesbrough,

2003), in which DARPA has relied on an external network, rather than internal staff, to source and commercialize innovation ideas for over 50 years.

In short, DARPA is a model of a sustainable radical innovation engine.

Nonetheless, DARPA remains relatively unstudied by the academic community. Of the few rigorous studies that exist, researchers have focused on DARPA-funded outcomes in the early computing field (e.g., Hafner & Lyon, 1996; Waldrop, 2001; Roland & Shiman,

2002). DARPA’s model of innovation has been covered superficially, often through personal reflections (Malakoff, 1999; Tennenhouse, 2004; Bonvillian, 2009). A former agency director reinforced the need to study the innovation mechanisms within DARPA, saying: “The literature on DARPA is large, though most is about projects and results, and not about how the magic is performed. Hence a lot of the Agency’s operations are shrouded in mystery” (S. Lukasik, personal communication, September 23, 2009).

A research question was crafted to delve into the mystery of DARPA. The question is stated as such: how do creators communicate their long-range, technical visions during early development? This question was further refined in order to operationalize the various components. “Creators” refer to the individuals responsible for defining and introducing the visions, and the literature notes a special role for the technical visionary (Hebda, Vojak, Griffin, & Price, 2007; Deschamps, 2008). The act of communicating focuses on the delivery of the visions from the creators to those responsible for embracing and building the visions. In the case of DARPA, the receiving group would be the various funding recipients, who hear about the new vision

4 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION opportunities through both private and public channels. The communication itself may take several forms, including in-person discussions and public documents. In the question, technical visions describe the future state of technology concepts that rely on engineering knowledge to build. The qualifier long-range specifies visions that require multiple years to address, and these visions are separate from goals in new product development and immediate R&D initiatives. Lastly, early development refers to the start of the innovation process, which includes conceptualization and prototyping, before formal research and product development begins (Van de Ven, Polley, Garud, &

Venkataraman, 1999).

Research Plan

The research goal was to study the full lifecycle of a technology vision within

DARPA using grounded theory methodology. Grounded theory methodology is an inductive research approach that draws insights directly from multiple cases in real-world contexts for the purpose of constructing theory (Goulding, 2002; Bryant & Charmaz,

2007; Corbin & Strauss, 2008). This research method was chosen for several reasons.

First, as a qualitative method, grounded theory allows a researcher to gain insight into an organizational culture, as well as people’s attitudes, behaviors, and motivations, which cannot be understood through a statistical approach. Qualitative research seeks out the

‘why’, not the ‘how’ or ‘how many’ of its topic through the analysis of unstructured information, such as interviews and observations. The objective is to study things in their natural settings, attempting to interpret phenomena in terms of the meanings that people bring to them. Grounded theory was a necessary starting methodology for a domain in

5 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION which the governing variables are not yet known. Once these variables are identified, then specific hypotheses can be tested in future studies to validate the model.

As the second reason, grounded theory methodology is useful when theories are not available. Limited research exists about DARPA’s innovation model, apart from various anecdotes, and adopting a theoretical framework without first collecting data or applying a framework out of context would be intellectually negligent. Grounded theory enables the construction of new theory from data. As opposed to developing a theory that attempts to correct or clarify previous theories, the aim of this study is to provide an explanation derived directly from reality.

Third, grounded theory methodology considers a phenomenon within its socio- historical context. A key assumption in this study is that organizational activity is heavily dependent on time and place. Individual personalities and projects within DARPA change constantly due to the rotating staff and network structure, so any in-depth research requires placing multiple particular instances in a larger situation.

Fourth, grounded theory offers an emergent approach to data collection and analysis. In particular, hypotheses are not formed in advance, and instead the literature review is updated in parallel with the data collection, so that both efforts may continuously inform each another. Many myths abound about DARPA, and a critical researcher would avoid starting with one model too rigidly, particularly if there may be a danger of missing nuances that could inform deeper understanding. Like the anthropologist entering an alien culture for the first time, it seemed wise to start without undue preconceived notions in order to view the actions at DARPA with clear eyes

(Howe, 1952).

6 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION

In this study, data is comprised of personal stories and archival documents.

Interviews provided the most effective way to hear about the development of visions within DARPA. The most direct source was the program managers, followed by related staff roles (e.g., office director) and then the funding recipients. In total, 47 in-depth interviews were conducted with agency personnel and funding recipients, and the subjects’ time at DARPA span the entire lifespan of the agency and range across a variety of programs.

An additional set of 12 interview transcripts with DARPA personnel were obtained from the Charles Babbage Institute (CBI) archive, which were recorded from

1988 to 1993. Several of the interviews touch on innovation practices at DARPA, helping to validate the responses from current subjects, as well as providing some perspective from early computing pioneers who have since passed away.

Finally, agency documents were reviewed as a secondary source of data. These documents were written at different points in DARPA’s history (1958 – present) and offer another view of agency procedure. The documents also provide a formal and public record of DARPA’s activities related to vision setting. In summary, by triangulating multiple sources, the aim is to reduce potential bias in the interpretation of the data.

Research Contribution

This study extends the current understanding of radical innovation by describing the central role that vision plays, building on studies of organizational innovation, technology management, visionary leadership, and industrial research and development.

This study further examines the specific mechanisms involved in developing visions of

7 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION radical innovation. Part of the study’s learning reinforces the deep-seated belief within the innovation community that processes of radical innovation should be treated separately from processes of incremental innovation. However, new data reveals that the actual processes for the selection and evaluation of ideas within radical innovation runs counter to the existing literature of radical innovation. The details behind these insights are found in the next few chapters.

8 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION

CHAPTER 2

LITERATURE REVIEW

This chapter summarizes the state of knowledge about processes of radical innovation, concentrating on the early stages when a daring technology idea is first formed and evaluated. In addition, the scholarly literature about technology visions is discussed in order to provide broader context for the big ideas that drive radical technological innovation. Lastly, the Defense Advanced Research Projects Agency

(DARPA) is presented as an organization entirely inspired, structured, and measured by radical technological innovation. Existing studies and reports about DARPA are reviewed in order to examine what has been documented about the organization’s internal innovation processes. Overall, research is drawn from multiple disciplines, primarily studies of organizational innovation, technology management, leadership, and industrial research and development.

Radical Technological Innovation

Innovation is a broad term, often used to describe both the process and its output interchangeably. Innovation is a relatively new area of inquiry. In 1942, Schumpeter first defined the modern innovation process, introducing the concept of creative destruction in which new technologies and new skill sets either create or redefine firms and existing markets. Since then, scholars have classified different types of innovation output. One category of innovation is technological innovation, which is defined as “the act of introducing a new device, method, or material for application to commercial or practical objectives” (Schilling, 2005, p. 1). Other example innovation categories are related to

9 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION services, processes, and organizations. Technological innovations can be further classified as either incremental or radical. The general consensus among scholars is that incremental innovations are solutions that improve an existing product or service, such as adding new features that do not create significant changes in the marketplace (Harvard

Business Essentials, 2003; Schilling, 2005). In contrast, radical innovation entails solutions that are very new and different from prior solutions, often generating new markets or industries.

Scholars differ on how radical should be perceived. A recent definition, revised by O’Connor and her management colleagues (2008), defines radical innovation relative to a company’s R&D portfolio, and both risk and impact are based on the management’s perception. Other scholars define radical innovation in terms of a technological breakthrough, which brings new knowledge to an industry overall. Lynn and Akgün

(2001) characterize radical innovations along two dimensions of very high market uncertainty and very high technology uncertainty, yet they do not describe nor attempt to quantify either dimension in detail.

To provide clearer guidelines, Abetti (2000) introduced a five-level scale to identify innovation uniqueness for technologies. In Abetti’s scale, which is reproduced in

Table 1, levels 1 and 2 correlate with “highly radical” and “radical” technological innovations, respectively. While straightforward, his taxonomy does not seem to be widely adopted by other innovation scholars. In addition, while his categories describe the level of “radicalism” for a new technology, they do not fully address the scale of commercial impact or societal change. Considering economic impact, Betz (2003) distinguishes between three scales of technological innovation, which span radical

10 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION innovation, incremental innovation, and next-generation technology innovation. Betz defines radical innovation as a “basic technological innovation that establishes a new functionality”, such as a steamboat engine (p. 73). However, impact is noted in terms of the solution’s functionality, versus other benefits to humanity. In short, a good, all-around workable definition of radical innovation is lacking in the literature.

Level Type of Innovation Description 1 Highly Radical Unique original product or system, which will obsolete existing ones, based on proprietary technology beyond the state-of-the-art, highly specialized and customized, major R&D 2 Radical New product or system, with original state-of-the-art proprietary technology, that will significantly expand the capabilities of existing ones, specialized product with many adaptations, significant R&D 3 Intermediate New product with proprietary technology, but may be duplicated by others, mix of standard and special features, average R&D 4 Significant Significant extension of product characteristics with Incremental original adaptations of available technology, product with standard variations, limited patent protection, minor R&D 5 Minor Incremental Incremental improvement over existing products, application of current technology, standardized product, no patent protection, no R&D

Table 1: Uniqueness levels of a technological innovation Abetti’s five-level scale (2000, p. 209) identified innovation uniqueness for technologies, in which levels 1 and 2 correlate with “highly radical” and “radical” technological innovations, respectively.

Radical is only one term to describe high-risk innovation. Scholars have used other various terms to describe radical innovation, such as disruptive (Christensen, 1997;

Kostoff, Boylan, & Simons, 2004), breakthrough (Fleming, 2002; O’Connor, 2008), game changing (Edelheit, 2004), blue oceans (Kim & Mauborgne, 2005), discontinuous

(Watts, 2001; Norling & Statz, 1998; Rice et al., 1998), transformative / transformational

11 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION

(House & Price, 2009; Johnson, 2010), technology push (Cotterman et al., 2009; Tzeng,

2009), and revolutionary (Dundon, 2002). Regardless of the exact term, radical innovation is critically needed in society as a way to drive progress. As economist Perez

(2002) points out, “technological change occurs by clusters of radical innovations forming successive and distinct revolutions” (p. 6). Additional studies of radical innovation will contribute to the growing scholarship and understanding of a process that has significant consequence on society and individual wellbeing.

A Time for Discovery & Invention

Generally, the start of the innovation process is considered a time of discovery

(O’Connor et al., 2008). In the literature of new product development, scholars refer to this stage as the “fuzzy front end”, which entails activities for the scouting, generation, and screening of ideas (Smith & Reinertsen, 1998; Rhea, 2003). There is less clarity in studies of radical innovation. Many of the companies that O’Connor and her colleagues studied pursued a practice of “open innovation”, looking outside the organization to customers, partners, and other groups for new innovation ideas (Chesbrough, 2003).

Ultimately, O’Connor and her team believe that the discovery process should be centralized internally into what they call an “innovation hub” for greater impact and effectiveness within an organization. Van de Ven and his colleagues (1999) conclude differently. Based on extensive longitudinal studies over 17 years, looking at all types of innovation, they believe that organizations with more divergent and chaotic activities were more likely to develop a tradition of innovation. In particular, processes for

12 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION technically complex innovations proved to be more disorderly than processes for technically simple innovations.

Specific to radical innovation, Betz (2003) posits that the process begins with new scientific knowledge, and he draws a linear connection from scientific discovery to basic invention to ultimately economic exploitation. Science looks to uncover and understand new knowledge. Next, invention puts new knowledge to practice, and invention is a critical step along to path to innovation. Prototypes are then used to test the technical feasibility of an invention before any product design begins. These early steps define the start of the innovation process, which then leads to technology development and commercialization.

Overall, Betz discusses the difficulty that management faces when integrating these different steps functionally within an organization and also in handling intellectual property rights appropriately with multiple partners. He argues that management should support the national development of technologies because next-generation technologies essentially occur at the industrial level, not at the organizational level. In particular, Betz makes a case that the overlap between proprietary and nonproprietary research happens at the step of functional prototyping, which determines the critical decisions in direction and pace for innovation.

Processes of Radical Innovation

Within organizations, the formula to operationalize radical innovation remains murky. O’Connor and her colleagues (2008) outline several historical approaches that established companies have used, which comprise intrapreneuring, skunks works, and

13 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION corporate venture units. Intrapreneuring depends on enterprising individual employees, who act as entrepreneurs in residence by taking direct responsibility for turning an idea into a profitable business. A “skunks works” approach is another model, in which companies allow a small group of experts, typically isolated and protected from headquarters, to work on a risky project through unconventional means. Well-known examples include Lockheed Martin (Johnson & Smith, 1989), Apple’s original Macintosh team (Levy, 2000), and arguably Xerox PARC (Hiltzik, 1999). Corporate venture units have undergone cycles of popularity, based on the success of venture capitalism, and companies use their venture units to invest strategically in startups in order to later adopt or acquire promising new technologies (O’Connor et al., 2008).

The main issue in all these models is that corporate management generally lacks the experience and expertise in understanding how to start and manage radical innovation separately from other R&D initiatives (O’Connor et al., 2008). Dekkers (2005) states:

“Where incremental innovation reinforces the capabilities of established organizations, radical innovation does force them to ask a new set of questions, to draw on new technical and commercial skills, and to employ new problem-solving approaches” (p.

189-190). Few empirical studies have documented which capabilities are needed for radical innovation versus traditional R&D work. An earlier study of food processors found that decision centralization and informal structures tend to support radical process adoption, regardless of organizational size (Ettlie, Bridges, & O’Keefe, 1984).

Other scholars have found a similar issue in competence within companies attempting to create platforms of disruptive growth (Stringer, 2000; Christensen &

Raynor, 2003). In order to develop the necessary capabilities for radical innovation,

14 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION

Christensen and Raynor (2003) argue that companies should focus on processes as a way to create and transform raw resources into value. Processes may be formal, informal, or embedded in the organizational culture. The problem occurs when innovating managers follow processes intended for new product development.

Similar to O’Connor and colleagues (2008), Christensen and Raynor argue that radical innovation requires an entirely different set of processes to be conducted effectively; however, large established companies tend to default to familiar practices.

The problem is compounded for these companies, who tend to judge radically new ideas with the same criteria and short-term biases as other R&D investments. Stringer (2000) posits four reasons why large companies struggle with radical innovation: the cost to change status quo, inflexible bureaucracy in place to maintain organizational stability, an over-reliance on internal R&D projects, and an inability to attract and reward radical innovators meaningfully.

Several management practices have been consistently identified that facilitate radical innovation. Many case studies of organizations show that significant consensus or committee measures – such as oversight boards, governance teams, and commercial councils – are used in order to continuously scan, surface, and select potential new opportunities (O’Connor et al., 2008). These consensus measures are similar with recommendations from other researchers, who discuss the use of innovation boards for reviewing and selecting promising new ideas during the early stages of innovation

(Bacon & Butler, 1973; Hamel, 2002; Snyder & Duarte, 2003; Skarzynski & Gibson,

2008). Innovation boards are typically represented by senior executives who manage successful existing business units, along with members from outside the company and

15 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION occasionally employees from different levels of the company. Innovation boards rotate members and are seldom dissolved as needed. One limit of these case studies is that they focus on large corporations, which tend to default to familiar patterns of committee decision-making. Corporations concentrate on building broad consensus to ensure organizational alignment and leadership support.

The equivalent in academic settings is peer review. Similar to innovation boards, a peer review model relies heavily on group consensus to ensure representative authority

(Daniel, 1993). In terms of supporting the pursuit of cutting-edge innovation, there has been a growing criticism that organizations and funding agencies, which rely on peer review to discover, evaluate and ultimately fund new research ideas, are less willing to sponsor scientists working outside traditional fields or across multiple fields (Chubin &

Hackett, 1990).

Another process often cited in the innovation literature is the stage-gate model

(Cooper, 2001), which organizes efforts in new product development in terms of stages and gates. Before the first stage formally occurs, a period of discovery is used to screen and identify new ideas, and at the end of each stage, a “gate” provides a deliberate decision point for project continuation. The stage-gate model is considered the dominant structure for decision-making in corporate R&D. However, some scholars posit that the stage-gate model is more effective for market-pull innovation within existing markets

(Christensen, 1997; Gassmann & von Zedtwitz, 2003), while other scholars argue that the underlying issue is with the criteria used in screening potential new ideas (Harvard

Business Essentials, 2003).

16 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION

Organizational Sustainability

The literature shows few examples of lasting practice. Despite management’s desire to pursue truly radical innovation, finding a repeatable organizational model is difficult. O’Connor and her colleagues (2008) conducted a longitudinal study from 1995 to 2007 of multiple companies, all of which were pursuing or planned to pursue efforts in radical innovation. They did not find a sustained model in the literature, and none of the companies in their data set had developed an enduring set of practices. In fact, their research found that the average life span of a radical innovation program is four years.

They state, “History shows that new business incubators, new ventures divisions, or other groups that have been founded to accomplish major innovation in companies often don’t last” (p. 22). This finding is consistent with research from Deschamps (2008), who states that most companies are unable to sustain innovation performance, largely due to poor leadership commitment.

Visions of Technology

Generally, scholars mark the start of the innovation process with the generation of new ideas (Harvard Business Essentials, 2003); however, recent work from Cockayne

(2004) demonstrates that significant groundwork is spent in the formative period preceding a complete idea. This is consistent with work from Van de Ven and colleagues

(1999), which found an extended gestation period, often lasting several years, occurs before the development of an innovation. Sources of new ideas are individuals, firms, universities, private nonprofits, or government-funded research, as well as the linkages between them (Schilling, 2005).

17 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION

There is limited research that demonstrates the connection between radical technological innovation and vision. Generally, studies about the processes of technological innovation do not mention vision (e.g., Roberts, 1987; Tornatzky,

Fleischer, & Chakrabarti, 1990). Lynn and Akgün (2001) studied how vision impacts new product success, finding that “research on vision at the project level is curiously lacking” (p. 374). They defined a vision as the desired future state and impact of an idea, what one might consider as a big idea. Examples of technological visions from the broader literature are largely anecdotal, such as the story of 3M listing vision as one of its six principles for the company’s successful tradition of innovation (Van de Ven et al.,

1999) or field advice that companies are missing vision as the key ingredient in their innovation strategy (Lundquist, 2004). Cases of radical product innovation, such as about

Hewlett-Packard’s thermal ink-jet (Fleming, 2002), are markedly absent mentioning vision.

Drawing from studies of organizational vision, Lynn and Akgün (2001) introduced three components to product vision that they note as clarity, support, and stability, which when combined, provides a clear image of what to do. According to their work, vision clarity refers to having a goal that is “a well-articulated, easy to understand target” (p. 375). Vision support is about securing the necessary commitment from the company to execute the vision, and vision stability ensures a consistent vision over time.

Lynn and Akgün found that only vision clarity was significantly associated with new product success for radical innovations, at least as defined by the subjects’ perception of market and technology uncertainty. One weakness of the study is that their sample of successful radical products was drawn entirely from the computing industry. In addition,

18 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION while some products may have introduced a new product category, none established a new market category, a core aspect that defines radical innovation. In comparison, Betz

(2003) describes two types of visions for the leaders who manage technology. The first is a vision of the enterprise system, and the other is a vision of the technologies within the enterprise system. A criticism is that his definition lacks a sense of future possibility or technical challenge; instead, he characterizes vision solely in terms of the engineering solution, either as a total integrated system or as a set of individual components.

Although literature about vision at the project level is scarce, in contrast, there is a much richer stream about vision at the organizational level. Several studies, such as by

Lynn and Akgün (2001) and Deschamps (2008), base their models of technological vision on these studies of organizational vision as a way to address this gap. Most frequently cited is the study about visionary companies by Collins and Porras (1991,

1996, 1997), who analyzed 20 Fortune 500 companies, selected from a popularity poll with CEOs. They posit that a vision consists of two major components: core ideology, which is captured in core values and core purpose, and envisioned future, which consists of a 10-to-30-year daring goal, plus vivid descriptions of what it will be like to achieve the goal. Collins and Porras found that vision was a tricky term, noting that, “vision has become one of the most overused and least understood words in the language” (1996, p.

65). As good management consultants, they coined the term “Big, Hairy, Audacious

Goal”, or BHAG for short, to describe an envisioned future (J. Porras, personal communication, May 5, 2008). The test for a successful BHAG includes several criteria, notably that it is clear, concise, compelling, “serves as a unifying focal point of effort”,

“acts as a catalyst for team spirit”, engaging, tangible, energizing, and highly focused

19 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION

(2006, p. 75). The authors do not present a process for developing visions, although they describe the components of a great vision.

Other scholars (e.g., Nanus, 1992; Quigley, 1993) have emphasized similar vision components within the organizational literature and describe different processes for formulating corporate visions. Quigley (1994) points out that that “little has been written on how to communicate vision, how to renew it, and how to sustain it over long periods”

(p. 37). He describes a corporate vision as clear, stimulating, accessible, as well as open- ended, and he argues that vision must precede strategy within a company. Quigley outlines a consensus-driven process for the formation of visions, which relies on a series of internal leadership conferences to build alignment and support from the start.

Deschamps (2008) takes a similar view. He recognizes that a vision for technology requires special formulation and argues that the process follows analogous mechanisms as a business vision. Deschamps states, “That process, which can be called

‘technology visioning’, is similar to what process management teams use to sharpen, reorient or renew their business vision” (p. 147). In his view, a technology vision is a means of implementing the business vision, and he notes that a typical output is a short policy statement or charter that guides R&D initiatives, not necessarily a driving view of the future. Overall, he mixes a technology vision with operational aspects within corporate R&D, such as group organization and technology acquisition strategy. He shifts the focus from building radical innovations to the procedures for technology deployment and management. In a related vein, Christiansen (2000) describes the process of vision formulation based on near-term innovation horizons:

Vision-building, in any particular market, involves identifying market problems or opportunities, and technical means to solve them. So, if you are looking for a

20 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION

vision, ask yourself if there are customer problems or opportunities you could address. (p. 178)

Christiansen takes a more reactive stance, in which the process of building visions entails finding an existing customer need for new products, versus creating an entirely new market. Overall, the scholarship about technology visions, particularly how these visions are crafted and communicated, would benefit from more in-depth study.

Visionaries and Other Innovation Roles

When discussing visions, it is almost inevitable to extend the discussion to visionaries. The business literature has a deep history of leadership studies that address the unique role of visionaries, typically chief executive officers (CEOs) who inspire the staff through top-down measures (e.g., Quigley, 1993; Sigismund, 2000). Based on a historical study of HP, a technology company that has pioneered many new product markets, House and Price (2009) distinguish between two approaches of visionary leadership. Either top management directs change visibly or instead fosters an organization culture that allows other visionaries and their visions to emerge.

There is limited scholarship about technology visionaries, who are technically trained and personally responsible for spearheading the creation of a new market- disrupting technology. Two related studies by Hebda, Vojak, Griffin, and Price (2007) and Vojak, Griffin, Price and Perlov (2006) differentiated the technical visionary as being separate from the CEO role and leading innovation efforts as the big idea driver. They note: “Technical visionaries are technologists who were involved intimately involved in a project, from its inception, which represented a significant technical breakthrough, came to market, and significantly changed life” (2007, p. 433). Rhetorical perspectives from

21 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION the field, such as commentary by Maccoby (2007), reinforce that another class of visionary leader exists within technology separate from the CEO. Deschamps (2008) argues a similar line of thinking, and he considers chief technology officers (CTOs) to be the primary innovation leaders in companies. Deschamps posits six attributes that define an innovation leader, which entail disciplined creativity, ability to handle uncertainty and risk, passion, willingness to explore and experiment with external ideas, the courage to stop projects, and a talent for recruiting and building innovation teams.

Beyond visionaries, other roles are described in the innovation literature, namely gatekeepers, project champions, business innovators, technical innovators, and user innovators (Howell & Higgins, 1990). Gatekeepers interface with the world outside an organization by helping to acquire, translate, and disseminate company information and resources. Project champions serve as evangelists, promoting a particular idea. Business innovators shepherd emerging ideas by providing political coverage and access to resources, and technical innovators are responsible for the idea’s design and development. Lastly, user innovators ensure user adoption through training and other efforts. Of these various roles, Howell and Higgins associate a vision most with the project champion, stating:

The literature on champions and innovation highlights the capacity of champions to inspire and enthuse others with their vision of the potential of an innovation, to persist in promoting confidence in themselves and their mission, and to gain the commitment of others to support the innovation. (p. 320)

Their data suggests that champions are effective because they appeal to the larger principles and values, including “the organization’s dream of what it can be”, in others

(p. 336). In their view of the literature, the technical innovator, the one most likely

22 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION technically trained and familiar with technology, does not take on the visionary role and instead focuses on an idea’s implementation.

Innovation and DARPA

The Defense Advanced Research Projects Agency (DARPA) is the central organization for research and development (R&D) for the United States Department of

Defense (DoD). DARPA is an organization entirely defined and driven by radical technological innovation. Founded in 1958 as a response to the Soviet Union’s launch of

Sputnik, DARPA’s mission has always been to prevent technological surprise. Neil

McElroy, then U.S. Secretary of Defense, stated, “I want an agency that makes sure no important thing remains undone because it does not fit somebody’s mission”

(Richardson, Larriva, & Tennyson, 2001, p. v).

The current agency director Regina Dugan has continued to uphold DARPA’s focus, stating: “From its founding more than 50 years ago to current day, this mission implies one imperative for the Agency: radical innovation for national security”

(DARPA, 2010b). DARPA focuses on revolutionary initiatives with high technical risk, which are differentiated from the “typically evolutionary efforts” appropriate to the R&D programs undertaken in the individual military services (Defense Science Board, 1999, p. vi). The Defense Science Board (1999) considers DARPA to be an “idea” agency because DARPA looks beyond today’s known needs and requirements. Other scholars echo this point; DARPA encourages and supports technology advances with non-defense organizations, both academic and commercial, rather than defense-only organizations, as its best means of gestating new concepts into implementation (Bonvillian, 2009).

23 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION

Most DARPA projects range in time horizons, in which short-term is defined as within two years and long-term is beyond 10-15 years (Defense Science Board, 1999).

DARPA also funds disruptive capabilities, which are defined as “more than just new technologies; they are transformations in operations and strategy enabled by synergistic combinations of technologies” (Van Atta et al., 2003a). Over the years, DARPA’s programs have concentrated on major national issues, such as missile defense, nuclear test detection, information processing, and counterinsurgency.

Organizational Structure

DARPA operates with limited overhead and does not maintain any research facilities or labs. The rationale is to “minimize any institutional interests that might distract the Agency from its imperative for innovation” (DARPA, 2007). Much of the work occurs offsite at the organizations funded by DARPA, presented in Figure 1. How

DARPA works is often attributed to the people within the organization, and these people have then reinforced and institutionalized certain procedures over time, which then has established a unique culture at DARPA. As Bonvillian (2006) writes, “An agency is its culture, not just its enabling statute or organizational chart” (p. 47).

24 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION

!"#$%&'()*

3%2"&%4* /&'0(1)'2()* 4%5"1%6"1'()*

+,-.,*

-()(%178* :"0(1&#(&6 &"&$1"96)* -;+*7(&6(1)*

<'4'6%1=* )(10'7()*

Figure 1: Open innovation and network partners of DARPA DARPA relies on a network of multiple external organizations, which perform the work of innovation offsite, in a business model known as “open innovation”.

Three main staff roles work at DARPA regarding its technology programs: the agency director, the office director, and the program manager, illustrated in Figure 2. The

DARPA agency director steers the organization, serves as the primary spokesperson, and determines the overall investment strategy and office structure. There have been 18 agency directors in DARPA’s history, averaging 2.8 years in office (DARPA, 2008). A deputy director assists the agency director, typically overseeing internal operations and management.

25 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION

!"#$%&' ()*#%+,*' (#34+&' ()*#%+,*'

-.%#' -.%#' ()*#%+,*' ()*#%+,*'

/*,"*01' /*,"*01' /*,"*01' 20$0"#*' 20$0"#*' 20$0"#*'

5433,*+'5+06'

Figure 2: Basic staff structure at DARPA A small number of roles are responsible for the technology programs at DARPA. The basic technical roles include the agency director, office director, and program manager, and all personnel in these roles serve short-term contracts.

Reporting to the agency director, an office director oversees a DARPA technology office, and typically, there are 6-8 active office directors at a given time.

Offices are organized around key development areas in technology and science, and they have been created and disbanded, as needed, throughout DARPA’s history (DARPA,

2010, April 7). The themes of each office are derived from DARPA’s strategy and set by the agency director as a result of assessments of technical advances worldwide and interactions with DoD stakeholders (DARPA, 2007). As examples, the Microsystems

Technology Office (MTO) spearheads pioneering research in integrated microsystems in order to enable revolutionary performance and functionality for future DoD systems, and the Information Processing Techniques Office (IPTO) supports research, development, and prototyping that spans the information lifecycle. Disbanded offices include the Joint

26 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION

Unmanned Combat Air Systems (J-UCAS) Office and Information Awareness Office.

See Appendix A for details on current DARPA technology offices.

A set of program managers is affiliated with a particular DARPA Office. A program manager, often abbreviated as PM, is responsible for a particular program or technology initiative, and programs often serve as an organizational structure for multiple related projects. Program managers are recruited from multiple areas, including companies, universities, national laboratories, nonprofit research organizations, federally funded R&D centers, and the military services. DARPA maintains its entrepreneurial edge by hiring expert program managers from industry at competitive salaries and through faster procedures than normal civil service rules due to special authority from

Congress (DARPA, 2007). Program managers bring deep technical knowledge and connections:

So what makes a DARPA PM unique? First and foremost, they’re subject matter experts, either already a leader in their technical community, or prepared to become a leader as they take on the challenge of creating and managing DARPA programs. They’re driven with the zeal of a venture capitalist, driven by the conviction that their program is creating a major technological advance. They’re ready and willing to take risks. Not a venture capitalist’s financial risk, but more precisely, a professional risk, the risk of trying to transform a far-side scientific or engineering insight into demonstrable warfighter capability, and with no guarantees of success. (Leheny, 2007)

Approximately 140 technical program managers serve at DARPA, and they often work with light oversight from DoD management (DARPA, 2007). While program managers were always hired into DARPA on temporary contracts, a four-year rotational cycle was enforced in the early 1990s as a way to improve quality and technical expertise

(Defense Science Board, 1999). By shortening contracts, at least three challenges arose

(ibid.). First, internal collaborations, particularly between DARPA Offices, were driven

27 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION more by staff personality than programmatic needs. Second, program managers were motivated to plan in shorter shifts based on their contracts, rather than longer-term horizons. Third, institutional memory and continuity dropped significantly, resulting in an annual personnel turnover rate of approximately 25 percent (DARPA 2008b).

Due to the staff rotation policy, DARPA program managers keep the organizational culture dynamic by regularly infusing the agency with new ideas and visions. The program managers are perceived as being visionaries because they are directly responsible for imagining and realizing their visions of technology. One report states the association clearly: “These individuals must be highly motivated visionaries who are successful at both management and technology” (Defense Science Board, 1999, p. 7, emphasis added). Moreover, DARPA program managers are expected to seek results, rather than just explore their visions. Having both a vision and the ability to facilitate building the vision is a critical hiring attribute because program managers bring with them “an inherent desire to accomplish a vision or goal within their tenure” (p. 8).

Program managers have developed an almost mystical power in scientific and popular press for their ability to imagine and build new ideas in a compressed timeframe.

For example, Malakoff (1999) notes in Science that “some [program managers] become influential figures in their subfields, capable of nudging established research communities in a particular direction or creating collaborations where none existed before.” Esquire magazine (Junod, 2003) wrote similarly:

All of them are on contracts that are finite, by nature and definition. That’s the signal element of the DARPA culture: It’s a government job, but it’s the opposite of the civil service, and even of academia. You don’t get tenure. You don’t get security. What you get, generally, is about four years – four years to do something that’s never been done before. Four years to do the impossible… It’s what accounts for the fact that whole fields of inquiry rise up around the enthusiasms of

28 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION

program managers – fields of inquiry that weren’t there before and that keep going after the program manager is gone. Call them program managers if you want, but really, if the whole DARPA ethos works, they become fathers of their fields. Mothers of invention. Inventors.

Several additional groups work at DARPA, and supporting personnel include the

DARPA Small Business Programs Office (SBPO), contract management office, and facilities office (DARPA, 2010d). In addition, Systems Engineering and Technical

Assistance (SETA) contractors are civilian employees or government contractors who are hired to assist with DoD programs, and a DARPA program managers may assign several

“SETAs” to a program who provide project management and administrative assistance.

In addition, various contracting officer representatives, who are trained and certified by the DoD, help DARPA program managers to negotiate and maintain project contracts with the teams that they fund. Operational liaisons from each U.S. military service and the National Geospatial Intelligence Agency further facilitate the technology transfer process for DARPA programs (DARPA, 2010e).

Processes of Innovation

DARPA’s focus is radical innovation, and its processes are designed to enable this goal. The agency does not have a systematic process for deliberately planning for the future; instead, DARPA encourages a decentralized process, trusting office directors and program managers to set their own research portfolios (Defense Science Board, 1999).

One recent report summarized the internal process as a series of “fast, flexible, and informal cycles of ‘think, propose, discuss, decide, and revise’ (DARPA, 2007). DARPA has been remarkably consistent in its approach to innovation. Work from Fuchs (2010) examined the way program managers planned new technologies within one DARPA

29 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION

Office, finding that the specific processes have not changed over time, but rather the situations in which program managers have applied these processes. The National

Research Council (2005) describes DARPA’s decision-making process as unique among other government agencies, stating: “It is informal and relatively flexible, taking a top- down approach to problem definition and a bottom-up approach to generating ideas and solutions, with the key emphasis on technical merit” (p. 14).

The Institute for Defense Analyses identified six management practices that have facilitated DARPA’s development and exploitation of various disruptive capabilities for military advantage (Van Atta et al. 2003a, Van Atta et al. 2003b). Across different programs and applications, these six practices are identified as follows (p. S-1):

• Investing in basic technologies that can lead to fundamental technical advantages; • Building communities of change-state advocates; • Defining strategic challenges in detail across multiple scenarios; • Supporting the conceptual development of integrated, disruptive capabilities; • Testing promising disruptive capabilities in large-scale, proof-of-concept demonstrations; and • Working with leadership from the Office of the Secretary of Defense (OSD) to broker Service commitment to the implementation of particular disruptive capabilities.

Research from Richardson and his colleagues (1999) mention similar management practices. For DARPA’s Technology Reinvestment Project (TRP), one of the largest commercial investment programs ever undertaken by the DoD, the agency pioneered a new mode of development called “dual use”, which refers to technology development that accommodates commercial interests while maintaining military utility.

Several new management processes and tools were introduced to make this program work, which included the use of cost share, agreements tailored to projects, consortia, and

30 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION flexibility on intellectual property rights (IPR) and foreign participation, many of which helped influence other DoD organizations.

Bonvillian (2009) introduces the notion of challenges, which he attributes to the influence of Vannevar Bush during WWII. In short, project teams were brought together around and inspired by major technology challenges, such as radar or atomic weapons.

More importantly, the people who worked on these challenges were able to connect their work on technology breakthroughs to the follow-on stages of development, prototyping and production. Bonvillian feels that DARPA effectively upholds this connected model by tying invention to innovation.

In addition, Bonvillian compares DARPA’s way of working with great groups from history, namely Edison’s “Invention Factory” in the 1900s, MIT’s Rad Lab in the

1940s, and Bell Labs in the late 1940s, which all produced amazing new developments in technology. Bonvillian notes similarities with other high-risk R&D efforts: “Many of the organizational features of these three ‘great groups’ are common to others, including the development of atomic weapons at Los Alamos, the integrated circuit and microchip at

Fairchild Semiconductor and Intel, the aeronautics and stealth advances at Lockheed’s

Skunk Works, the personal computer at Xerox Parc and Apple, biotech at Genentech and

Craig Venter’s genomics projects” (p. 218). According to Bonvillian, these organizations share common characteristics, which include a highly collaborative team of experts pulled from multiple disciplines, a flat and meritocratic organizational structure, and direct access to top decision makers. DARPA develops this model further, and Table 2 summarizes the 12 organizing elements that define DARPA’s model for successful

31 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION innovation, as presented by the agency across its various materials (Bonvillian, 2006, p.

46-47).

Element Description Small and flexible DARPA consists of only 100–150 professionals; some have referred to DARPA as “100 geniuses connected by a travel agent.” Flat organization DARPA avoids military hierarchy, essentially operating at only two levels to ensure participation. Autonomy and DARPA operates outside the civil-service hiring process and standard freedom from government contracting rules, which gives it unusual access to talent, bureaucratic plus speed and flexibility in organizing R&D efforts. impediments Eclectic, world- DARPA seeks great talent, drawn from industry, universities, and class technical staff government laboratories and R&D centers, mixing disciplines and theoretical and experimental strengths. This talent is hybridized through joint corporate-academic collaborations. Teams and At its very best, DARPA creates and sustains great teams of networks researchers that are networked to collaborate and share in the teamʼs advances, so that DARPA operates at the personal, face-to- face level of innovation. It isnʼt simply about funding research; its program managers are dynamic playwrights and directors. Hiring continuity DARPAʼs technical staff are hired or assigned for three to five years. and change Like any strong organization, DARPA mixes experience and change. It retains a base of experienced experts who know their way around DoD, but rotates most of its staff from the outside to ensure fresh thinking and perspectives. Project-based DARPA organizes a significant part of its portfolio around specific assignments technology challenges. It works “right-to-left” in the R&D pipeline, organized around a foreseeing new innovation-based capabilities and then working back to challenge model the fundamental breakthroughs that take them there. Although its projects typically last three to five years, major technological challenges may be addressed over longer time periods, ensuring patient investment on a series of focused steps and keeping teams together for ongoing collaboration. Outsourced DARPA uses technical, contracting and administrative services from support personnel other agencies on a temporary basis. This provides DARPA the flexibility to get into and out of a technology field area without the burden of sustaining staff, while building cooperative alliances with the line agencies it works with. Outstanding In DARPAʼs words, “The best DARPA Program managers have always program managers been freewheeling zealots in pursuit of their goals.” The DARPA directorʼs most important job historically has been to recruit highly talented program managers and then empower their creativity to put together great teams around great advances. Acceptance of At its best, DARPA pursues a high-risk model for breakthrough failure opportunities and is very tolerant of failure if the payoff from potential success is great enough.

32 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION

Element Description Orientation to DARPA historically has focused not on incremental but radical revolutionary innovation. It emphasizes high-risk investment, moves from breakthroughs in a fundamental technological advances to prototyping, and then hands off connected the production stage to the armed services or the commercial sector. approach From an institutional innovation perspective, DARPA is a connected model, crossing the barriers between innovation stages. Mix of connected DARPA typically builds strong teams and networks of collaborators, collaborators bringing in a range of technical expertise and applicable disciplines and involving university researchers and technology firms that are usually not significant defense contractors or beltway consultants (neither of which focus on radical innovation). The aim of DARPAʼs “hybrid” approach, unique among American R&D agencies, is to ensure strong collaborative “mindshare” on the challenge and the capability to connect fundamentals with applications.

Table 2: Twelve organizing elements of DARPA Bonvillian (2006, p. 46–47) outlines 12 organizing elements that define DARPA’s model for successful innovation, many of which are shared with other great groups in history, such as MIT’s Rad Lab and Bell Labs in the 1940s.

At DARPA, project managers are empowered to make decisions quickly, and the agency minimizes bureaucracy by distributing administrative tasks and contract negotiations through a large support staff. For example, in one large initiative, program managers were not required to follow normal acquisition procedure, instead skipping initial decisions concerning “make or buy” and “co-develop or procure” (Richardson et al., 1999, p. vii). In fact, DARPA personnel are exempt from standard DoD training for acquisitions (L. Schuette, personal communication, October 16, 2009). In addition,

DARPA received additional authority in 1993 to conduct prototype projects outside the traditional contracting statutes (Defense Science Board, 1999).

Among its network, DARPA encourages diverse collaboration on projects. For example, projects in the Technology Reinvestment Program regularly combined defense teams with teams from academia, the commercial sectors, and other nonprofit organizations, and with the right combination, teams were often able to address a broader

33 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION set of problems than an organization working alone (Richardson et al., 1999). A former program manager describes similar mixing while at DARPA, noting cycles of competition and hybridization between university and industry researchers: “These cycles of competition and hybridization lead researchers to quickly adopt the best of each other’s ideas and are highly effective in accelerating the transfer of people and technology across university and industry boundaries” (Tennenhouse, 2004, p. 22). Fuchs

(2010) has found that collaboration across groups within DARPA has increased since

2001. Before 2001, DARPA program managers relied more heavily on successfully published professors, whom Fuchs calls “star scientists”, to work on DARPA-funded projects, and after 2001, program managers engaged a broader mix of subjects, namely established vendors within universities, industries, and start-ups. One common mechanism for exchanging knowledge was through invite-only workshops (Fuchs, 2010).

In addition, DARPA takes an unusual approach to intellectual property rights

(IPR). Unlike most businesses and government labs, which rely heavily on IPR to maintain a competitive advantage, DARPA does not file patents or claim legal ownership to the various technologies that it funds. Instead, the agency allows ownership and control to reside within the teams. DARPA desires visibility to ideas of radical innovation, as well as the influence to ensure that these ideas are provoked and built

(Defense Science Board, 1999). Richardson and his colleagues (1999) explain, “By allowing industry to retain rights to the intellectual property generated during their project, government gained early access to some of industry’s best ideas” (p. xi).

34 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION

Measures of Program Success

Arguably, the first measure of success is the actual physical instantiation of a daring idea. In other words, can you build it? For its 50-year anniversary, DARPA chronicled its major contributions to the world, which include interactive computers, internet computing, virtual reality, intelligent systems, stealth technology, and more (Van

Atta, 2008). Once new technologies are built, DARPA considers the effective transfer of technology to the U.S. military services as its primary measure of success. One internal report (DARPA, 1998) concludes that: “The success of DARPA has been measured historically by the transition of its technologies and concepts into military capabilities in the hands of U.S. forces. By that measure, the Agency has been phenomenally successful, considering its size” (p. 9). The same report notes that nearly every current military system has been affected to a substantial degree by DARPA work in information technology, microelectronics, and materials.

Collaboration with industry partners plays a significant role in the success of

DARPA’s programs in the marketplace. According to Richardson, Larriva, and Tennyson

(2001), part of the agency’s success comes from its ability to seek a broad range of customers, unlike similar R&D counterparts in the military services that typically address one or two groups of users. After tabulating transfers of technology to the military services for three periods in DARPA’s history, they found that that highest impact came from working with industry partners, which was pursued roughly 30 percent of the time.

In DARPA’s Technology Reinvestment Program, which funded dual use technologies, 33 percent of the projects were already selling their technologies or products in the commercial market (Richardson et al., 1999).

35 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION

Conclusion

Two general conclusions can be drawn from the past research on radical technological innovation, which identify a critical gap in the literature. First, although organizations deeply desire radical innovation, their efforts tend to be short-lived, often lasting less than four years. Little is understood about the specific mechanisms that organizations should pursue on a repeatable basis and which factors are influential or decisive to the start of the overall process. Second, the role of vision in radical innovation has not been well understood or documented. Scholars have been heavily influenced by numerous organizational studies, which address corporate vision, and a research opportunity exists to define and clarify different dimensions of vision as related to technological innovation.

DARPA provides an example of an organization that has consistently sustained radical technological innovation over 50 years. As a networked organization with high staff turnover, the agency presents several challenges in data collection. The next chapter presents the methodology used to document and uncover DARPA’s model of innovation.

36 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION

CHAPTER 3

METHODOLOGY

A qualitative approach was selected for this research study in order to most effectively address the proposed research question. When variables are unknown, as in this case, a qualitative approach enables the discovery and identification of the governing variables. The goal of qualitative research is to discover patterns forming a complex and holistic picture, which emerge after close observation, careful documentation, and thoughtful analysis of the research topic. In addition, using qualitative methods in empirical studies within engineering allows researchers to address the human aspects that are often overlooked (Seaman, 1999). This section describes grounded theory methodology as the research method.

Grounded Theory Methodology

In this study, data was collected and analyzed using grounded theory methodology. This methodology has a long history, and since the late 1960s, it has been used extensively to address questions of interest to researchers in various fields, including medicine, social science, and management (Glaser & Strauss, 1967; Goulding, 2002;

Bryant & Charmaz, 2007; Corbin & Strauss, 2008). Instead of testing theory developed by other scholars, grounded theory methodology builds new theory directly from data, as illustrated in Figure 3 (adapted from de Vaus, 2001, p. 6).

37 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION

23$)%4+"97:;7*8+'<<%)'=3+

!"#$%&'()*+ !"#$%&'()*+ !"#$%&'()*+ !"#$%&'()*+ ,+ -+ .+ /+

!"#$%&'()*(+,-"."/)

0$1++ 23$)%4+

23$)%4+>$#(*8+'<<%)'=3+

567#(*8+ 23$)%4+

0(#$%&'()*(+,-"."/)

!"#$%&'()*+ !"#$%&'()*+ !"#$%&'()*+ !"#$%&'()*+ ,+ -+ .+ /+

Figure 3: Theory building and theory testing approaches to research As a theory-building approach, grounded theory methodology uses inductive reasoning to craft a new theory from observations in the field (adapted from de Vaus, 2001, p. 6). Theory-building approaches are recommended when studying new or uncharted areas of research.

Grounded theory begins with observations and uses inductive reasoning to derive a theory from these observations. Theories are general explanations based on a large amount of data. The aim is to generate hypotheses that fit the data, not to build up the weight of evidence necessary to confirm existing hypotheses. This method was adopted because it allows for discovery when studying new or uncharted areas of research in which little theory has been developed, as evidenced by the literature review about

DARPA’s innovation process. The method also supports the study of things in real-world

38 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION contexts, attempting to interpret phenomena in terms of the meanings that people bring to them.

Grounded theory seeks the discovery of unforeseen activities in natural activities and settings. By applying grounded theory to DARPA, the intention is to uncover the inner workings of the agency in terms of radical innovation. Hypotheses are not developed in advance, and working explanations are instead allowed to emerge throughout an iterative process of data collection and analysis. Data saturation occurs when data categories and themes become repetitive, and nothing substantially new is learned.

The final output is a new theory, which may be explanatory (what is the nature of the phenomenon), descriptive (why the phenomenon occurs), or predictive (how the phenomenon could reoccur under similar future conditions). In applied fields, good theory is required to be of practical value, ultimately to “explain the meaning, nature, and challenges of a phenomenon, often experienced but unexplained in the world in which we live” (Lynham, 2002, p. 222). Good theory results in two kinds of knowledge: outcome knowledge, which is usually used to help explain and predict a phenomenon, and process knowledge, which is used to increase understanding about how something works and what it means (ibid.).

Iterative Process of Data Collection and Analysis

Grounded theory methodology relies on an iterative and systematic process of data collection and analysis, and each cycle of data analysis determines the next cycle of data collection. Data collection is based on empirical evidence, and interviews are

39 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION frequently the main source of information in grounded theory. Based on the study requirements, interviews were chosen in this study. Direct observation was not possible due to the historical perspective desired, security considerations of multiple projects, and distributed nature of the innovation work at DARPA. An excerpt from a raw interview transcript follows as an example of data collected in this study:

Data Example: Excerpt From an Interview Transcript

“One of the things about program managers – I donʼt know if the others found this as well – but as a program manager, my perspective was that once you funded somebody, you remained involved. At those times, you were remaining involved and you would go out every 3-6 months or sometimes more frequently, but you would go out a visit with them with the purpose of seeing how things were going, and if they are not going as well as youʼd like, see what you can do to help them, even if it required more money, but whatever it would take, just to help them get either back on track or hire somebody else to get something done that they werenʼt able to get done. It was truly, in my mind, a partnership between the program manager and the contractors, the ones that were actually doing the work, and I almost felt like a venture capitalist – if you need more money Iʼll get it for you, type of thing.”

In grounded theory, data analysis begins by coding the interview transcripts and field notes. The researcher defines a set of different codes, or categories, based on a particular theme or idea that is of interest in the study. Coding is a dynamic, intuitive and creative process of inductive reasoning, thinking and theorizing. These codes are then assigned to specific portions within the text or interview transcript. In coding concepts, a researcher must pay close attention to the words and meanings of the subjects, as contained in the field notes, to make sure that they are being interpreted accurately.

Codes can be either preformed or post-formed based on the objectives of the study.

Various passages of text are then grouped into patterns according to the codes and sub-

40 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION codes they have been assigned. Seaman (1999) notes that “intuition may help guide the process of analyzing the data, but it does not constitute conclusions unless it is clearly supported by the data” (p. 567).

For this study, coding started with open coding, in which codes were identified without any restrictions. Example codes in this study included “processes for discovery”,

“definitions”, “DARPA Hard”, and “community”. The purpose was to compare emerging properties and dimensions between codes, as well as to discover promising nuggets that could lead to meaningful findings. Open coding produces a preliminary structure of categories and sub-categories, which are then refined through subsequent cycles of data collection and analysis.

As part of each cycle of coding, a researcher writes a field memo that articulates a proposition (working hypothesis) or an observation synthesized from the coded data. The field memo is meant to be an informal way to record a researcher’s discoveries quickly before they are lost. Throughout analysis, a researcher attempts to gain a deeper understanding of what he or she has studied and to continually refine interpretations of the data. More detailed memos also show how strong or weak the support for a particular proposition is thus far in the data collection process. An excerpt from a field memo in this study follows as an example:

41 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION

Data Example: Excerpt From a Field Memo

August 23, 2009

One observation emerging from my data is that the type of PM has significantly changed over the years. At least within IPTO, in the early days, several PMs were graduate students – some fresh starting their doctorates. Now they are largely senior and accomplished (narrow?) experts. While youth brings enthusiasm, age has experience… and more connections? As Iʼve seen in the foresight work at Stanford, age matters. Graying PMs would also age the agency in many ways, bringing / creating different views of innovation. I should explore this idea further in the next interviews.

A specific type of memo is an integrative memo, which aims to synthesis previous concepts and coding threads, including making connections with previous research. An example of an integrative memo for this study follows:

Data Example: Excerpt From an Integrative Memo

November 7, 2009

I feel that OʼConnor (2008) and team are dismissive of projects, which they felt were disconnected and one-off efforts. Instead, they recommend a platform approach because platforms “can be the foundation for a variety of business models, products, and applications” (p. 62) either around a technology or a business problem. It is interesting to compare this viewpoint to DARPAʼs structure. Perhaps the agencyʼs offices are equivalent to meta- platforms, in which the programs serve as platforms, and the projects are units of innovation that can be easily funded, tracked, and transferred.

In this study, the process of coding and drafting field notes was manually handled.

Done mainly for convenience, manual coding in a field log (notebook) supported faster reflection in a greater variety of locations, particularly when a computer was not readily accessible. The decision to use manual coding is dependent on the size of the project, the funds and time available, and the inclination and expertise of the researcher (Basit, 2003).

42 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION

Theoretical Sampling

Ideally, data collection and analysis occur concurrently, and the feasibility of the new proposition is then checked in the next round of data collection. This procedure is called “theoretical sampling”, which lets the research guide the data collection (Corbin &

Strauss, 2008). Instead of sampling people, the focus is on the core concepts within the data. The act of theoretical sampling is used to maximize opportunities to develop concepts in terms of their properties and relationships with other concepts, and the sampling decisions evolve during the research study. This means that relevant concepts are elaborated upon and refined through purposeful gathering of data pertaining to these concepts. Corbin and Strauss explain that, “The researcher is purposefully looking for indicators of those concepts so that he or she might examine the data to discover how concepts vary under different conditions” (p. 144).

New concepts might generate questions, and these questions lead to more data collection. As new insights are gained or new leads open up, the researcher can shift perspective quickly and explore new areas of inquiry. This is not possible with most quantitative instruments, such as surveys – because once survey questions are set, all are asked in the same way for all respondents. In contrast, grounded theory allows the researcher to have greater flexibility in gathering relevant data. Theoretical sampling continues until the point of saturation, when all concepts are well defined and explained.

In this way, the research process feeds on itself as continual redesign, and Corbin and

Strauss liken the researcher to a detective, who follows the leads of promising concepts

43 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION as clues. Figure 4 illustrates the iterative process of data collection and analysis in grounded theory methodology.

INTERVIEW Primary FIELD NOTES # ___ Role? Director / Program DARPA technical Manager area / ~~tor (office)? / Other: Start date f/’S-~, /f’1~, with DARPA: ~5S ~ Primary Reports ~7 location? (U.S. to: ~/~w~5 Contract state) __ 13< ~ II length: ~Gender? ~ M # of __ Projects: Age bracket? ______4/ ~2J2 / 21-30 Team size:

______

____

~W~ # / 41-50

5/J / 51-60 ~ ~- /

~ 61-70 /≥2.Q

‘I ~ ~ ~ Tr~2 FIELD NOTES~ ~ INTERVIEW - •j~I~ ~ ~ ~ /~f

/ (Wh~r~ ~

I) I / Contractor r

Manager ~___(~4

______Pro~r~rn state)

~

(U.S. ~1 j7’v2f~

Role? Primary location? -~1Y~ ~

~ ~ Primary ?°~ 7I~

~ fC~71~

C~7

1~ ~ 741,4C_{ I ~ ~ r rJ~1~ (office)? ~

technical area ~Ic -

DARPA ~ Contract length:

r~ ~ I ~ rT~2V~

5~ ______~

/ 9’~’ ~ 1~S~

with DARPA: Teamsize:

date _____

Start - /k~- ~i)) ~

#ofProjects: mA~t~1 /~) ~ I~ ~,(/ 61-70,~ to: ~ / 51-60

Reports 31-40 / 41-50 ~t~Jci2~

/ 21-30 / ~

bracket? ~2J2 ~t~-~cy ~

Age ~r1~47~~/ ~ ‘I F ~ t~5~j~~)2~ Genderi~/ ~ (7~’ /e~’~ ~ ~ ~ ~1 4~€L~ Ic ~ (k~’e I, 7~ ~A~PPr Lv1~tt ~~‘k ~~ - 4~s~ ~ 5~fr (mi~) ~ y’k~~ ~ Pm ~i~e /~ ~ ~J. L~ ~4kr, ~:~(

~ ~efr /i~s

~ 75 /~ ~

~ ~d ~ h~. I)

~ F’ ~ ~ )~Y ~ ~ ~ ‘~‘ 2 ~ 4 ~ C’ ~

~ ~

~ frt~C-( ~ ~ / ~- /~/~

~ €~,, —J,~JJ(P 7~

~ o,v

~iP~ w~ ~ ~ ~ /

‘~

~x ~

~r~7 ~ /

1’ ia’1 ~ I ,1 / ~ ~ ~ ~ A~1-~11~ ~ ~ ~hi~t/9~P4-,&~ ~

~4~-d ~

fl4~Qf/ /~ ~ ~ li 11~ ‘I ~ ~t~J

(‘~I-€~ ~

j~tf d~- c,~4 ~

1~ ~ pp~~ /~‘~“~ ~ - 1,~ ~‘

Ci~—&o( 4~-

— ~iW4 ~z~LU~

I ~

~ 5o~,

;.-~. ~ ~7-~ ~fr /fl~) ~

/t ~ T~7J~)

~rf ~ ~

~ ~ ~ 7 y ~ jJQ~ ~

~ (~ )~ ~j ) /~ ~ ~ ‘~‘ ~ ‘- ~- ~ ~ ~ I, ~ -

!"##$%&'()&)' !"*$'()&)' ! "#$%&'()*$(+,-*+./)0$%) ! "#%+)%0(09)(,0$/',*1(/)0$%) '01(&,+)2+3%)$#(+/)) %#'&5+$(/) ! 4,*(+)2+3%)5+5#/)(#) ! :,#'+//)#;)<(=+#,+6'03)/0513*$8>) 0,6'&30(+)$+.)1,#1#/*6#$/) (#)*%+$6;?)@+?)*$%*'0(#,/)&1)(#) ! 7+-+3#1)%*08,05/)0$%)5#%+3/) (=+)1#*$()#;)%0(0)/0(&,06#$)

Figure 4: Iterative process of data collection and analysis Grounded theory methodology relies on an iterative and systematic process of data collection and analysis, and each cycle of data analysis determines the next cycle of data collection.

Method Limitations

As with all methods of research, grounded theory methodology has limitations that can be easily minimized during a research study (Corbin & Straus, 2008). First, careful training and strong attention to detail can help prepare the researcher to conduct interviews and record observations completely and accurately. Considerable amount of training and skills is required in collecting qualitative data appropriately. In these studies,

44 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION the investigator is the research instrument. He or she conducts the interviews, records the observations, and then analyzes and interprets them.

Second, ingrained beliefs and cultural predispositions are often hard to escape, and the researcher may bring a potential bias that influences the direction of data analysis. Therefore, a researcher can help minimize bias by keeping a personal journal along with the field documentation and being relentlessly reflective throughout the process of data collection and analysis.

Third, there is a risk that deficient coding categories support the final theoretical argument. Researchers trained in grounded theory methodology become so familiar with the data that they rigorously examine it at each cycle of data collection, and the constant refinement process removes the likelihood of incomplete or subjective analysis. Corbin and Strauss (2008) suggest relying on additional data sources to verify the coding categories to further strengthen the analysis. In this study, historical interviews and archival documents help to crosscheck interpretations in the primary interview data.

Conclusion

As a theory-building approach, grounded research methodology provides the framework for data collection and analysis in this study. Grounded theory methodology is valuable when studying new or uncharted areas of research in which little theory has been developed. Data collection begins with observations and interviews in the field, which are then coded in categories. As part of coding, field memos are written to reflect on and synthesize emerging insights. This process of analysis determines the next cycle of data

45 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION collection, and through theoretical sampling, the analysis is ultimately refined until the point of data saturation. The next section describes the data set in detail.

46 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION

CHAPTER 4

DATA SET

This study was designed as exploratory research, intended to generate a plausible and compelling new theory, as well as thick description about a particular phenomenon.

Primary data was gathered through interviews and further supplemented and validated through historical interviews and documents. Research protocols are described in detail below.

DARPA is a good data set for at least three reasons: predefined scope, long timeframe, and a clear unit of study. The first reason relates to organizational clarity. As described in the previous chapter, the agency’s entire charter is predicated on radical innovation, which DARPA defines as “high risk, high reward”. All efforts from planning to application undertaken by DARPA personnel fall under radical innovation, based on the agency’s own interpretation, which removes the need to filter conflicting or unnecessary activity. In a study commissioned by DARPA, Van Atta and his colleagues

(2003a) state: “Vision, the primary role of DARPA management, involves conceiving, developing, and demonstrating disruptive capabilities… Perhaps the most important effect of DARPA’s work is to change people’s minds as to what is possible” (p. S-12).

Subsequently, studying DARPA provides a clear and predefined scope for radical innovation.

The second reason is an extended timeframe, and DARPA provides a repeated process of radical innovation. The agency demonstrates over 51 years of sustained and successful practice, evidenced by multiple new and different technologies built over time

47 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION

– including the Internet, the global positioning system (GPS), virtual reality, and stealth airplanes. In terms of national impact, DARPA (1998) reports that nearly every current military system has been affected to a substantial degree by their work. In terms of business impact, 33% of dual-use projects were already selling their technologies or products in the commercial market (Richardson et al., 1999). In all cases, an assumption can be made that organizational processes are both successful and consistent across the different eras within DARPA’s history.

A third reason is that it is relatively easy to identify the unit of study, which is the

DARPA program manager and his/her program vision. Each DARPA program manager manages a small number of programs, and each program has one vision that results in a formal document called a Broad Agency Announcement (BAA).

Two questions may arise when selecting DARPA as an organization to study: Is

DARPA actually innovative? And does DARPA’s work in technology generalize outside the realm of national defense? It is true that DARPA program managers do not build the innovations with their own hands; instead, based on their personal visions of technology, they define, fund, foster, and facilitate innovation with the help of others. As an organization, DARPA is critical to the overall lifecycle of innovation because many new engineering ideas and their related implementations may not have been developed as rapidly or invented without the agency’s support. Van Atta and his colleagues (2003a) explain DARPA’s role as an accelerator: “DARPA acted as a catalyst for innovation by defining research programs linked to DoD strategic needs, seeding and coordinating external research communities, and funding large-scale demonstrations of disruptive concepts” (p. S-10). Some critics might constrain DARPA’s influence to invention only,

48 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION arguing that DARPA funding recipients deliver singular prototypes. This is an incorrect view. In actuality, there is both intent and action to commercialize. DARPA program managers are expected to guide all their projects toward military and commercial applications and, when ready, transition the technologies to the appropriate users. Thus,

DARPA technical staff demonstrates an unmistakable understanding of their role in the broader innovation lifecycle.

Lessons from DARPA also extend past the realm of defense. Although DARPA- funded projects are largely designed for military use, scholars such as Bonvillian (2009) point out the shared relationship between innovation pursuits in the commercial and defense sectors. He writes, “Defense technology cannot be discussed as though it is separate and apart from the technology that drives the expansion of the economy – they are both part of the same technology paradigms” (p. 207). Both types of technology ultimately contribute to societal wealth and economic growth in the same economic system. One national newspaper made a similar point to its readers: “While many military technologies go on to find civilian applications, one might say that this is especially true of DARPA’s successes – the agency is perhaps already quite interested in new technologies as much for their general coolness and applicability as for their military potential” (Page, 2009). Returning to two of DARPA’s many innovations, today’s world of GPS in your hand and Internet everywhere refute this claim.

Data Sources

In this study, primary research was gathered through in-depth interviews.

Interviews offered a way to hear directly from subjects and gain further insight into their

49 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION motivations and beliefs about their experiences at or with DARPA. The assumption was that personal stories would provide meaning and context for behavior. Generally, interviews allow greater depth for research questions where knowledge is often taken for granted, not readily articulated by most members, and/or subjected to multiple interpretations and perspectives – all of which hold true at DARPA – than other research instruments.

Interviews were conducted over a three-month period from September to

November in 2009. Subjects were recruited through purposeful sampling, in which subjects are selected based on a particular characteristic. Subjects were first screened to confirm that they were related to DARPA in one of three groups: DARPA personnel,

DARPA funding recipients, or staff who worked with DARPA in related R&D roles

(e.g., director of innovation at the Office of Naval Research). Subjects who were DARPA personnel worked either as an agency director, office director, deputy director, or program manager at the organization.

The study’s sample was then expanded through a snowballing process, a research technique that relies on existing study subjects to suggest future subjects from among their personal network. (As the term indicates, the sample group appears to grow like a rolling snowball.) This sampling technique is useful when hidden populations are difficult for researchers to access. In this case, previous studies and knowledge about

DARPA demonstrated that program managers and funding recipients are tightly connected through closed networks. As more subjects were recruited, they were deliberately selected to reflect different offices and points in the agency’s history in order to maximize population diversity. To avoid a sampling bias, a broad net was cast to

50 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION recruit subjects who came from different industries, served in a range of DARPA program offices, and lived in different U.S. regions.

Conducted by phone or in person, interviews were semi-structured in format, following standard protocols (Seidman, 2006). Interview questions were drawn from themes within the literature review and included a mixture of open-ended and specific questions, designed to elicit both foreseen and unexpected information. For example, subjects were asked to explain the concept of “DARPA Hard”, which is described in detail in the next chapter, and their experience with any efforts pre-program. Following grounded research methodology, interview questions were further refined in each data cycle to gather additional evidence to clarify and/or refute potential propositions.

Interviews averaged 60 minutes in length, totaling 2632 minutes or approximately 44 hours of interview time. All responses were treated as confidential and anonymous. Field notes were also taken during the interviews and provided 157 pages of raw documentation. When possible, interviews were audio recorded with each subject’s permission; however, not all interviews could be recorded.

An additional 12 interviews were drawn from an oral history database maintained by the Charles Babbage Institute (CBI), a research center dedicated to promoting the study of the history of computing. The CBI interviews were conducted from 1983 to

1993 with key figures in DARPA’s history, including several individuals who have since passed away, such as J. C. R. Licklider (a well-known DARPA office director, who provided the initial vision for the Internet) and Charles Herzfeld (a DARPA agency director). Running 475 pages, the transcripts from the CBI interviews helped supplement and validate the emerging themes in the primary interviews. A coding process similar to

51 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION the primary interviews was applied to the analysis of these oral history transcripts. An example excerpt from a CBI historical transcript follows:

Data Example: Excerpt From an Historical Interview Transcript

“One thing that stands out was there was a large number of helper organizations that had been hired to help design, run and evaluate programs. A lot of money went into that, and we were under constant pressure to explain what were we doing with all these helper organizations. So we finally put together a sort of master plan about who does what, and who is good for what, at what, and so on. We had all that worked out. Then there was a kind of built-in natural, but I think very appropriate, tension between the people who were thinking in terms of systems, people who were thinking in terms of the technology and science, and people who were thinking about the military utility.”

Documents from DARPA’s public archive and personnel provide a third data source. These documents include program technology reports, reflections from the agency’s 50-year anniversary commemoration in 2008, and various agency symposium proceedings. Similar to the CBI oral histories, the documents helped check the language and details provided by DARPA personnel in the primary research interviews. An excerpt from DARPA archival documents follows as an example:

52 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION

Data Example: Excerpt From a DARPA Document

“DARPAʼs mission implies one imperative for the Agency: radical innovation for national security. DARPAʼs business processes reflect this in a straightforward way: bring in expert, entrepreneurial program managers; empower them; protect them from red tape; and quickly make decisions about starting, continuing, or stopping research projects. To maintain an entrepreneurial atmosphere and the flow of new ideas, DARPA hires program managers for only 4 to 6 years because the best way to foster new ideas is to bring in new people with fresh outlooks. New people also ensure that DARPA has very few institutional interests be- sides innovation. New program managers are willing to redirect the work of their predecessors – and even undo it, if necessary. Since program managers are not at DARPA for a career, they are willing to pursue high-risk technical ideas even if there is a reasonable chance the idea will fail.” (DARPA, 2009, p. 1)

Subject Profile

A total of 47 primary interviews were conducted, which comprise 24 interviews with DARPA personnel, 18 with DARPA funding recipients, and 5 with related R&D roles. Related R&D roles included, for example, the director of innovation at the Office of Naval Research and the deputy director of the National Science Foundation, both of whom have worked closely with DARPA in their careers.

The supplemental CBI interviews comprise 10 interviews with DARPA personnel, one with a DARPA funding recipient, and one with a related role. As a side note, five of the CBI transcripts are with subjects who were interviewed in this study. For example, Robert Taylor was interviewed in 1989 by CBI staff and then again by this researcher in 2009. These historical interviews provide an earlier capture of similar concepts, further validating each individual’s comments and perceptions about DARPA captured in this study.

Figure 5 presents the combination of primary and historical interviews, which total 59 interviews in this study.

53 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION 3%2%+&-4'*+&,-./+012'506789'' :;'<+1%=(&;' "'# (&# ('# !"#"$%& $'# !"#$%&"'()*"+$,&-",.* !&# /&"0(&1*"+$,&-",.* !'# $# $&# !"# $'# $%# !"#$%&'()'*+&,-./+012' &# $# &# '#

)*+,*#,-./011-2# 3415617#+-8696-1:/# +-2;:-5#+<)#+02-/#

Figure 5: Research interviews (n=59) by category This study draws on a total of 59 research interviews, which includes 47 primary interviews and 12 historical interviews categorized in three groups: DARPA personnel, funding recipients, and related R&D roles. Five of the historical transcripts are with the same subjects in the primary interviews.

DARPA Personnel

A total of 34 interviews were with DARPA personnel. The roles for DARPA personnel were typically fluid for subjects. It was common for a subject to begin as a funding recipient and then be recruited later as a DARPA program manager.

Alternatively, a program manager could transition into a new internal role, such as an office director or deputy director, or return to the agency as a program manager for another term at a different point in the future. When removing the five historical interviews conducted with the same subjects later, approximately 66 percent of the overall group had been program managers.

Subjects held a total of 38 different roles at DARPA. Six subjects served in multiple roles, such as first as an office director and later as a deputy director. Subjects

54 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION who may have returned for another term as a program manager were not counted because they held the same role again. Figure 6 shows the specific breakdown of agency roles, which accounts for multiple different roles per individual. It was valuable to hear from multiple roles within the organization’s hierarchy for two reasons. First, each role influenced pre-program efforts differently and could provide greater context. Second, the senior roles of office director, deputy director, and agency director could help verify the various accounts from the program managers.

*+,-+'-%&.(//%0'1/2345'' 67',(0%' '&# !"# !)# !%# !(# !'# !&# "# "# )# %# %# (# $# !"#$%&'()'*+,-+'-%&.(//%0' '# &# *+,-+./# 3452# 629:8;# <-215;# =2>.82?#=,>2# 0.1.-2+# 67+258,+# 67+258,+# 67+258,+# *+,-+',(0%'

Figure 6: Different roles held by DARPA subjects (n=29) Accounting for the overlap of five interviews in the primary and historical interviews with the same subjects, a total of 29 DARPA subjects worked in a range of functional roles. Some subjects served multiple roles at the agency; for example, a program manager might continue as an office director.

55 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION

Subjects spent an average of 6.7 years with DARPA, which ranged from one to 13 years. This average tenure is longer than a typical contract, noted as 3-4 years, because several subjects transferred or returned to DARPA in other roles. For example, a program manager recruited as an office director would serve longer. Figure 7 presents the distribution of roles held by DARPA subjects over time in this study, which total 42 roles. In some cases, a single subject may have held multiple roles; for example, a program manager may have returned as an office director. This distribution shows that accounts were provided for all decades in DARPA’s history. In addition, accounts were continuous; instead of hearing about a one-time event, subjects could describe multiple activities over a given timeframe.

56 THE VALUE OF VISION!"#$"%$&'()**&+%,*-./0% IN RADICAL TECHNOLOGICAL INVENTION 12(&3%)*%"4&*56%#)+&%

563+%012+3.42% *+,-./%012+3.42% 563+%012+3.42% /01%2$3) 563+%012+3.42% *+,-./%012+3.42% !"#$%&'$()*"+,-.) <8+;3/%012+3.42% 724829:%:9;98+2% 4'5#0%'267)*"+8.) 724829:%:9;98+2% <8+;3/%012+3.42% 724829:%:9;98+2% 724829:%:9;98+2% 724829:%:9;98+2% *+,-./%012+3.42% 724829:%:9;98+2% 563+%012+3.42% <8+;3/%012+3.42% 724829:%:9;98+2% 724829:%:9;98+2% 563+%012+3.42% 724829:%:9;98+2% 724829:%:9;98+2% 724829:%:9;98+2% 563+%012+3.42% 724829:%:9;98+2% 563+%012+3.42% 563+%012+3.42% 724829:%:9;98+2% 724829:%:9;98+2% 724829:%:9;98+2% 724829:%:9;98+2% 724829:%:9;98+2% 724829:%:9;98+2% 724829:%:9;98+2% 724829:%:9;98+2% *+,-./%012+3.42% 724829:%:9;98+2% 563+%012+3.42% 563+%012+3.42% *+,-./%012+3.42% *+,-./%012+3.42% 563+%012+3.42% *+,-./%012+3.42%

!"#$% !"#&% !"'$% !"'&% !"($% !"(&% !""$% !""&% )$$$% )$$&% )$!$%

Figure 7: Distribution of roles (n=42) held by DARPA subjects over time Interviews with DARPA subjects span across the organizational life of DARPA, providing detailed accounts for all decades in DARPA’s history. DARPA subjects interviewed for this study held a total of 42 roles. In some cases, a single subject may have held multiple roles; for example, a program manager may have returned as an office director. Roles held by DARPA subjects from the primary interviews are coded in red, and DARPA roles from the historical transcripts are coded in blue.

57 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION

Funding Recipients

DARPA funding recipients spanned three main groups: universities, research labs, and corporations, all of which are depicted in Figure 8. Not one group dominates the data sample. Interviews from funding recipients helped validate the various accounts from

DARPA personnel, particularly program managers, who potentially could have emphasized certain details in recollection. 345*4'6"07.08'5%-./.%012'90:;<='' >?'@&8+0.A+,(0+B'C?/%''' %"

!" !"

#" #" #" !"#$%&'()'*+&,-./+012'

$" &'()*+,(-*," .*,*/+01"2/3," 45+65+/-5',"

Figure 8: DARPA funding recipients (n=19) by organizational type DARPA funding recipients span three main groups: universities, research labs, and corporations. This figure shows the organizational affiliation of the subjects in this study.

Based on the length of their primary engagements with DARPA, the interviews with funding recipients covered the majority of DARPA’s history from the early 1960s to recent years, which is depicted in Figure 9. This group averaged 17 years of financial support from DARPA.

58 THE VALUE OF!"#$%"&'()*+),+!-./-+0)*$%12$)%+ VISION IN RADICAL TECHNOLOGICAL INVENTION 3*$4%5"46#+71#48+)*+-94*2:+;4*'%4+

3456+.,578% 9:.;:.-<:4% 9:.;:.-<:4% 3456+.,578% 9:.;:.-<:4% 9:.;:.-<:4% 3456+.,578% *+,+-./0%1-2% 3456+.,578% 9:.;:.-<:4% *+,+-./0%1-2% 3456+.,578% /01%2$3) 9:.;:.-<:4% !"#$%&'$()*"+,-.) 3456+.,578% 4'5#0%'267)*"+,.) *+,+-./0%1-2% *+,+-./0%1-2% *+,+-./0%1-2% *+,+-./0%1-2%

!"#$% !"#&% !"'$% !"'&% !"($% !"(&% !""$% !""&% )$$$% )$$&% )$!$%

Figure 9: Distribution of funding recipients (n=19) based on primary DARPA engagements over time Interviews with DARPA funding recipients cover the majority of DARPA’s history from the early 1960s to recent years. This group averaged 17 years of financial support from DARPA.

DARPA Office Affiliation

A total of 44 subjects were affiliated with a specific DARPA office, in contrast to general management positions. This number includes subjects who served as program managers, but excludes agency directors and deputy directors who oversaw multiple agency offices. Figure 10 shows the distribution of DARPA personnel and funding recipients across various DARPA offices. Approximately 57 percent of these subjects are affiliated with DARPA’s Information Processing Technology Office (IPTO), which was expected due to the snowball sampling, as well as the computing focus of the Charles

Babbage Institute. The other 43 percent were distributed across multiple other offices,

59 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION which provided an additional check of internal validity that processes reported within

IPTO were consistent*+&,-./+01'489.+,(0':0;<<='' with processes reported from subjects in other DARPA offices.

>?'345*4'78-%' %&"

$'" !"#"$%& $&" !"#$%#&'()*%+%)#,-' #%" ./(0/'0)1-2##)3' #'"

#&"

!"#$%&'()'*+&,-./+012' #$" '" $" $" !" #" !" $" $" #" &" #" ()*" +,-*" .-*" )-*" --*" */012" 345*4'*&(6&+#'78-%'

Figure 10: Subject affiliation (n=44) by DARPA office A total of 44 subjects were affiliated with a specific DARPA office, either as an employee or funding recipient. Approximately 57 percent of subjects were affiliated with one office, and the remaining 43 percent were distributed across multiple other DARPA offices.

Conclusion

The study draws on 47 primary interviews with DARPA personnel, funding recipients, and related R&D roles. The subjects’ time with DARPA span the entire life of the agency, allowing commonalities to be found across agency eras and offices. An additional set of 12 interviews from historical transcripts was used to validate the primary interviews and temper any potential interviewer bias. Agency documents were also reviewed as a secondary source of data. The results of this data set are discussed in the next section.

60 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION

61 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION

CHAPTER 5

ANALYSIS

Four major research findings are derived directly from the data. First, the process of radical technological innovation starts with vision. All subjects in the study who were

DARPA personnel reported starting their work with a big technology idea that then led into an executable program. In all cases, the visions were characterized as “DARPA

Hard”. Second, subjects refined any incomplete technology visions through interaction and iteration. Consistently throughout DARPA’s history, subjects described two main mechanisms, specifically expert workshops and proof-of-concepts, to advance partial visions into clear visions. Third, the skills for vision development are learned largely through socialization. Subjects reported that DARPA does not provide training for new program managers, expecting them instead to learn through experience and informal mentoring. Fourth, the approval of technology visions excludes majority opinion. The practice of consensus is actively discouraged in the DARPA vision approval process. The subjects described direct reviews that allowed them to take their technology visions forward rapidly. Additional findings are discussed in light of the four major findings in this section.

Process Model of Radical Innovation

After reviewing the data, a high-level process model of innovation activities at

DARPA emerged, which is depicted in Figure 11. The process presents five stages, which generally follow a linear progression starting with Recruitment and then leading to Vision

Formulation, Program Launch, Portfolio Management, and finally Technology Transfer.

62 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION

The five stages were identified from subjects’ recollections and descriptions across all eras and program offices. Each of these stages is described below.

+&,&-)* 2$-3$0(* 2-$6-/&-* 8"#5)-/-39* !"#$%&'(")'* .-$(%/01-)* 40%)#5* 70)03"(")'* 8$0),:"$*

Figure 11: Process model of radical innovation at DARPA A high-level process model of innovation activities at DARPA emerged from subject accounts, spanning five stages from Recruitment to Technology Transfer.

Stage 1: Recruitment

Recruitment refers to efforts in identifying and hiring talented candidates, mainly as program managers, to work at DARPA. Technical personnel are hired explicitly with new and daring ideas of innovation, and this stage tightly links people and ideas. Subjects mentioned campus visits and networking as common activities during this phase. The end result of this stage is a new hire for a program manager.

Stage 2: Vision Formulation

Vision Formulation is focused on the initial work that new programs managers do when they first join DARPA. Some program managers arrive with a clear and long view of their programs, while others start with partial ideas that they then refine quickly through different activities, such as workshops and proof-of-concepts, the latter of which were called “seedlings” under the directorship of Tony Tether. Some may consider this step as a process of envisioning. Towards the end of this stage is a Broad Agency

Announcement (BAA), in which each vision is internally formalized, documented, and

63 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION put forward for approval as a new program. The final step in this stage is the approval of the BAA for external distribution.

Stage 3: Program Launch

Once a program is approved, DARPA program managers proceed next to a stage of Program Launch, in which they publicly announce and start a new program through an official solicitation procedure. In the early days of DARPA, program managers would solicit bids or request participation on a more ad hoc basis. After 1984, Congress required the selection and evaluation of DARPA performers, those responsible for exploring and implementing a particular vision, to be handled formally through the BAAs. A BAA is different from a Competitive Request for Proposal (RFP) procedure, a more common document within the U.S. federal government that is related to the development of a specific system or hardware procurement (DARPA, 2010a). The end result of this stage is the final selection, funding, and orientation of team(s) external to DARPA to explore and build the vision. One DARPA program can have multiple projects being executed by different teams.

Stage 4: Portfolio Management

In the stage of Portfolio Management, DARAP program managers oversee the development of all active team projects within a new program, as well as end any unproductive projects that do not meet the original vision or offer meaningful learning to the organization. This stage emphasizes the practical management skills of a DARPA

64 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION program manager, who handles project evaluations and budget reviews for all his/her technology investments, typically on an annual basis.

Stage 5: Technology Transfer

The final step is Technology Transfer, in which program managers transition promising technologies, often as a functional prototype, to early adopters in the military services or relevant industries. Memorandums of Understanding (MOUs) and user testing procedures are frequently part of this stage. Although not all DARPA projects end with technology transfer, this step is considered the critical closure in “bridging the gap” – borrowing the slogan of DARPA’s 50-year anniversary campaign – between far-reaching ideas and near-term solutions.

Table 3 summarizes several common processes that occur at each stage in

DARPA’s process model, as well as presents multiple supporting quotes from the interviews that are representative of the broader data set.

65 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION

Stage Example Process Example Quotes From Research Interviews Recruitment • Campus visits • “If youʼre going to foster innovation, you should • Mentoring get smart people.” • “The hiring process becomes an important part of the process.” • “Go find good people; go do good things.” • “DARPA is shaped by people. Its content is a function of the people.” • “There is a kind of networking. You learn to trust certain people, and they expand your acquaintance.” • “It was literally all word of mouth.” • “I teamed up with members of my senior community; that was the recipe then.” • “My bias was toward very bright people. I did not care very much what they knew or what they were interested in.” • “Well, certainly when I arrived it was, as I mentioned before, an environment in which you gave money to good people and expected good results.” Vision • Workshops • “I came with a technology vision.” Formulation • Proof-of- • “In some sense, everything I knew I got either concepts from thinking myself or from visiting labs and talking with people.” • “You have to have a new unique idea.” • “I sure as hell knew what I wanted to do.” • “When I came in, I had a couple of ideas about things I was interested in… but after I got there, I actually changed course entirely.” • “Most of the programs I eventually got started ended up because I saw this idea over here, and I saw that idea over there, and if I put them together, we could really do something incredible.” • “We were constantly inundated by ideas, some good, some not – lots of good ones. So this was a very lively time. People would come in and out all the time with new ideas. And there were indeed many places where they could sell these, but in the end they would come to us. So there was a great traffic of ideas, flow of ideas.”

66 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION

Stage Example Process Example Quotes From Research Interviews Program • Industry Day • “I am sure you have heard the fact that we could Launch • Contract get an idea in the morning and have the guy negotiations working under a Letter of Intent by 4:00 the next day.” • “One thing that I learned early on, and I was doing back in the 90ʼs, is that when you do a project you had to have a contracting officerʼs representative.” • “As written it looks like I had it all scheduled out, but there was a lot more of improvising and ʻoh my gosh, I have to do this now if I want anything to happenʼ in it.” Portfolio • Project • “I believe the most important word in our Management evaluations [DARPA] name is projects.” • Budget reviews • “Part of the DARPA recipe … is that projects are the output. They are the units of success and failure.” • “We donʼt want to own it; we actuate it.” • ”DARPA program managers really act like managers and manage the research." Technology • Memorandums • “I picked the government agent based on the Transfer of Under- application area, which helps with the standing technology transfer.” (MOUs) • “I consider myself a technology harvester.” • User testing • “Most of the people, 90% of the DARPA program managers, in my mind, are 100% clueless about technology transfer – they have no idea. I was very naïve about it, even when I thought I knew.” • “The research that we understand as revolutionary research is only 10% of the problem, and 90% of the work in order to get a new idea successful is after youʼve got the prototype, not before.”

Table 3: Common processes and interview quotes per DARPA innovation stage DARPA’s process of innovation follows five general stages: Recruitment, Vision Formulation, Program Launch, Portfolio Management, and Technology Transfer. Each stage is associated with a set of common processes, which were identified based on dominant themes from the research interviews.

Vision and Recruitment

Vision plays a central role in the first two stages, Recruitment and Vision

Formulation, starting the process of radical innovation at DARPA, depicted in Figure 12.

During these two stages, a vision for a radical technological innovation is identified,

67 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION defined, evaluated, and approved as a DARPA program. These first two stages in the process are tightly entwined because DARPA program managers are hired in part for their initial technology visions. Moreover, each DARPA program requires a technology vision, and this vision precedes any efforts for a bid solicitation or project team selection.

Simply put, a new DARPA program is not approved for funding until it has established a vision. One internal report indicates this inseparable combination: “DARPA insists that all programs start with good ideas and good people to pursue them; without both of these things, DARPA will not start a program” (DARPA, 2007).

+&,&-)* 2$-3$0(* 2-$6-/&-* 8"#9)-/-3:* !"#$%&'(")'* .-$(%/01-)* 4"5)&1-)* 70)03"(")'* 8$0),;"$*

Figure 12: Radical innovation starts with vision at DARPA During the first two stages, a vision for a radical technological innovation is identified, defined, evaluated, and approved as a DARPA program. These first two stages are tightly entwined because DARPA program managers are hired in part for their initial technology visions.

Each vision is associated with one person: a DARPA program manager, who serves as the vision’s champion. A program manager from the 1990s stated, “A technology vision is the way that I came in,” and similar words were echoed by a program manager serving a decade later, who said, “I came with a technology vision.”

All program managers have the sole responsibility for realizing their particular visions during their time at DARPA by pursuing and funding relevant projects. When their tenure ends, if any substantial work remains on a project, they may be able to hand it off to another program manager; however, the vision itself rarely transfers between program

68 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION managers. One exception was the computing vision behind the ARPANET project in the early 1970s, which engaged multiple program managers over an extended period. While the data does not describe a specific reason for this, various accounts from subjects support the interpretation that this vision was bigger than any one person.

At this point, it may be helpful to see some examples of technical visions that

DARPA program managers ultimately document in their Broad Area Announcements

(BAAs). The BAA serves as a formal and public description of the initial program vision.

The Augmented Cognition program described a vision for a new type of soldier helmet or cockpit, which then lays the groundwork for the field of empathetic avionics, among other areas (BAA, 2001). Another program, the Legged Squad Support System, described a vision for a new type of military pack mule (BAA, 2008). While DARPA program managers present their general requirements for a range of radical ideas, they look to the funding recipients to decide exactly how to build and execute on the respective vision.

Table 4 presents two visions as documented in BAAs.

DARPA Excerpt From Broad Area Announcement Example Prototype Built Program Augmented “The goal of the DARPA Augmented Cognition Cognition effort is to extend, by an order of magnitude or more, the information management capacity of the human- computer relationship for the 21st century warfighter. This will be achieved by developing and demonstrating quantifiable enhancements to human cognitive ability in diverse, stressful, operational environments. Specifically, this effort will empower one humanʼs ability to accomplish the functions currently carried out by three or more Boeing Phantom Works, individuals and will improve and enhance the AugCog quality of military decision-making.” (BAA, 2001)

69 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION

DARPA Excerpt From Broad Area Announcement Example Prototype Built Program Legged “The Legged Squad Support System (LS3) Squad Program is an effort to develop a walking Support platform, preferably a quadruped, which can System accompany dismounted soldiers and (LS3) increase their combat capability. LS3 is envisioned to augment squads by maneuvering with them in complex terrain where wheeled tactical vehicles cannot go, carry traditional squad equipment (in an effort to improve squad performance), carry new Boston Dynamics, BigDog squad equipment (in an effort to give new combat and sustainment capabilities to the squad), and do so in a self-controlled fashion (requiring minimal human interaction and control).” (BAA, 2008)

Table 4: A selection of DARPA program visions A DARPA program manager broadcasts his program vision in a document called a Broad Area Announcement (BAA), which officially launches the program and begins a public competitive solicitation procedure. The BAAs provide a public record of DARPA program visions, and a program vision may generate multiple and different prototypes from the funding recipients.

Recruiting Criteria and Evaluation

Program visions are vital to the recruiting stage, and an important finding in this study was that vision is considered during the hiring of program managers. A DARPA program manager is recruited primarily based on having an initial vision of technology, plus the ability to communicate that vision to different groups. An office director explained the focus simply, “We look for people with good ideas,” and another director felt an ideal candidate was someone who could “articulate what they want to do and why”. A program manager compared his role to a technology evangelist, saying,

“Sometimes I feel a little like Billy Graham”, because he was endlessly communicating and exciting others about his ideas.

70 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION

The contents of a vision and its presentation differed by personality; however, the common theme across subjects was that each vision be radical and actionable. In other words, each person described a technology concept that could be achieved within several years, potentially creating an important new application and user base (e.g., market) that had never existed before. One program manager explained this focus as a combination of vision and action:

I mean it’s a unique combination of strategic vision and tactical implementation that is required to be a very effective program manager. You can dream all you want, but if you don’t have the ability to tactically implement your dreams, then all you are is a dreamer. If all you do is tactically implement, then you’re just putting one foot in front of the other without picking your head up and looking to the horizon and understanding where you are going.

In nearly all interviews, DARPA personnel were technically trained with advanced degrees in engineering or science. DARPA does adapt a general boilerplate hiring description for its technical offices (M. Peterson, personal communication, June

10, 2010). However, no subjects described an official job profile, and instead they listed several gut criteria for finding the right program manager, including: technical expertise, visionary thinking, leadership, communication skills, and fiscal responsibility. A deputy director described his view of the hiring practices at DARPA, highlighting the importance of creativity coupled with technical knowledge:

We need people to be thinking big thoughts, really pushing the edge, but doing it in a rigorous way. It’s not just to say: I want to build a teleporting machine, exactly how are you going to do that. And why should we believe it is going to happen. So that’s what I mean by technical creativity, rigor and honesty. It’s creativity, but it’s tied to things that make sense.

A recent public job listing from DARPA (Defense AT&L, 2007) called for similar traits, highlighting the need at the personal level for a radical idea that generates lasting change:

71 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION

Job Listing for a DARPA Program Manager

Are you a scientist or engineer with a radical idea (or ideas) that you believe could provide meaningful change of lasting benefit for the U.S. military? Would you like to lead the countryʼs most capable academic and industrial experts to make that idea become reality in a period of just a few years? If so, you should consider joining the Defense Advanced Research Projects Agency (DARPA) as a program manager.

What is a DARPA PM? A DARPA program manager is ... An idea generator A technical expert An entrepreneur A visionary A patriot dedicated to national service

Generally, DARPA’s hiring process does not rely on specific evaluation tests, case interview formats, or personality profiling. With the exception of J.C.R Licklider who was known to assess graduate students by their test scores on the Miller Analogies

Test and Graduate Record Examination (GRE) in the 1970s (Aspray & Norberg, 1988), candidate evaluation has historically been more informal at DARPA. An office director explained, “It’s not a formal process. We don’t give them tests or anything.” Instead, a candidate is largely evaluated based on the relevancy of his/her technology vision, as well as cultural fit with the agency. Overall, subjects indicated that hiring visionaries has benefited DARPA in at least two ways. First, these visionaries open new areas of inquiry, which often do not exist before. Second, office directors can influence the broader defense research agenda by strategically hiring program managers with big visions.

Based on the research sample, a secondary finding emerges related to recruiting.

In the instances when subjects offered their ages when starting at the agency, a clear trend emerges that the average hiring age of DARPA technical staff has increased, particularly among its program managers, throughout the agency’s history. Presented in Figure 13,

72 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION this trend was focused more in the computing areas, particularly within DARPA’s

Information Processing Technology Office (IPTO).

'"# $$# $"# &$# &"# %$# %"# !"#$%&#'$()*#+$,-'$.#/*01$ !$# !"# ()'"# ()*"# ()+"# ())"# !"""# !"("# .#/*$23/*3#+$/3$4!56!$

Figure 13: Rise of starting ages for new DARPA personnel The data shows a preliminary trend that the average hiring age of DARPA technical staff, particularly among program managers, has increased throughout the agency’s history.

While subjects did not explain or address the issue, several subjects shared anecdotes that supported the general trend. Before the 1980s, they described hiring university graduate students and newly minted doctorates, who brought their energy and enthusiasm into the agency. One program manager said, “You took smart, young, energetic people and let them try.” Similarly, an office director said, “I even made an agreement with all these kids: I told them that if they wanted to keep on working on their Ph.D. thesis project, they

73 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION could do that up to 50% of their time.” More broadly, an agency director explained his focus on younger talent for the agency:

[We should] produce graduates of those organizations who were expert in the technologies that we were promoting… what I wanted were more Ph.D.s who could go out into industry, and particularly in the defense industry… Now that guy is a product of ARPA. That company is a product of ARPA’s activities, and it’s going to have an influence on the commercial enterprises that this country has.

In more recent years, much older and established professionals have been recruited to the agency.

Channels for Recruiting Technology Visionaries

How does DARPA find new personnel? The use of internal referrals has been the dominant vehicle for hiring new program managers. Office directors are often groomed from talented program managers or enlisted by former DARPA personnel. Subjects indicated the importance of highly specialized talent, which required other experts in the same area or field to qualify. An open job call would be unsuitable, and even the current agency website does not offer the standard career information or link to a hiring page.

Instead, DARPA expects potential recruits to be already embedded in the relevant R&D community. Repeatedly, stories were told in which the individual was part of a team funded by DARPA or working in a related defense area; subsequently, the current director or program manager could observe that recruit in action and then reach out directly. The following interview quotes all underscore the importance of leveraging existing relationships to find talent: “There’s a fair amount of relying on past connections,” “Every PM has some connection to a DARPA PM before they got there [to

74 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION

DARPA],” and “DARPA is always looking for people who’ve had previous interactions.”

In addition, former program managers tend to serve as volunteer talent scouts. A deputy director described the role of DARPA alumni:

They are people who can feed us – because they know what the job is like, they know what are the characteristics of a good PM and a good Office Director, or whatever. They can be the people who help us recruit the talent to the agency. That’s the way it has always been, and I think that’s the way we get the best people.

The alumni typically remain active in the extended DARPA community. After their contracts ended, many program managers and directors spoke about referring potential new candidates, informally mentoring new program managers, competing for DARPA funding, serving on technical advisory boards for DARPA projects, and consulting on different DARPA initiatives. One program manager explained, “You don’t really leave,” and another said, “It’s a fraternity… not an honor society.” (As a side note, historically, program managers at DARPA have been mainly men.) These close alumni relationships offer another value to DARPA’s model of open innovation because these relationships reinforce the network strength and emphasis on people. However, several subjects noted the limits of a network model of recruiting. For instance, one funding recipient observed that, “DARPA has always operated with a set of insiders.”

An average recruiting period could not be determined from the data. The recruiting period varied widely from weeks to years, heavily dependent on the recruit’s career schedule, as well as openness to working at DARPA. For example, a program manager said, “I was targeting and hassling people for three years.” For personnel coming from academia, DARPA often accommodates the constraints of tenure, and some

75 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION subjects noted that they joined the agency only after their faculty positions were secured at their universities. Ultimately, subjects noted that recruiting is about finding the right people at the right time, and in addition to being persistent, several of them described the importance of timing. One deputy director said:

Many times, most people think that science is just some linear progression, but a lot of times it’s like a startup. If you get the right group of people together at just the right time, great things happen. If you wait six months, the person who would have been the great integrator has moved on to another job and is no longer available, and you’ve lost that.

Vision Formulation

The second stage in DARPA’s model of radical innovation focuses on the first few months that a program manager spends at DARPA. Program managers indicated that they begin their roles with either complete or partial visions of technology, which then they must shape and pitch as a new program. In all cases, as one program manager put it,

“You have to have a new unique idea.” When the vision is incomplete, program managers have relied on several mechanisms to formulate and advance partial visions, and these mechanisms have been used consistently through the agency’s history.

Collectively, the visions from all program managers determine the focus of the agency. A program manager explained this shared direction: “The scope of what DARPA works on is a function of what its people works on, not a function of its physical facilities.”

Operating without labs or research space, DARPA directly relies on its people and their radical ideas to spur innovation efforts in the teams that they fund and foster.

76 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION

Sources of Technology Visions

A related finding has to do with where to look for possible visions. Although program managers are responsible for bringing a vision to DARPA, many credit the broader network of partners for inspiration. All of the program managers advocated being in the field and interacting with other researchers. They felt that this immersion allowed them to directly hear the latest thinking of the R&D community, as well as develop personal knowledge of the problem context. For example, a program manager said, “In some sense, everything I knew I got either from thinking myself or from visiting labs and talking with people.” Another program manager explained, “They are my ideas, but they are inspired by other people, people that I’ve seen. The more that you go out and the more different things that you see, you integrate them into what you think would be a great idea to do.” An office director was more blunt when he said, “The ideas don’t really come from program managers. They come from the researchers [funded by DARPA].” In short, program managers find that their ideas advance through community participation and interactions. The common theme was that new ideas occur more readily through two simple actions, when program managers leave the office and then talk with a diversity of people. A deputy director underscored these points with DARPA’s R&D community during a speech:

So how does the process work? Often it begins with you, a potential contractor, articulating your idea, your vision, in a conversation with a DARPA Program Manager, who, caught up with your enthusiasm, might help you flesh this idea out by finding funds to support you in a seedling effort, just to see where it might lead. More typically, he may try mapping what you want to accomplish against either his existing program portfolio, or incorporate your thinking with his own nascent ideas to create a broader plan, one that might capture the imagination of a whole community, one that might result in a new DARPA program. (Leheny, 2007, p. 1)

77 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION

Criteria of a Radical Vision

Another related finding identifies the criteria for a radical program vision at

DARPA. Although individual visions ranged widely in content, four general attributes for a program vision can be drawn from subject accounts. Radical visions must be technically challenging, actionable, multidisciplinary, and far-reaching. These attributes are summarized in Table 5.

Vision Attribute Description Technically Visions must extend beyond the current limits of Challenging technology and state of knowledge Actionable Visions are expected to be built and produced Multidisciplinary Visions draw on multiple experts and areas of knowledge Far-reaching Visions are ambitious about advancing society on a grand scale

Table 5: Criteria for a radical vision at DARPA Four attributes for a good DARPA program vision were drawn from subject accounts, and together these attributes characterize a notion of “DARPA Hard”.

First, at the core, all visions at DARPA constitute a challenge and must be almost technically impossible to achieve. The challenge is not based on the limits of a particular organization. Instead, visions must extend beyond the current limits of technology, and the current state of knowledge in the world for a given problem determines the difficulty of challenge. These problems are focused on technological breakthroughs. DARPA program managers described relying on multiple experts in the field, often across different domains of knowledge, to help them qualify true technical challenges.

Second, the visions must be actionable. The visions instill an expectation for action, and the right people can see a path to the impossible and can make progress

78 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION beginning today. Ultimately, program managers are responsible for delivering at least part of their idea during their time at DARPA, and all subjects underscored the pressure of this expectation. Successful program managers must be both dreamers and producers, even if they leave the implementation to the project teams. An agency director explained the importance of application: “We required what I called a pin thread of logic that got you from what you were trying to accomplish in each one of the programs that you were supporting to a capability that would make a real difference in the future.” This belief was echoed by other subjects, including a deputy director who noted, “Again it is not enough to say, ‘hey, I think it’s really cool if we were to look at this phenomenon.’ Why is it interesting from a capability perspective? That’s the most fundamentally important question [DARPA program managers] need to answer.”

Third, visions are multidisciplinary. The inherent problem in the vision is complex because it sits at the intersection of multiple disciplines. Several subjects noted this multidisciplinary aspect, in which the problems became so complex that they required a team approach. As subjects explained, achieving a good vision that would lead to radical innovation requires more than one person. If one person can build the vision, then the vision is not big enough. One program manager said, “I don’t remember a single case where we ever funded a single individual’s work.”

Fourth, good visions were far-reaching as a mix of long-range and broadly complex, which ultimately produced a broad impact on society and overall progress for humanity. In terms of long range, some subjects described efforts that simply required more time to pursue, often because certain advances in science and engineering research could not be skipped. In other cases, subjects described efforts that entailed integrating

79 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION multiple parts into a complete and complex solution, and while many of these parts did not have a long development lead-time in themselves, the sheer complexity of integration made the vision incredibly challenging to achieve. One office director spoke about making “an order of magnitude difference”. Another program manager described how he directly shortened the timeline for an ambitious program by increasing the available funding:

[DARPA management] asked me why [the problem] was so hard, and I told them why it was so hard… I just went ahead and had what I thought was a really cool idea, which would be virtual reality for simulation in for robot and we put them all together and put them on the switchable robot. That was something I thought was a really hard program, and I wasn’t sure that [the funding recipient] was going to be able to do it in the time or the amount of money that he said, so that’s why I doubled the amount that I wanted to make sure he would be getting. We went forward, and we dumped the money on him, and they performed admirably.

In sum, these four attributes of a good vision – technically challenging, actionable, multidisciplinary, and far-reaching – have become encapsulated into a term known as “DARPA Hard” within the agency. DARPA Hard is a recent term, coined during the directorship of Tony Tether (2001–2009). Subjects who worked at DARPA before the 1990s described the same concept and, in many cases, adopted the recent jargon of DARPA Hard because they felt it was apt terminology and evocative. Table 6 highlights several select quotes about DARPA Hard from the research interviews, as well as from the literature review.

In short, DARPA Hard is a daring technology challenge that cannot be solved easily, typically requiring new knowledge or invention to be truly addressed through bold measures. Subjects shared a sense of pride that the challenge was especially worthy of

DARPA’s attention and support. A deputy director stated, “Sometimes it’s a great opportunity, sometimes [DARPA program managers] can answer the why now, but they

80 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION can’t really answer why DARPA.” When program managers were able to address why tackling a given vision was genuinely DARPA Hard, they were able to move onto the next stage.

Source Example Quotes Research • “Some ideas in technology areas you really canʼt initiate unless Interviews youʼre here at DARPA because theyʼre seemingly impossible.” • “So they provided a leading edge where they addressed some hard problems early on. A lot of the knowledge that they gained from that work fed through the rest of the computer research community. A synonym for being the leading edge might be the lunatic fringe.” • “If it is successful, it will change by an order of magnitude.” • “I really didnʼt have a concept of DARPA Hard until the second or third year or so.” • “We were explicitly looking for things which, if they could succeed, would really be a large step beyond what technology could then permit.” Literature • “DARPAʼs approach has often been to take on projects that seem Review impossible, and the term DARPA Hard – now a part of the agencyʼs legacy – refers to technology challenges considered too difficult for other parts of the militaryʼs research and development laboratories.” (Weinberger, 2006, p. 4, italics included) • “DARPA focuses its investments, therefore, in an area referred to as ʻDARPA Hard,ʼ a set of challenges that, if solved – even with a solution that initially poses a high risk of technical failure – will be of enormous benefit to U.S. military capabilities and to national security.” (National Research Council, 2005, p. 13) • “DARPA prides itself on addressing only the most difficult ʻDARPA-hardʼ problems, the ʻgrand challengesʼ in technology, by nurturing the science needed to meet them.” (Weiss & Bonvillian, 2009, p. 163)

Table 6: The concept of DARPA Hard Multiple quotes drawn from the research interviews and the literature review about innovation show a coherent understanding about the concept of DARPA Hard.

Advancing Partial Visions of Technology

Visions may start as either clear or partially formulated at DARPA. Many program managers arrive with an unambiguous vision of they want to do and spend the

81 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION initial months at DARPA finessing program mechanics and budgets, so they do not feel the need to spend much time clarifying or validating their ideas. What does a clear vision look like? Several subjects described their visions very simply, and examples include swimming robots, portable ultrasound devices, nonstick paint, direct robotic surgery, microsystem fabrication at the nano scale, artificial blood, and a reusable, rapid-strike hypersonic weapons system. In the 1970s, J. C. R. Licklider introduced one of the more well-known DARPA visions that took on a life of its own, ultimately inspiring the invention of the Internet:

The thing called “Man Computer Symbiosis”… No, that didn’t come out of any particular research. That happened frequently: you do a lot of work, you get in a position to see some relationship or make some decision. And then it was obvious. In fact, even before you could quite get finished, you knew how it was going to come out.

Years later, a program manager described a similar process, in which the vision was evidently clear, as well as defined by aspects of DARPA Hard:

So about two, three years into the MEMS program, there were some people who said you know you can make Radio Frequency components from MEMS. You can make oscillators, you can make detectors, you can make mixers, modulators. Now we can make whole types of RF systems using MEMS. Well, I wasn’t convinced you could do that, and so I said, well what’s the hardest thing you can do in RF MEMS was build some high quality resonators.

When program managers felt that their visions were clear, they quickly moved to pitch these visions internally as new programs and secure funding. Little time was spent during the Vision Formulation stage.

Although other program managers felt that they had a good understanding of the research limits and market demands to their roles, these managers had not defined their visions of technology as clearly as their counterparts. Instead they worked more from a gut feeling or half-formed concept, trusting their intuition that they were pursuing the

82 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION right path. Program managers used descriptors such as “informal”, “concept”, and

“demos” to describe their early ideas. Their language indicates partial visions, or visions that required some additional effort to finalize to a more advanced state of readiness.

Program managers described two mechanisms to develop their partial visions into a complete and clear vision, which is illustrated in Figure 14. These mechanisms are expert workshops and proof-of-concepts, which have consistently been applied by program managers throughout DARPA’s history. Program managers and other DARPA personnel described no other methods for refining their visions.

./0#%1& 2*%3)4*0& ,$%-$"& !"#$%&& '()(*+& '()(*+& ,%**56*56& !*+7#01&8#9*&

:##;<$73&=&>#$%+(+?&

Figure 14: Formulation of partial visions of technology for radical innovation Expert workshops and proof-of-concepts have consistently been applied by program managers throughout DARPA’s history to refine any partial visions into clear and complete visions.

Expert Workshops

Expert workshops are invite-only, small group discussions between the program manager and key members of the R&D community, planned over 2-3 days offsite, that emphasizes the exchange of ideas and the potential demonstration and application of a new technology. The objective is to gather feedback about a particular topic, its potential technical issues, and overall direction. Participants are not paid for attending the event,

83 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION and primarily, only experts who specialize in a particular problem domain are invited.

Program managers were adamant about the value of a workshop, saying: “The first thing

I did was a workshop,” and “I did two to three workshops over different subjects to understand the state of the art.” Several funding recipients echoed this point, and one subject said, “The program manager is trying to come up with a germ of an idea.”

Proof-of-Concepts

At DARPA, proof-of-concepts are small design studies that explore and pre-test a particular technology concept. Proof-of-concepts may result in written documentation, physical prototypes, or simulations that confirm initial assumptions. Generally, a DARPA program manager provides a set of early requirements to a specific R&D contractor, who then undertakes the project as a paid assignment. These proof-of-concepts were dubbed

“seedlings” during the agency directorship of Tony Tether (2001–2009) based on the germination of an idea.

Program managers described efforts for proof-of-concepts that took several months to conduct. The outcome may not be clear, and a program manager explained,

“It’s a fishing expedition in advance of a program.” Comments from funding recipients matched descriptions from DARPA program managers, and one funding recipient stated,

“The aim is always the same: help the PM to do a successful pitch or write a good BAA or both.”

The process is intentionally emergent. Another program manager explained his approach to proof-of-concepts, while visiting teams in the field:

As I was walking down the corridors visiting the lab, I saw a 3-D image that looked like an X-ray. I said that is absolutely incredible, but you know what, I

84 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION

would like to get an ultrasound that is that good – something small and portable that I could get a full 3-D image without having to have X-rays. They said that’s the ultrasound laboratory. So we stop what we’re doing, we go into the laboratory, and I immediately funded these people with a seedling to get up and running on 3- D ultrasound.

This quote shows the DARPA program manager out in the field, interacting with his teams, scouting for possible new ideas, and deliberately comparing his observations with the big vision in his head. When he saw a promising part of the vision, the program manager took instant action to support and accelerate its growth as a small focused study.

He had the discretion and control to make these decisions while in field. All of these elements in this example match other descriptions from subjects about an effective proof- of-concept.

In addition, subjects who served as program managers at DARPA before the

1980s spoke about a greater flexibility to pursue proof-of-concepts without additional approval. One program manager said, “When I was program manager, I had a discretionary budget of $250,000. If I wanted to go ahead and spend it, I didn’t need approval. I didn’t need approval from my office director. I had pre-approval from the director.” The ability to act quickly and with minimal oversight was crucial to subjects, who felt empowered by their organization to ensure that the technology vision was complete and truly DARPA Hard from the start.

Similarities Between the Mechanisms

The expert workshops and proof-of-concepts are similar in three ways. First, the objective is to gain continual learning and feedback from the relevant R&D community, which then is used to sharpen the original technology vision. A program manager

85 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION explained his process, saying, “You muddle along, and whatever appears in front of you is the course of action that you judge to be the one that is going to lead you in the right direction, and you make a lot of false steps.” Goals are loosely set in order to support an unexpected result. Another program manager described the importance of staying flexible: “Sometimes you do workshops, sometimes you do seedlings, sometimes it just falls in your lap.” Second, these activities provide rapid feedback because they can be conducted within a matter of days or months. One program manager explained the time range for two of his seedling efforts: “One of them took six months; one of them took almost 18 months.” Third, both mechanisms are planned during the embryonic phases of a project, and once the vision is clear, program managers do not rely on these mechanisms further.

The frequency and success rate of these activities was discussed in a few of the interviews. One program manager shared his personal experience, saying, “I’ve probably held a dozen workshops, and I would tell you a quarter of them have led to programs.

Half of them turned out to be useless, and a quarter of them led to a program for somebody else.” Another program manager explained, “Of the programs I’m working with, I think I started something like nine programs in my time here, maybe 10. That’s a relatively high number and very few of those started as a seedling.”

Learning the Capacity for Envisioning

As the third major finding, DARPA does not directly help its program managers learn how to develop their visions of technology. Although some subjects described previous work experience with technology funding or startup development, the

86 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION entrepreneurial skills and business mindset required to be a DAPRA program manager are typically learned “on the job”. There is no direct teaching. Aside from standard review of DoD policies, DARPA does not require new hire training or provide an orientation specific to program creation for new program managers.

Instead, the prevailing beliefs are to start quickly and learn by doing. One program manager mentioned, “There really weren’t any guidelines. The best preparation was to be agile… and honest.” Another program manager shared a similar viewpoint, noting, “Most of these people are experienced researchers, so there isn’t a lot of training required… There’s not a whole lot of instruction here.” The socialization occurs during the first two stages at DARPA, in which new program managers are shaped, both deliberately and informally, by the organizational culture and ultimately assimilate the agency’s practices, rituals, and values regarding radical innovation.

A Community of Informal Learning

Based on subject accounts, DARPA’s culture of innovation is based on experts learning from other experts through informal channels. As one subject said, “It’s about the history, the continuity, the sociology.” Although many subjects struggled to explain the specifics of DARPA’s tacit culture, a clear thread emerged across their descriptions that former and current personnel served as mentors and masters of art to new personnel.

Program managers said they would observe others around them and emulate their behavior. One program manager explained the importance of the informal interactions between the groups, saying:

There is not really training on how to manage a program. There’s training on the mechanics, training on contracting, training on security, training on those kinds of

87 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION

things, but management of programs is really done by trial and error and an informal network that forms the day you walk in with the other program managers in your office or in your field, which may actually be outside of your office, but related field, that you run into and bump elbows with and talk to. Those people who’ve been here before, they learn lessons, and you really learn by talking and going down and sitting in their office for an hour and just talking through issues and problems. It’s like having an encyclopedia of how to do things right and wrong sitting in front of you and having direct access to it all the time.

An office director shared a similar story, in which relaxed dialogue helped him to advance his ideas:

We would get our gang together, and there would be lots of discussion, and we would stay up late at night, and maybe drink a little alcohol and such. So I thought I had a plan at that level. I could talk about it to people in ARPA. It was easy to have plenty of topics in the outline carried down three or four levels if I found a guy who wanted it that way.

An implicit theme emerged from subjects that effective program managers learned the necessary skills to start a program at DARPA faster than their less effective colleagues. Being more effective generally entailed having greater program funding, longer employment contracts, and fewer review sessions with directors. This learning occurred informally, and a certain amount of ego was involved in absorbing the organizational culture and belonging to the “in” community who understood how to start new programs.

An Entrepreneurial Spirit

Based on subject accounts, the culture at DARPA relies heavily on self-initiative.

In lieu of formal training, one aspect raised repeatedly by subjects was learning how to think as a technology investor and also as an entrepreneur. Program managers fancied themselves as venture capitalists, seeing a similarity in funding early ideas. Beyond the role of venture capitalists, DARPA program managers first and foremost seek to identify

88 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION new technologies without regard to market potential, provide seed funding for ideas that may never come to fruition, and help to drive these ideas toward reality. Further, DARPA program managers have no vested interest in the actual outcome of the teams and ideas that they fund.

In addition, several subjects compared DARPA program managers to entrepreneurs. They felt that both roles exhibited similar characteristics to find execution teams, pitch and evangelize new ideas, and seek high-risk, high-payoff opportunities. For instance, one program manager stated, “DARPA is the Venture Capitalists of the DoD driven by a herd of entrepreneurial Program Managers from industry, government & academia” (Pappert, 2009, pg. 5). Similarly, a deputy director perceived the program managers as acting as entrepreneurs when they sought funding internally for a new program. He said, “The way we help [the program managers] is really through a process by which they sort of pitch their idea for a new start… we act like a venture capital corporation – where someone is coming in to pitch a program, you’re really pitching your company.” Unlike real-world entrepreneurs, who ultimately have to choose one team and one goal, DARPA program managers can foster multiple teams, engage in the creation of multiple ideas, and seek to build a bigger vision than a classic entrepreneur.

Converting Visions to Programs

By “pitching”, program managers meant that they presented their visions for new programs internally for funding approval. The step between the vision formulation and program definition is a critical point because not all visions are automatically allowed to

89 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION move forward. Figure 15 illustrates the transition between the two stages, which was seen an important part of the overall process.

+&,&-)* 2$-3$0(* 2-$6-/&-* 8"#9)-/-3:* !"#$%&'(")'* .-$(%/01-)* 4"5)&1-)* 70)03"(")'* 8$0),;"$*

Figure 15: Transition between stages of Vision Formulation & Program Definition The step between the stages of Vision Formulation and Program Definition is a critical point because not all visions are automatically allowed to move forward as new programs at DARPA.

DARPA program managers rarely present an idea until they feel it is subjectively

“ready and complete”. Unlike corporations, there are no annual reviews or comparable stage gate procedures. Program managers schedule a review session as needed. A deputy director explained the importance of putting control in the hands of the program manager, who decides when and how to pitch to his superiors:

[Program managers] are empowered to make funding decisions. I think that’s really important. They obviously take a lot of advice; they get input from various scientists and advisors to a program… But ultimately they’re the ones that make the decision. So, it’s not a peer review model, which is perhaps the traditional one that’s employed in other federal agencies.

Subjects reported that the respective office director and, ultimately, the agency director provided the final decision for a new program. This proved true across agency offices and eras. While other individuals might be present for input and perspective, such as other DARPA office directors and military liaisons, the primary decision-maker was always the agency director. One program manager said:

90 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION

Now, going back to the joy of [D]ARPA: you know, if a program manager has a good idea, he has got two people to convince that that is a good idea before the guy goes to work. He has got the director of his office and the director of [D]ARPA, and that is it. It is such a short chain of command.

Another program manager spoke of “three signatures, one of which was yours and two were other people within DARPA… [who] would agree that this was a good project and then you would be able to go forward”. Similarly, a program manager from the mid-

1970s described his experience presenting a technology vision to only his office director and the agency director:

Well, before then, you would go up to the director’s office with [the office director] and sit with your feet on the desk and [the office director] would tell him, "Well, gee, I have got this great idea," and you could talk and knock it back and forth. I do not think that [the office director] had ever, ever had anything turned down by [the agency director], because [the agency director] understood.

Decisions for new programs were described as occurring quickly, generally under an hour. The decision was made and communicated by the end of the meeting, and no additional waiting time was required for evaluation. Once the vision was approved, it was considered as a new start program and on its way to documentation as a Broad Area

Announcement (BAA).

All of the subjects described DARPA’s evaluation process as distinct from peer review and other forms of committee or consensus decision-making. Multiple statements from the subjects demonstrate the strong sentiment and deliberate intention behind this aspect of DARPA’s process. A program manager noted, “We were the only agency that didn’t require external peer review. I think that was probably one of the most important things about DARPA. It was the only reason DARPA succeeds.” Another program manager expressed it as:

91 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION

Outside reviews get rid of any innovative ideas… They mediocritize any revolutionary idea. You can guarantee that the project will fail, or you won’t get the revolutionary project you want if it’s subjected to external peer review.

A further comment, “Outside review, clearly and unequivocally, gets rid of any innovative ideas.” An office director said, “That process [peer review] is guaranteed to never fund an idiot. It’s also guaranteed to never fund a visionary.” One subject felt strongly that DARPA’s model of evaluation was misperceived as elitist. He said:

Now, with regard to whether or not that’s elitism, or on the other hand whether or not having a committee decision structure as the NSF [National Science Foundation] does is really more democratic, I think is subject to argument. I can just as easily argue that having a committee of folks make a decision about supporting a particular proposal that goes to the National Science Foundation easily is an example of elitism. That is to say, there is this club of people who all work, let’s say, in natural languages. There are a set of people who are accepted in that club, and there are a set of people who aren’t. You float that proposal around to a group of people that NSF has chosen to be its peer review group. And if you’re in the club you might get accepted, and if you’re not you might not get accepted. So, I think I reject the notion of elitism versus democracy, and just say there was a different set of objectives, a different modus operandi. ARPA initially was to be a quick reaction operation – let’s see where we make a big difference in a few places.

In short, the 52 years of success at DARPA are built on a model that eschews peer review and consensus decision-making. DARPA’s process of evaluation is viewed as positive and highly efficient in advancing technology visions in the overall mandate of radical innovation.

Conclusion

The research data revealed five stages that define the process of radical innovation at DARPA: Recruitment, Vision Formulation, Program Definition, Portfolio

Management, and Technology Transfer. This study investigated the start of the process, specifically describing activities that occurred in the first two stages of Recruitment and

92 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION

Vision Formulation. Recruitment begins with both the people and their ideas. In terms of people, new program managers are hired into DARPA for their radical visions of technology in addition to their technical expertise. In terms of ideas, the attributes of a good vision are to be technically challenging, actionable, multidisciplinary, and far- reaching – DARPA Hard. If program managers arrive to DARPA with partial visions, they develop them through a combination of expert workshops and proof-of-concepts.

However, DARPA does not teach program managers how to build visions. Instead, new program managers learn the necessary skills for finding and formulating visions through socialization at DARPA, and all learning is gained through immersive experience and interaction. Finally, when program managers are ready to take their visions forward for new program status, they submit to a highly efficient and, by extension of 52 years of operation, successful process of review by the organization’s leaders.

93 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION

CHAPTER 6

DISCUSSION

This study provides empirical evidence of the role of vision in fostering technological invention, adding to the existing literature about radical innovation (e.g.,

Roberts, 1987; Tornatzky, Fleischer, & Chakrabarti, 1990; O’Connor et al., 2008).

DARPA provides a long history of examples of technical program visions and how these visions are formed and communicated time after time. In this section, the four main findings of the study are discussed in detail and in context of the literature.

First, this study shows a relationship between the formation of a technological vision and the sustained creation of radical innovation, providing new knowledge about the role of vision in radical innovation. Since its inception in 1958, new programs at

DARPA have required a vision to be started, which then guides subsequent work and development. Several dimensions arise regarding the role of vision, which entail functioning primarily at the program level, characterized as DARPA Hard, and relying on the program manager as a vision champion. Second, this study describes the use of expert workshops and proof-of-concepts, used steadily by DARPA to shape partial visions into complete visions, which demonstrates critical efforts occurring pre-vision. Third, this study describes the importance of socialization in order to prepare and instruct program managers in their envisioning skills. Immersed in the culture at DARPA, new program managers learn from each other and their network connections. Fourth, this study provides new evidence about radical innovation governance models. DARPA relies on small group decisions by organizational leadership to approve promising new visions,

94 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION running counter to the dominant literature about stage-gate reviews, peer reviews, and extended consensus-seeking processes.

A Process Model of Radical Innovation

As described in the previous chapter, DARPA follows certain high-level steps in its quest for radical innovation, and this process is reproduced in Figure 16. By documenting the process at DARPA, this study helps other researchers and practitioners to understand one organization’s formula for sustained radical innovation. Documented processes are the basis for repetition and become the springboard for continuous and measurable performance.

1C;>CD-($<#".--(3#,.+(#B(<0,%"0+(%&Î#&(

8%-%#&( ><#'<03( >#

5A$%"0+(-40'.E'04.(3#,.+(B#<(&.F($<#,*"4(,.2.+#$3.&4(

)*%+,(( 5.-6&'(7( !"#$%&'( 1.2.+#$3.&4( 90*&":( )*-%&.--(/0-.( 80+%,06#&(

Figure 16: Comparison between DARPA’s process model and the stage-gate model Although DARPA’s process model of innovation looks similar to the typical stage- gate model for new product development (Cooper, 2001), the two models differ in terms of objectives, activity timing, and evaluation mechanisms.

95 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION

In addition, some scholars may see a similarity between the depiction of

DARPA’s process model and the typical stage-gate model for new product development

(Cooper, 2001), depicted in Figure 16. Both models are comprised of five stages that sequence categories of cross-functional activities, which help to invite a comparison.

However, there are at least three key differences between the two models. First, the two models differ in objectives. DARPA’s goal is radical innovation, which is intended to produce new technologies that ultimately may lead to new products. In contrast, the stage-gate process is designed to build and launch new products.

Second, the two models differ in their activity timing. DARPA’s model is focused on the early stages that precede project scope. The stage-gate model is missing the preliminary or ideation phase, often called Discovery, which occurs before the start of the first stage of scoping.

Third, the two models differ in evaluation mechanisms. DARPA’s process is fluid, and although transition arrows are noted between stages, formal decision points are not necessarily required before proceeding onto the next set of activities. In comparison, the stage-gate model is predicated on predefined deliverables and checkpoints with go/no go criteria at the end of each stage (what are called gates).

New Dimensions of Vision

Vision plays a central role in DARPA’s process of innovation; indeed, DARPA starts its process with vision. It matters where and how a vision is started, as does who starts and maintains the vision. DARPA program managers are hired deliberately for their visions of technology, even if partially formed. Then, program managers codify their

96 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION visions at the start of each new program in a specialized document called a Broad Area

Announcement (BAA), which are used to generate interest in the broader R&D community. Thus, the vision is formulated before groups are funded because DARPA’s funding recipients rely on these BAAs to determine potential solutions.

Visions at the Program Level

By studying the role of vision within DARPA, this study reveals several new dimensions of vision as related to innovation. One dimension is the level at which a vision operates. The dominant business literature has largely studied vision at the organizational level (e.g., Collins & Porras, 1991); at the other end of the literature, several studies have investigated technological visions at the product or project level

(e.g., Lynn & Akgün, 2001). Within DARPA, work is broken down at three levels: organizational, program, and project, and the data shows that vision is introduced and functions primarily at the program level. Figure 17 illustrates the multiple levels of visions that could exist within an organization, and visions at DARPA address the gap between the organizational and project/product levels.

97 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION

./0*123% !"#$%&'$()%$*+,&-&)%+ -+*+4-(,% !"#$"% 546%02%7,+% &'()*+*% .")#"$/+,&-&)%+ 807+-47)-+% ,+-++ ./0*123% .")0123+4+.")5623+,&-&)%+ -+*+4-(,+

Figure 17: Visions at DARPA operate at the program level The literature on innovation predominantly discusses technological visions at the organizational level, and several studies have investigated technological visions at the project level. The literature fails to discuss vision at the program level, which is equivalent to the business unit or market level. At DARPA, technological visions function at the program level.

In fact, DARPA lacks a traditional corporate vision, which identifies a set of organizational values and direction for the enterprise. Since its inception in 1958, the agency has not defined (or even reinvented) its long-term goals, aspirations, and values at the organizational level. Instead, DARPA emphasizes visions at the program level, which correlates with a traditional business unit or market focus. Multiple visions – often totaling over a hundred, depending on the number of program managers actively serving at DARPA – exist in parallel at a given time. Programs serve as new, broad-scale technical initiatives that typically encompass multiple projects, and projects are equivalent to product teams in industry. Again, a DARPA program could be considered equivalent to a business unit or new market category. At DARPA, a program is more of an open-ended question or challenge posed to the R&D community, which might have multiple solutions and product possibilities, and scholars have documented the benefit of a challenge model within an R&D setting (Bonvillian, 2006, 2009).

98 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION

In addition, visions at the program level allow DARPA program managers to direct multiple projects, multiple teams, and even multiple products over multiple years.

Through visions at the program level, DARPA can excite and rally interest across several different technical areas, helping to distribute resources more effectively. Program visions provide a way to organize multiple projects and smaller-scale efforts across a range of funding recipients, who each may interpret the vision differently in application.

This approach, in turn, increases the likelihood of a greater diversity of solutions. A program structure also allows for greater flexibility in engendering commitment.

Vision Quality

A second dimension is the quality of the vision. In the literature, few studies focus on technological visions, and most scholars draw on studies of corporate vision. For example, Lynn and Akgün (2001) describe product visions as a combination of clarity, support, and stability, which are determined relative to the larger organization. While these attributes offer a sense of an ideal vision, they do not provide meaningful guidelines on how to develop a vision, including the type of vision to create in the technology space.

This study shows that technological visions at DARPA have several attributes that are essential to the creation of its visions. Since its inception, DARPA has socialized a catch phrase known as DARPA Hard. Drawn from the data, a DARPA Hard program vision is characterized as technically challenging, actionable, multidisciplinary, and far-reaching.

Taken apart, these attributes can be found discussed in prior studies.

The first attribute of technically challenging is understood within the operations research and engineering design community as a “wicked problem” (Buchanan, 2009). A

99 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION wicked problem is a technically difficult problem that is nearly impossible to solve due to complex interdependencies, a high level of ambiguity, and conflicting interests from stakeholders. Wicked problems cannot be solved through classic experimentation and logic, instead requiring a different and more creative strategy of reasoning.

By focusing on these types of problems at DARPA, program managers have ensured that they push the limits of innovation sought, what might be interpreted as

“highly radical” innovation according to Abetti’s scale (2000). When most definitions of radical innovation argue for market changes, DARPA is pushing for a radical technology shift, which then may lead to a radical market shift. Each attempt at creating a new technical solution changes the understanding of the problem in two fundamental ways.

First, more information helps to reformulate the initial requirements, and second, every prototype and implementation built advances the state of knowledge overall in the world.

In other words, there is no turning back or reverting to the former understanding of the problem. The vision for a DARPA program provides the high-level guidelines to inspire potential funding recipients, and by engaging both more and different groups to respond,

DARPA is able to cast a wider net for solutions and likewise accelerate the experimentation process.

This approach helps to drive toward action, and actionable is the second attribute of DARPA Hard. Program visions are intentionally grounded in reality because they are expected to improve and extend the limits of existing technologies. Visions cannot exist as science-fiction fantasy, political rhetoric, or policy scenarios. This attribute is partly captured in earlier research about the reflective practitioner, in which Schön (1983) describes how professionals, such as engineers, address problematic situations that are

100 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION fraught with uncertainty, disorder, and indeterminacy by taking action through real-time cycles of feedback and learning. In DARPA’s case, program managers rely on their visions as a way to simulate broader learning in their research networks.

A growing body of research about learning in inter-organizational networks shows that networks facilitate rapid responses. Powell (1990) states that, “Whether it is the case that one firm’s technological competence has outdistanced the others, or that innovations would be hard to replicate internally, as suggested by the growing reliance on external sources of research and development, network forms of organization represent a fast means of gaining access to know-how that cannot be produced internally” (p. 316).

The third attribute of multidisciplinary is equally critical to forming the right program visions at DARPA. As many subjects noted, they needed to redefine problems outside of usual boundaries, and complex situations required drawing from more than one discipline. Multidisciplinary efforts are not new to government-sponsored R&D and can be evidenced in the rise of systems engineering in the 1950s that supported large-scale efforts, such as the Atlas missile program and ARPANET (Hughes, 1998). This type of approach encourages less hierarchical control and more network-based management techniques.

The fourth attribute of far-reaching is important when creating program visions at

DARPA. One part of far-reaching is about having a broad impact in society. Subjects spoke about making a difference in magnitude. DARPA program managers stated that they need to think big in order to have big results. Another aspect of far-reaching is the ability to plan long-term. The importance of planning long-term has its roots in World

101 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION

War II, notably the founding of RAND (Campbell, 2004), which transition into today’s management science through thinkers such as Drucker (1959, 1973).

The real test of a good vision in R&D is whether others will commit resources to action, which will bring results in the future. DARPA deliberately couples action with future intent. However, the conundrum is that traditional R&D results may not be produced or easy to measure because the extent of far-reaching effects take time and are broadly distributed across society. The attribute of far-reaching is consistent with recent work in foresight engineering, which focuses on long-range technology cycles as part of an organization’s ongoing search for innovation opportunities (Carleton & Cockayne,

2009).

Together, these four attributes – technically challenging, actionable, multidisciplinary, and far-reaching – that make up a DARPA Hard program provide a metric that can be instrumented and tested. Based on pioneering work in taxonomies

(Bloom, 1956; Fleishman & Quaintance, 1984), Figure 18 presents a sample classification using a 7-point scale that was used for the quantification of human performance variables, specifically describing human ability for side-to-side equilibrium

(Cockayne & Darken, 2004). This type of scale could be adapted in order to classify each of the four attributes characterizing DARPA Hard. Follow-on studies can further define and test the scale values as related to radical innovation. Ultimately, if other organizations seek to recreate a variant of DARPA Hard, they will benefit from defining and using a clear classification of technological visions.

102 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION

7

6

5

4

3

2

1

Figure 18: A sample 7-point scale for quantification of human performance variables A sample 7-point scale, drawn from another study, could be adapted to classify and evaluate each of the four attributes characterizing DARPA Hard. Follow-on studies can define and test the scale values as related to radical innovation.

103 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION

Visionaries of Technology

A third dimension of vision is the person responsible for fostering it. Visions cannot exist without creators, who must imagine and invent them. Within DARPA, the work on innovation is driven as much by ideas as by individuals. Program managers are hired as technical visionaries, and they are solely responsible for shaping, spearheading, and promoting their respective visions of technology. The project champion is a critically recognized role in innovation, and findings from this study are consistent with literature on this topic (Howell & Higgins, 1990). At DARPA, a new program is not confounded with multiple organizational champions; instead, there is a clear relationship in that each program manager builds one vision per program. Figure 19 depicts this relationship.

However, the DARPA program manager does not operate in isolation. He (or she) is part of a broader ecosystem and network, in which multiple players – both internally and externally to the agency – are engaged to support the formation and execution of a program vision.

0*)&1*2$ 3#14"-2-'&1*2$ +&,&-"$

!"#$$ !"#$ %&'$()#*$ +&,&-"*./$

Figure 19: A radical technological vision relies on one big idea and one visionary At DARPA, a program vision relies on a program manager, who serves as the vision’s primary champion internally and externally. Moreover, there is a clear relationship in that each program manager builds one vision per program.

104 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION

DARPA program managers serve in other innovation roles that have been documented separately in literature. For example, they share some characteristics with business innovators because DARPA program managers provide substantial funding, as well as some organizational credibility and access to other resources (Howell & Higgins,

1990). Although DARPA program managers do not build and develop their own visions, instead relying on the various funding recipients, they do act as technical innovators in other ways (ibid.). More informed than the usual project champion, DARPA program managers are nearly all technically educated and bring deep expertise from various fields of engineering and science. This background allows them to more effectively understand the given technical problem, as well as advise and guide the technical teams that they sponsor. A growing number of studies discuss the special role of a technical visionary, who combines technical knowledge with project oversight (Hebda et al., 2007;

Deschamps, 2008).

DARPA program managers also play the role of technology licenser or technology transfer manager. They are directly responsible for finding potential user groups, typically in the U.S. military services, who might test and ultimately adopt a functional prototype. The final success of DARPA program visions hinges on user adoption.

At DARPA, potential program managers – the champions of new technological visions – are found and recruited through the extended research network. Studies show that as networks mature, they tend to petrify (Powell, 1990). People prefer to work with familiar connections, which limits network access to new connections. Current program managers will find new program managers based on similar qualities and will continue

105 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION funding the same relationships. When this happens, an innovation network does not diversify, and the development of new ideas can be potentially severely limited. DARPA has addressed this limitation by deliberately hiring program managers new to the network, who, in turn, bring new visions of technology. Subsequently, the new-to-the- network program manager finds and funds research groups that bring additional new ideas to the network, which helps to refresh institutional thinking and challenge engrained assumptions.

Lastly, DARPA is now over 50 years old as an organization, and historically, the agency has relied on its network for internal job referrals. As the people in DARPA’s network have aged, they may not be cultivating as many new relationships with other research groups or also with junior engineers and scientists. Age plays a substantial role in creating new fields, and research shows that younger scientists are more likely to be drawn to a new field than older scientists (Rappa & Debackere, 1993).

Some scholars have studied how large mature organizations must continually reconfigure their systems of power in order to sustain innovation (Dougherty & Hardy,

1996). Recently, DARPA leadership has recognized the need to recruit younger program managers into its mix. For example, the press observed former agency director Tony

Tether “has managed to draw younger researchers into an agency whose stalwart backers are growing greyer every year” (Nature, 2008). However, more research is needed to understand the affects of age on DARPA’s ability to foster radical innovation.

106 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION

The Development of Partial Visions

In the key texts that mention vision, few descriptions are provided about how to generate a vision or develop a partial vision into a complete technological vision (e.g.,

Roberts, 1987). Scholars underscore the importance of having a vision, yet they assume a complete vision. Findings from this study demonstrate that multiple steps are consistently taken by DARPA program managers in order to advance their early ideas and thinking before the complete vision is formed. Figure 20 illustrates the actions that must occur before a complete vision is achieved. In addition, while the technological idea drives action, the path to the vision itself is emergent.

./0#%1& 2*%3)4*0& ,$%-$"& !"#$%&& '()(*+& '()(*+& ,%**56*56& !*+7#01&8#9*&

:##;<$73&=&>#$%+(+?&

Figure 20: Efforts preceding a complete vision of technology Earlier actions occur before a complete vision is achieved at DARPA.

This study describes the formation of partial visions via two primary mechanisms, specifically expert workshops and proof-of-concepts, which are used consistently throughout DARPA’s history to develop partial visions into clear visions. While details may differ, the objective is the same between the two mechanisms: to gain more insight into a promising yet incomplete vision. Expert workshops and proof-of-concepts address the people and the idea, respectively. Through expert workshops, each program manager

107 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION engages his (or her) network, and the network serves as a way to gain perspective through dialogue among trusted colleagues. In studies about knowledge networks and communities of practice, network members regularly share information through both formal and informal channels (Hildreth & Kimble, 2004; Powell & Grodal, 2005), and the DARPA workshops positively exploit the broader knowledge network for the agency.

The DARPA workshops are effective because they draw on the collective wisdom for a field, helping DAPRA program managers to gain access to the latest knowledge about a particular topic.

If the workshops rely on people, the proof-of-concepts depend on the idea. The objective of the proof-of-concepts is to explore and test the feasibility of an emerging idea. Each proof-of-concept serves as a directed demonstration. Proof-of-concepts are regularly discussed in engineering design research and business studies as a form of prototyping (Schrage, 1999; Betz, 2003; Moss & Atre, 2003), and specifically, Carleton and Cockayne (2009) discuss the growing role that physical prototypes serve in long- range planning. This study provides new information about the use of proof-of-concepts in vision development as a way to demonstrate feasibility and test early hunches before undertaking a new technical initiative. There is an opportunity to expand on the relationship between prototyping and vision formation.

It is important to note that this combination of expert workshops and proof-of- concepts has provided the primary mechanisms for converting partial visions into full visions at DARPA; no other mechanisms were pursued as long or as reliably, as reported by DARPA program managers and funding recipients. This approach has implications for organizations pursuing radical or disruptive innovation. O’Connor and her colleagues

108 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION

(2008) discuss the different experiments that big companies have attempted in order to scout for and generate radical ideas. Some of these experiments resemble the expert workshops at DARPA. IBM has held a large annual R&D event to order to stimulate new ideas internally and identify potential emerging business opportunities. This event has been called by multiple names – including idea jams, idea cafes, and deep dives – and while the organizers continually tinker with the process, the event itself remains constant every year. The annual event has led to a high number of opportunities, which in turn have become profitable business lines at IBM.

Learning Radical Innovation Through Socialization

The third finding relates to the culture of innovation at DARPA. Program managers come from a variety of backgrounds. While they have impeccable academic and professional credentials, many lack direct experience with certain innovation skill sets, such as documenting a vision, recruiting and leading others, and technology transfer.

Regardless of their background, expectations are high for DARPA program managers to develop and deliver on their program visions quickly.

In addition, DARPA does not provide formal training in innovation “know how”, particularly the skills needed to develop program visions. Is staff training necessary for radical innovation? According to subjects, DARPA has not codified much of its internal procedures historically; so new program managers cannot rely on manuals or similar process guides. Instead, knowing is a matter of participating. At DARPA, subjects reported learning primarily from immersion. From the start, a candidate for a new program manager has to be already embedded in the research community to be

109 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION considered for recruiting. Once at DARPA, program managers described learning by doing, particularly by proactively reaching out to colleagues, alumni and other members in the network for advice and resources, as well as by gaining new knowledge from regular field visits.

In many ways, DARPA is a culture of show, not tell. Through a process of socialization, program managers acquire the habits, beliefs, and accumulated knowledge of the organization. In sociology, this period is known as metamorphosis, when a newcomer becomes an established organizational member (Kramer, 2010). How people behave and interact with one another over time shapes an organizational culture, and the data from DARPA is consistent with prior studies about tacit knowledge and informal learning occurring within innovation organizations and communities of practice (Von

Krogh et al., 2000; Wenger, 2007).

If an organization is to survive, then research shows that stability over time is required, so that one generation of employees transmits the dominant social and cultural patterns to the next generation (Alvesson, 1995). In other words, practice is transferred from those who have done it to those who need to do it. At DARPA, this transfer of knowledge occurs through informal conversations, and given the short contracts of

DARPA technical staff, the cycle of generations is rapid. It is remarkable that a knowledge-generating organization over 50 years old, which has resisted lasting knowledge capture, has maintained such a stable set of practices as DARPA has. Based on subject reports, two factors have likely contributed most to the unusual stability of

DARPA’s culture. First, the broader infrastructure supporting DARPA program managers, namely the support staff, provide continuity across leadership turnovers. This

110 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION support staff functions as an underlying layer of institutional permanence, handling the same routines and project coordination tasks. Second, the agency’s network structure supports ongoing learning. For example, even when program managers leave their agency roles officially, they typically stay connected to DARPA in other ways. This connection creates additional channels of knowledge sharing between staff and also ensures that some institutional memory is maintained across staff rotations. New staff relies on the stories and experiences shared within the network in order to prepare themselves at DARPA.

Internal Review of Radical Innovation Ideas

Even with the right person and the right idea, a promising technological vision may not become a new program at DARPA. There is one final test before a Broad Area

Announcement (BAA) is released to the public. A program manager must pitch his vision internally with a small audience for funding approval, and decision making authority resides namely with the agency director and respective office director. Subject reports demonstrate that DARPA has consistently followed this governance model over the years, actively discouraging larger evaluations in the agency’s innovation process.

Subjects especially note the benefits of speed, convenience, and flexibility from these small group reviews.

DARPA’s model runs counter to the literature and practice of innovation, which discusses consensus-based governance models – such as innovation boards, technology councils, R&D committees, task forces, and stage gates – as a dominant best practice

(Bacon & Butler, 1973; Hamel, 2002; Snyder & Duarte, 2003; O’Connor et al., 2008;

111 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION

Skarzynski & Gibson, 2008). These models provide a decision-making framework that help to define evaluation criteria, grant decision-making power, and verify feasibility of a new research idea. A growing body of literature has noted that certain models have limitations for radical innovation. Gassmann and von Zedtwitz (2003) note:

In industries or projects where the science or technology push is the dominant driver of innovation, stage-gate processes are too rigid and slow. Innovations that are triggered by a technological invention with unknown market potential need different processes and techniques to succeed. (p. 704)

Overall, innovation studies endorse a strong philosophy that the processes for radical or disruptive innovation must differ from traditional R&D processes in order to be effective within an organization. By deliberately adopting a model of limited, leadership- driven review and following it for over 50 years, DARPA provides empirical support for this belief. Instead of creating large task forces, DARPA relies on its leadership to approve and support the visions. Instead of formally scheduled sessions, DARPA program managers arrange meetings when they feel that their new program visions are ready for funding. Most of corporate R&D, the work of funding agencies, and academic research are actually structured in direct opposition to this approach. Members of the science community, who believe that DARPA provides an enduring and effective model for advancing radical innovation, understand this difference. Penman and Bates (1999) write, “Those wishing to emulate the success of DARPA and Bell Labs might consider another important aspect: freedom from the so called ‘peer review’ that weighs down most National Institutes of Health (NIH) and National Science Foundation efforts.”

112 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION

Conclusion

Four main findings were discussed in relation to the literature review. By describing how visions serve an integral role in DARPA’s innovation process, the first finding brings new perspective to innovation studies about the role of visions in radical innovation. In particular, new program visions must meet the criteria of being DARPA

Hard, and this term of art introduces a working metric for technical breakthroughs that are nearly impossible to achieve based on the current state of knowledge and tools.

Second, the discovery that expert workshops and proof-of-concepts have been used repeatedly to convert partial visions into complete visions at DARPA shows that activities exist pre-vision and directly influence the formation of technological visions.

Third, the discovery that new program managers receive no formal documentation or training for their roles and instead rely on acculturation is consistent with prior research on innovation networks and communities of practice. Finally, by showing that DARPA has a leadership-driven, decision-making model, in which leadership approves a new program vision, the fourth finding introduces contradictory evidence to the dominant literature. These four findings, supported by empirical evidence, add to the current understanding of technological visions and radical innovation research.

113 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION

CHAPTER 7

CONCLUSION

In conclusion, this research investigated the role of vision in the process of radical technological innovation. DARPA provides a relevant data set because the federal agency’s mandate has explicitly focused on sponsoring technology ideas that are revolutionary and high-payoff. Moreover, DARPA has been actively pursuing radical innovation since 1958, which demonstrates over 50 years of sustained practice. By drawing on 47 interviews with DARPA personnel and funding recipients, plus archival documents and an additional set of 12 historical interviews, a dominant set of practices were discovered that span multiple DARPA eras and offices.

This study relied on grounded theory methodology to analyze the data. The purpose of an inductive study, such as the one reported here, is to guide and inspire new research, not to validate existing ideas. Four major findings demonstrate that vision fosters radical innovation at DARPA. Specifically, vision starts DARPA’s process of radical innovation through the recruitment of innovation champions who develop

DARPA Hard visions at the program level; partial visions can be advanced to complete visions through expert workshops and proof-of-concepts; job skills are learned largely through socialization within DARPA’s culture; and promising new visions are ultimately selected through leadership-driven approval. The extent that these findings generalize to other innovation-seeking organizations can be determined through additional research; however, data from existing literature suggest that any resemblance to efforts in other organizations is piecemeal at best. Follow-on studies can further quantify these findings.

114 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION

Contribution to the Field

Outsiders have long viewed DARPA as a black box, in which the agency’s input and output are clear, yet knowledge of its internal workings remains murky. By studying

DARPA’s internal operations, this study provides new empirical evidence about the role of vision and its relationship to the early stages of radical innovation in an organizational setting. In addition, by taking a multidisciplinary stance, this study synthesizes research conducted about radical technological innovation across different fields, including organizational innovation, technology management, visionary leadership, and industrial research and development. Therefore, a framework is introduced to unify the literature around the concept of technological visions. This study also provides a novel data set.

While most studies about DARPA have been fragmented or largely anecdotal to date, this study provides a coherent and complete data set on the innovation processes at DARPA.

Contribution to the Practice of Innovation

This study provides lessons to industry, and the four major findings offer insights for both researchers and practitioners of radical innovation. Multiple organizations – including corporations, research labs, and funding agencies – can gain insight in facilitating efforts during the early stages of innovation, notably in recruitment and vision formation. Specifically, if vision operates effectively and powerfully at the program level, then R&D management may want to consider how their current organization is structured and at which levels of execution. In addition, the four attributes that characterize DARPA

Hard – technically challenging, actionable, multidisciplinary, and far-reaching – provide a set of evaluation criteria for a good vision that can be instrumented and tested.

115 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION

Moreover, DARPA recruits new program managers from a broad and diverse network that includes universities, corporations, national laboratories, research nonprofits, and military services. If R&D organizations want to enhance their own research talent, then they should consider adopting DARPA’s procedures. In particular,

DARPA program managers are hired for their technological visions; thus, other innovation-seeking organizations should make vision a critical qualification for their intrapreneurs and radical innovation leaders. Envisioning ability may be the most important skill for organizations seeking radical innovation.

In addition, DARPA has relied on expert workshops and proof-of-concepts as the sole means to refine partial visions. Other innovation-seeking organizations should evaluate and potentially eliminate all other mechanisms in idea scouting and formation, which function as extracurricular activities.

This study raises implications for building an organizational culture that fosters radical technological innovation. Innovation-seeking organizations can look to DARPA’s model of acculturation as an effective way to socialize innovation leaders, who must act as champions of new technological visions.

Lastly, DARPA’s model of leadership-driven decision-making challenges dominant practices and beliefs within R&D. Fully embracing DARPA’s model may be the most disconcerting change that an organization will face in its quest for radical technological innovation.

116 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION

Research Limitations

First, this study relies on interview data. Although hearing from subjects provided direct and first-hand accounts, subjects may potentially inflate their own roles in context or forget certain details over time. Reviewing agency documents as a secondary data source, as well as identifying stable patterns of behavior across various agency eras and directorships, addressed this limitation.

A second limitation is a hindsight bias, in which subjects emphasize only successes from their past. Several subjects remarked about the strong ego of DARPA program managers, which might influence how they perceive their self-worth and impact.

By interviewing DARPA funding recipients and related R&D roles, the various accounts from program managers and other DARPA personnel were further verified and cross- checked.

A third limitation is a researcher bias. A researcher may project, consciously or unconsciously, personal beliefs and values into the choice of methodology and the interpretation of findings. Comparing primary interview data to historical interviews, the latter of which were conducted by other scholars for different objectives, helped reduce a researcher bias.

Lastly, the findings reported here could be idiosyncratic to DARPA. While the agency was deliberately selected as a data source due to exceptional status, in order to more effectively study radical innovation in its purest form, the findings may not be as broadly generalized to other organizations or settings. By gathering data from different

DARPA offices, as well as from multiple programs across the agency’s entire history, the applicability of the findings has been increased.

117 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION

Directions for Future Research

As primary research, this study conducted extensive foundational work to describe innovation practices at DARPA and outline a working model of vision formation. Building on the data and insights in this study, a logical next step is to formulate and test hypotheses for comprehensive investigation and experimentation.

Multiple areas remain unexamined and are suggested below as potential directions for future research.

One research thread could probe the development of DARPA program visions more deeply. For example, how has the quality of DARPA visions changed over time?

Could these visions be categorized in some fashion?

Another thread is to study the relationship between DARPA projects and their overarching programs. Specifically, how do programs serve as strategic or technical platforms for projects? How have project characteristics, such as the time required to deliver solutions or the frequency and quality of interaction between DARPA program managers and funding recipients, changed over time in relation to the program visions?

From an organizational view, a research opportunity exists to understand DARPA in terms of other organizational structures and roles. For instance, how does DARPA’s model of radical innovation compare to corporate R&D or national funding agencies?

How could DARPA’s innovation practices be transferred to other types of organizations?

And at a personal level, DARPA program managers could be analyzed against complementary roles in a national innovation ecosystem, such as venture capitalists or entrepreneurs.

118 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION

A related thread is to compare DARPA’s mode of operations and historical output to other R&D institutions, such as Stanford Research Institute / SRI, Bell Labs, HP Labs,

Apple Research, and Interval Research (C. House, personal communication, June 11,

2010). In addition, given the high turnover rate, the pool of DARPA alumni continues to grow. How do former program managers influence other R&D organizations in their later careers, and which innovation practices from DARPA do they continue, adapt, or even reject in their subsequent work?

Lastly, DARPA’s pursuit of radical innovation does not function in isolation. An additional research thread could investigate how DARPA operates as a pure model of open innovation and how DARPA fits into the broader ecosystem of innovation, both nationally and globally. In short, this study presents an early step in understanding how one exceptional organization has organized and sustained radical innovation for so long and as consistently as DARPA has.

119 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION

APPENDIX A:

DARPA TECHNOLOGY OFFICES

As of the time of writing, DARPA programs are organized in the following technology offices (DARPA, 2010d):

Adaptive Execution Office (AEO) – Conceive and execute novel technology and system developments that are adaptive both in end function and in the process by which they are developed. Improve the transition worthiness of DARPA programs across the enterprise.

Defense Sciences Office (DSO) – Bridge the gap from fundamental science to applications by identifying and pursuing the most promising ideas within the science and engineering research communities, and is committed to transform these ideas into new DoD capabilities.

Information Processing Techniques Office (IPTO) – Supports research, development, and prototyping that span the information lifecycle: sense, process, understand, and apply.

Microsystems Technology Office (MTO) – Leading pioneering research in integrated Microsystems as “platforms-on-a-chip” to enable revolutionary performance and functionality for future DoD systems.

Strategic Technology Office (STO) – Focus on technologies that have a global theater- wide impact and that involve multiple Services.

Tactical Technology Office (TTO) – Engage in high-risk, high-payoff advanced technology development of military systems, emphasizing the “system” and “subsystem” approach to the development of aerospace systems.

Transformational Convergence Technology Office (TCTO) – Advance new crosscutting capabilities derived from a broad range of emerging technological and social trends, particularly in areas related to computing and computing-reliant subareas of the life sciences, social sciences, manufacturing, and commerce.

120 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION

121 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION

APPENDIX B:

FEDERAL BUDGET CATEGORIES

The U.S. government classifies federal budgetary activities into functional and sub- functional categories that represent the major purposes of the federal government (U. S. Department of Defense, 2008). DARPA follows this model to characterize its investment portfolio. Generally, R&D activities are divided into either Science & Technology (S&T) or Research & Development (R&D) categories. Science & Technology encompasses budget categories 6.1, 6.2, and 6.3, which correlate with basic research, applied research, and development, respectively. Budget categories 6.4 through 6.7 correlate with testing, evaluation, and design activities. Categories are summarized in Table 7.

DARPA personnel use these budgeting terms to differentiate their areas of investment, funding projects in categories 6.1 through 6.3. For example, a recent strategic plan stated, “DARPA uses its 6.1 funding to ‘get a seat at the table’ with basic research performers so that DARPA can understand what is going on at the forefront of science and can mine that research for new Defense capabilities” (DARPA, 2007). In addition, the taxonomy helps to show the relationships between the various funding areas, which are addressed by the other Department of Defense (DoD) agencies and military labs.

Category Category Name Description Science & Technology 6.1 Basic Research “Basic research is systematic study directed toward greater knowledge or understanding of the fundamental aspects of phenomena and of observable facts without specific applications towards processes or products in mind…” (p. 5-2) 6.2 Applied Research “Applied research is systematic study to understand the means to meet a recognized and specific need. It is a systematic expansion and application of knowledge to develop useful materials, devices, and systems or methods…” (p. 5-2) 6.3 Advanced “This budget activity includes development of Technology subsystems and components and efforts to integrate Development subsystems and components into system prototypes for field experiments and/or tests in a simulated environment…” (p. 5-2) Research & Development 6.4 Advanced “Efforts necessary to evaluate integrated Component technologies, representative modes or prototype Development & systems in a high fidelity and realistic operating Prototypes environment are funded in this budget activity…” (p. 5-3)

122 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION

Category Category Name Description 6.5 System “SDD programs have passed Milestone B approval Development & and are conducting engineering and manufacturing Demonstration development tasks aimed at meeting validated requirements prior to full-rate production…” (p. 5-3) 6.6 RDT&E “This budget activity includes research, development, Management test and evaluation efforts and funds to sustain and/or Support modernize the installations or operations required for general research, development, test and evaluation…” (p. 5-4) 6.7 Operational “This budget activity includes development efforts to Systems upgrade systems that have been fielded or have Development received approval for full rate production and anticipate production funding in the current or subsequent fiscal year…” (p. 5-4)

Table 7: U.S. federal defense budget categories

DARPA’s estimated FY2010 budget is $3.25 billion (DARPA, 2009; personal communication, Mark Peterson, June 10, 2010). DARPA typically accounts for about 25 percent of DoD’s S&T budget (DARPA, 2007). DARPA finds this emphasis in line with common industry practice, which devotes about 75 percent of R&D funding to product improvement and the remaining 25 percent to new ideas, products, and markets (ibid.).

123 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION

APPENDIX C:

RESEARCH INTERVIEWS CONDUCTED

A total list of the research interviews is provided below as reference. The list of DARPA staff includes both current and former personnel, and interviews with current DARPA staff were conducted with DARPA’s full support. Several of the interviewees are also subjects of the oral histories done by the Charles Babbage Institute.

DARPA Staff

Agency Directors & Deputy Directors

1. Ken Gabriel 2. Bob Leheny 3. Steve Lukasik

Office Directors

4. Robert Kahn, Information Processing Technology Office 5. Greg Kovacs, Microsystems Technology Office 6. Ivan Sutherland, Information Processing Technology Office 7. Robert Taylor, Information Processing Technology Office 8. David Tennenhouse, Information Processing Technology Office

Program Managers

9. Ralph Chatham, Cybernetics Technology Office 10. Vinton Cerf, Information Processing Technology Office 11. Joseph Cohn, Defense Sciences Office 12. Steve Crocker, Information Processing Technology Office 13. John Dexter Fletcher, Cybernetics Technology Office 14. Tom Kenny, Microsystems Technology Office 15. Doug Kirkpatrick, Defense Sciences Office 16. John Main, Defense Sciences Office 17. Dylan Schmorrow, Information Processing Technology Office 18. Barry Wessler, Information Processing Technology Office

Related Personnel

19. Henry Girolamo, U.S. Army, Advanced Technology Manager – Integration & Transition 20. John Murphy, U.S. Navy, DARPA Special Assistant

124 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION

Funding Recipients

Universities

21. Paul Cohen, University of Arizona 22. Les Earnest, Stanford University 23. Edward Feigenbaum, Stanford University 24. Richard Fikes, Stanford University 25. Sebastian Thrun, Stanford University 26. Bill Wulf, University of Virginia 27. Mike Zyda, University of Southern California

Research Labs

28. Jim Allen, Sandia National Labs 29. Nate Blaylock, Institute for Human and Machine Cognition 30. Greg Hebner, Sandia National Labs 31. David Israel, SRI International 32. Cynthia Matuszek, Cycorp 33. Jonathan Salton, Sandia National Laboratories

Corporations

34. Dalibor Hodko, Nanogen 35. Harvey Lehtman, retired (formerly SRI) 36. John Lekashman, Change Research Inc. (formerly NASA) 37. Jeff Rulifson, Sun Microsystems Laboratories 38. Bob Sproull Jr., Sun Microsystems Laboratories

Related Defense Roles

39. Joe Bordogna, National Science Foundation, former Deputy Director and COO 40. Harry Rowen, Hoover Institution, former DoD Assistant Secretary of Defense 41. Larry Schuette, Office of Naval Research, Director of Innovation 42. Col. Jocelyn Seng, U.S. Department of Defense, Acquisition Integration 43. Rudy Darken, Naval Postgraduate School, Professor

Other Researchers

44. Michael Belfiore, freelance technology writer 45. Andreu Veà i Baró, Internet Society (ISOC)

125 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION

Oral History (Interview) Transcripts

The Charles Babbage Institute recorded multiple oral histories from 1988 to 1993, and the transcripts have been made available to the public for free. In addition, separate interviews were also conducted with Vinton Cerf, Stephen Crocker, Robert Kahn, and Steve Lukasik as part of this research study, which are noted with asterisks below.

46. Allan Blue 47. Bruce G. Buchanan 48. Vinton Cerf * 49. Robert S. Cooper 50. Stephen Crocker * 51. Charles M. Herzfeld 52. Robert E. Kahn * 53. J. C. R. Licklider 54. Steve Lukasik * 55. Robert L. Simpson, Jr. 56. Charles A. Zkaret

126 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION

LIST OF REFERENCES

Abetti, P. A. (2000). Critical success factors for radical technological innovations: A five

case study. Creativity and Innovation Management Journal, 9(4): 208–221.

Alvesson, M. (1995). Management of knowledge–intensive companies. New York, NY:

Walter de Gruyter.

Aspray, W., & Norberg, A. (1998, October 28). An interview with J. C. R. Licklider

(transcript). University of Minnesota, MN: Charles Babbage Institute, Center for the

History of Information Processing.

Bacon, F. R., Jr., & Butler, T.W., Jr. (1973). Achieving planned innovation: A proven

system for creating successful new products and services. New York, NY: Simon &

Schuster.

Basit, T. N. (2003). Manual or electronic? The role of coding in qualitative data analysis.

Educational Research, 45(2): 143–154.

Belfiore, M. (2009). The department of mad scientists: How DARPA is remaking our

world, from the Internet to artificial limbs. New York, NY: Smithsonian Books /

HarperCollins.

Betz, F. (2003). Managing technological innovation: Competitive advantage from change

(2nd edition). New York, NY: John Wiley.

Bloom, B. S. (Ed.). (1956). Taxonomy of educational objectives: The classification of

educational goals: Handbook I, cognitive domain. New York, NY: Green.

Bonvillian, W. B. (2006). Power play: The DARPA model and U.S. energy policy.

The American Interest, 39–48.

127 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION

Bonvillian, W. B. (2009). The connected science model for innovation – The DARPA

Role. 21st century innovation systems for Japan and the United States: Lessons from

a decade of change: Report of a symposium. Washington, D.C.: National Academies

Press.

Broad Area Announcement: Augmented Cognition. (2001, July 27). Solicitation Number:

DARPA–BAA 01–38. Commerce Business Daily, PSA #2902.

Broad Area Announcement: Legged Squad Support System. (2008, Oct. 24). Solicitation

Number: DARPA–BAA–08–71. Retrieved from

https://www.fbo.gov/index?id=6c67f326ad4b836ce3217e3b7c56f45c.

Bryant, A., & Charmaz, K. (Eds.) (2007). The SAGE handbook of grounded theory. Los

Angeles: Sage Publications.

Buchanan, R. (2009). Thinking about design: An historical perspective. In A. Meijers,

(Ed.). Philosophy of technology and engineering sciences. Amsterdam, The

Netherlands: Elsevier B.V.

Campbell, V. (2004). How RAND invented the postwar world. Invention & Technology,

20(1): 50-59.

Carleton, T. and Cockayne, W. (2009). The power of prototypes in foresight engineering.

In M. Norell Bergendahl, M. Grimheden, L. Leifer, P. Skogstad, & U. Lindemann

(Eds.), Proceedings of the 17th International Conference on Engineering Design

(ICED’09), 6: 267–276. Stanford, Calif.: The Design Society.

Chesbrough, H. W. (2003). Open innovation: The new imperative for creating and

profiting from technology. Boston, MA: Harvard Business School Press.

128 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION

Christensen, C. M. (1997). The innovator’s dilemma: When new technologies cause great

firms to fail. Boston, MA: Harvard Business School Press.

Christensen, C. M., & Raynor, M. E. (2003). The innovator’s solution: Creating and

sustaining successful growth. Boston, MA: Harvard Business School Press.

Christiansen, J. A. (2000). Building the innovative organization: Management systems

that encourage innovation. New York, NY: St. Martin’s Press.

Chubin, D. E., & Hackett, E. J. (1990). Peerless science: peer review and U.S. science

policy. Albany, NY: State University of New York Press.

Cockayne, W. R. (2004). A study of the formation of innovation ideas in informal

networks (Doctoral dissertation). Available from Dissertations and Theses database.

(UMI No. 3128637)

Cockayne W., & Darken, R. (2004). The application of human ability requirements to

virtual environment interface design and evaluation. In D. Diaper, & N. Stanton

(Eds.), The Handbook of Task Analysis for Human-Computer Interaction. Mahwah,

NJ: Lawrence Erlbaum Associates.

Collins, J. C., & Porras, J. I. (1991). Organizational vision and visionary organizations.

California Management Review, 34(1): 30–52.

Collins, J. C., & Porras, J. I. (1996). Building your company’s vision. Harvard Business

Review, 74(4), 65–77.

Collins, J. C., & Porras, J. I. (1997). Built to last: Successful habits of visionary

companies. New York, NY: HarperBusiness.

Cooper, R. G. (2001). Winning at new products: accelerating the process from idea to

launch (3rd edition). Cambridge, MA: Basic Books.

129 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION

Corbin, J. M., & Strauss, A. (2008). Basics of qualitative research: Techniques and

procedures for developing grounded theory (third edition). Los Angeles, Calif.: Sage

Publications.

Cotterman, R., Fusfeld, A., Henderson, P., Leder, J., Loweth, C., & Metoyer, A. (2009).

Aligning marketing and technology to drive innovation. Research Technology

Management, 52(5): 14–20.

Daniel, H.–D. (1993). Guardians of science: Fairness and reliability of peer review.

Weinheim, Germany: VCH. Retrieved from

http://onlinelibrary.wiley.com/doi/10.1002/3527602208.fmatter/summary

Defense Advanced Research Projects Agency. (1998). DARPA technology transition.

Retrieved from http://www.darpa.mil/Docs/transition.pdf

Defense Advanced Research Projects Agency. (2003, February). Strategic plan.

Defense Advanced Research Projects Agency. (2007, February). Strategic plan.

Retrieved from

http://www.darpa.mil/Docs/DARPA2007StrategicPlanfinalMarch14.pdf

Defense Advanced Research Projects Agency. (2008a). DARPA: 50 Years of bridging the

gap. Retrieved from http://www.darpa.mil/50thanniversary.html

Defense Advanced Research Projects Agency. (2008, April 7). Human Resources (HR)

recruitment – sourcing and placement of DARPA Program Managers (PMs). Request

for Information, reference number SN08–27.

Defense Advanced Research Projects Agency. (2009). Department of Defense Fiscal

Year (FY) 2010 Budget Estimates, Volume 1 – Defense Advanced Research Projects

Agency. Retrieved from http://www.darpa.mil/budget.html

130 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION

Defense Advanced Research Projects Agency. (2010a). Broad Area Announcements.

Retrieved from http://www.darpa.mil/cmo/baa.html

Defense Advanced Research Projects Agency. (2010b). DARPA mission. Retrieved from

http://www.darpa.mil/mission.html

Defense Advanced Research Projects Agency. (2010b). Director’s biography. Retrieved

from http://www.darpa.mil/directorbio.html

Defense Advanced Research Projects Agency. (2010d). Offices. Retrieved from

http://www.darpa.mil/offices.html

Defense Advanced Research Projects Agency. (2010e). Technology transition team.

Retrieved from http://www.darpa.mil/techtransteam.html

Defense AT&L. (2007, November 1). Joining DARPA as a program manager starts with

your idea [advertisement copy].

Defense Science Board. (1999, July). Investment strategy for DARPA (Storming Media

Report No. A014763). Washington, DC: Pentagon reports.

Dekkers, R. (2005). (R)evolution: Organizations and the dynamics of the environment.

New York, NY: Springer Science+Business Media.

Deschamps, J. (2008). Innovation leaders: How senior executives stimulate, steer and

sustain innovation. Hoboken, NJ: Jossey-Bass. de Vaus, D. (2001). Research design in social research. Thousand Oaks, CA: Sage

Publications.

Dougherty, D., & Hardy, C. (1996). Sustained product innovation in large, mature

organizations: Overcoming innovation–to–organization problems. Academy of

Management Journal, 39(5): 1120–1153.

131 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION

Drucker, P. F. (1959). Long–range planning: Challenge to management science.

Management Science, 5(3): 238–249.

Drucker, P. F. (1973). Management: Tasks, responsibilities, practice. New York, NY:

Harper Colophon.

Dundon, E. (2002). The seeds of innovation: Cultivating the synergy that fosters new

ideas. New York, NY: AMACOM.

Edelheit, L. S. (2004). Perspective on GE research & development. Research Technology

Management, 47(1): 49–54.

Ettlie, J. E., Bridges, W. P., & O’Keefe, R. D. (1984). Organization strategy and

structural differences for radical versus incremental innovation. Management Science,

30(6): 682–695.

Fleishman, E., & Quaintance, M. (1984). Taxonomies of human performance: the

description of human tasks. Orlando, FL: Academic Press.

Fleming, L. (2002). Finding the organizational sources of technological breakthroughs:

the story of Hewlett–Packard’s thermal ink–jet. Industrial and Corporate Change,

11(5): 1059–1084.

Fuchs, E. R. H. (2010, forthcoming). Rethinking the role of the state in technology

development: DARPA and the case for embedded network governance. Research

Policy. Retrieved from http://ssrn.com/abstract=1545155

Gassmann, O., & von Zedtwitz, M. (2003). Innovation processes in transnational

corporations. In L. V. Shavinina (Ed.), The international handbook on innovation.

Oxford, UK: Elsevier Science.

132 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION

Glaser, B. G. & Strauss, A. L. (1967). The discovery of grounded theory. New York, NY:

Aldine.

Goulding, C. (2002). Grounded theory: A practical guide for management, business and

market researchers. Thousand Oaks, CA: Sage Publications.

Hafner, K., & Lyon, M. (1996). Where wizards stay up late: The origins of the Internet.

New York, NY: Touchstone.

Hamel, G. (2002). Leading the revolution: how to thrive in turbulent times by making

innovation a way of life. Boston, MA: Harvard Business School Press.

Harvard Business Essentials. (2003). Managing creativity and innovation. Boston, MA:

Harvard Business School Press.

Hebda, J. M., Vojak, B. A., Griffin, A., & Price, R. L. (2007). Motivating technical

visionaries in large American companies. IEEE Transactions on Engineering

Management, 54(3): 433–444.

Hildreth, P. M., & Kimble, C. (Eds.) (2004). Knowledge networks: innovation through

communities of practice. Hershey, PA: Idea Group Publishing.

Hiltzik, M. (1999). Dealers of lightning: Xerox PARC and the dawn of the computer age.

New York, NY: HarperBusiness.

Howe, H. E. (1952). ‘Space men’ make college men think. Popular Science, 161(4):

124–127.

House, C. H., & Price, R. L. (2009). The HP phenomenon: Innovation and business

transformation. Stanford, CA: Stanford Business Books.

Howell, J. M., & Higgins, C. A. (1990). Champions of technological innovation.

Administratively Science Quarterly, 35: 317–341.

133 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION

Hughes, T. P. (1998). Rescuing Prometheus: Four monumental projects that changed the

modern world. New York, NY: Pantheon Books.

Johnson, C. L., & Smith, M. (1989). Kelly: More than my share of it all. Washington,

DC: Smithsonian.

Johnson, M. W. (2010). Seizing the white space: Business model innovation for growth

and renewal. Boston, MA: Harvard Business School Press.

Junod, T. (2003, December 1). Science & industry: DARPA. Esquire. Retrieved at

http://www.esquire.com/features/best–n–brightest–2003/ESQ1203–DEC_DARPA

Kim, W. C., & Mauborgne, R. (2005). Blue ocean strategy: How to create uncontested

market space and make competition irrelevant. Boston, MA: Harvard Business

School Press.

Kostoff, R. N., Boylan, R., & Simon, G. R. (2004). Disruptive technology roadmaps.

Technological Forecasting and Social Change, 71: 141–159.

Kramer, M. W. (2010). Organizational socialization: Joining and leaving organizations.

Cambridge, UK: Polity.

Leheny, R. (2007, August 7). What is a DARPA Program Manager? Speech presented at

DARPATech, DARPA’s 25th Systems and Technology Symposium. Anaheim, CA.

Leifer, R., McDermott, C. M., O’Connor, G. C., Peters, L. S., Rice, M. P., & Veryzer, R.

W. (2000). Radical innovation: How mature companies can outsmart upstarts.

Boston, MA: Harvard Business Press.

Levy, S. (2000). Insanely great: The life and times of Macintosh, the computer that

changed everything. New York, NY: Penguin Books.

134 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION

Lundquist, G. (2004). The missing ingredients in corporate innovation. Research–

Technology, 47(5): 11–12.

Lynham, S. A. (2002). The general method of theory-building research in applied

disciplines. Advances in Developing Human Resources, 4: 221–241.

Lynn, G. S., & Akgün, A. E. (2001). Project visioning: Its components and impact on

new product success. Journal of Product Innovation Management, 18: 374–387.

Maccoby, M. (2007). Developing research/technology leaders. Research–Technology

Management, 50(2): 65–67.

Malakoff, D. (1999). Pentagon agency thrives on in–your–face science. Science,

285(5433): 1476–1480.

Moss, L. T. & Atre, S. (2003). Business intelligence roadmap: the complete project

lifecycle for decision–support applications. Boston, MA: Addison-Wesley.

National Research Council. (2005). Government/industry/academic relationships for

technology development: A workshop report. Steering Committee for Workshops on

Issues of Technology Development for Human and Robotic Exploration and

Development of Space, Aeronautics and Space Engineering Board, Division on

Engineering and Physical Sciences. Washington, D.C.: The National Academies

Press.

Nanus, B. (1992). Visionary leadership: Creating a compelling sense of direction for

your organization. San Francisco, CA: Jossey-Bass.

Norling, P. M., & Statz, R. J. How discontinuous innovation really happens. Research

Technology Management, 41(3): 41–44.

135 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION

O’Connor, G. C., Leifer, R., Paulson, A. S., & Peters, L. S. (2008). Grabbing lightning:

Building a capability for breakthrough innovation. San Francisco, CA: Jossey–Bass.

O’Connor, G. C., & Rice, M. P. (2001). Opportunity recognition and breakthrough

innovation in large established firms. California Management Review, 43(20: 95–116.

Page, L. (2009, January 21). Spy chief to Obama: Let DARPA fix economy. The

Register. Retrieved from

http://www.theregister.co.uk/2009/01/21/darpa_economy_fix_plan/

Pappert, S. (2009). Succeeding with DARPA/MTO: Tips from an ex–Program Manager.

Proceedings from the 2009 Microsystems Technology Office Symposium, March 2–

5.

Penman, S., & Bates, C. C. (1999). DARPA in the spotlight. Science, 286(5438): 239.

Perez, C. (2002). Technological revolutions and financial capital: The dynamics of

bubbles and golden ages. Northampton, MA: Edward Elgar Publishing.

Powell, W. W. (1990). Neither market nor hierarchy: Network forms of organization.

Organizational Behavior, 12: 295–336.

Powell, W. W., & Grodal, S. (2005). Networks of innovators. In J. Fagerberg, D.

Mowery, and R. R. Nelson (Eds.), The Oxford handbook of innovation (pp. 56–85).

New York, NY: Oxford University Press.

Quigley, J. V. (1993). Vision: How leaders develop it, share it, and sustain it. New York,

NY: McGraw–Hill.

Quigley, J. V. (1994). Vision: how leaders develop it, share it, and sustain it. Business

Horizons, 37(5): 37–41.

136 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION

Rappa, M., & Debackere, K. (1993). Youth and scientific innovation: The role of young

scientists in the development of a new field. Minerva, 31(1): 1–20.

Rhea, D. (2003). Bringing clarity to the “fuzzy front end”: A predictable process for

innovation. In B. Laurel (Ed.), Design research: methods and perspectives. Boston,

MA: MIT Press.

Richardson, J. J., Bosma, J., Roosild, S., & Larriva, D. (1999). A review of the

technology reinvestment project. Arlington, VA: Potomac Institute for Policy Studies.

Richardson, J. J., Larriva, D. L., & Tennyson, S. L. (2001). Transitioning DARPA

technology. Arlington, VA: Potomac Institute for Policy Studies.

Rice, M. P., O’Connor, G. C., Peters, L. S., & Morone, J. G. (1998). Managing

discontinuous innovation. Research Technology Management, 41(3): 52–58.

Roberts, E. B. (Ed.). (1987). Generating technological innovation. New York, NY:

Oxford University Press.

Roland, A., & Shiman, P. (2002). Strategic computing: DARPA and the quest for

machine intelligence, 1983–1993. Cambridge, MA: MIT Press.

Schilling, M. A. (2005). Strategic management of technological innovation. New York,

NY: McGraw–Hill/Irwin.

Schön, D. A. (1983). The reflective practitioner: how professionals think in action. New

York, NY: Basic Books.

Schumpeter, J. (1942). Capitalism, socialism, and democracy. New York, NY: Harper &

Row.

Schrage, M. (1999). Serious play: How the world’s best companies simulate to innovate.

Boston, MA: Harvard Business School Press.

137 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION

Siedman, I. (2006). Interviewing as qualitative research: A guide for researchers in

education and the social sciences. New York, NY: Teachers College Press.

Sigismund, C. G. (2000). Champions of Silicon Valley: Visionary thinking from today’s

technology pioneers. New York, NY: John Wiley.

Smith, P. G., & Reinertsen, D. G. (1998). Developing products in half the time: New

rules, new tools (2nd edition). New York, NY: John Wiley.

Snyder, N. T., & Duarte, D. L. (2003). Strategic innovation: Embedding innovation as a

core competency in your organization. San Francisco, CA: Jossey-Bass.

Stringer, R. (2000). How to manage radical innovation. California Management Review,

42(4): 70–88.

Skarzynski, P., & Gibson, R. (2008). Innovation to the core: A blueprint for transforming

the way your company innovates. Boston, MA: Harvard Business School Press.

Tennenhouse, D. (2004). Intel’s open collaborative model of industry–university

research. Research Technology Management, 47(4): 19–26.

Tether, T. (2005, May 12). Multidisciplinary research. Presented to Committee on

Science, United States House of Representatives. Retrieved from

http://www.darpa.mil/testimony/HseSciCom51205url.pdf

Tornatzky, L. G., Fleischer, M., & Chakrabarti, A. K. (1990). The processes of

technological innovation. Lexington, MA: Lexington Books.

Tzeng, C. (2009). A review of contemporary innovation literature: A Schumpeterian

perspective. Innovation: Management, Policy & Practice, 11: 373–394.

United States Department of Defense. (2008, July). DoD’s Financial Management

Regulation (Publication No. DOD 7000.14–R), volume 2B, chapter 5 (Research,

138 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION

Development and Evaluation Appropriations). Washington, DC: U.S. Government

Printing Office.

Urban Dictionary. (2010). DARPA hard. Retrieved from

http://www.urbandictionary.com/define.php?term=DARPA%20hard

Van Atta, R. H., Lippitz M. J., Lupo, J. C., Mahoney, J., & Nunn, J. H. (2003a).

Transformation and transition: DARPA’s role in fostering an emerging revolution in

military affairs. Volume 1 – Overall assessment. Washington, DC: U.S. Government

Printing Office.

Van Atta, R. H., Cook, A., Gutmanis, I., Lippitz, M. J., Lupo, J., Mahoney, R., & Nunn,

J. H. (2003b). Transformation and transition: DARPA’s role in fostering an emerging

revolution in military affairs. Volume 2 – Detailed assessments. Washington, DC:

U.S. Government Printing Office.

Van Atta, R. (2008). Fifty years of innovation and discovery. In DARPA: 50 years of

bridging the gap. Washington, DC: U.S. Government Printing Office.

Van de Ven, A. H., Angles, H. L., & Poole, M. S. (2000). Research on the management

of innovation: The Minnesota studies. New York, NY: Oxford University Press.

Van de Ven, A. H., Polley, D. E., Garud, R., & Venkataraman, S. (1999). The innovation

journey. New York, NY: Oxford University Press.

Vojak, B. A., Griffin, A., Price, R. L., & Perlov, K. (2006). Characteristics of technical

visionaries as perceived by American and British industrial physicists. R&D

Management, 36(1): 17–26.

Waldrop, M. M. (2001). The dream machine: J. C. Licklider and the revolution that made

computing personal. New York, NY: Viking Press.

139 THE VALUE OF VISION IN RADICAL TECHNOLOGICAL INVENTION

Watts, R. M. (2001). Commercializing discontinuous innovations. Research–Technology

Management, 44(6): 26–31.

Weinberger, S. (2006). Imaginary weapons: A journey through the Pentagon’s scientific

underworld. New York, NY: Nation Books.

140