JOURNAL OF SMALL BUSINESS

S TRATEG Y

CUSTOMER DEVELOPMENT, INNOVATION, AND DECISION-MAKING IN THE LEAN STARTUP

Jonathan L. York California Polytechnic State University [email protected]

Jeffrey E. Danes California Polytechnic State University [email protected]

ABSTRACT

This paper reviews current research relevant to new product development, customer development, and the lean startup. Customer development and the lean startup are a new and increasingly used form of entrepreneurship process, which rely on hypothesis testing but not in the traditional sense; the entrepreneur is encouraged to scan the environment, collect information, and form and evaluate educated guesses so as to make accurate judgments and decisions. The present research provides a review of the customer development model for entrepreneurial activities and a critique of this hypothesis testing methodology. We then consider ways in which to improve decision making within the startup via a systematic study of System 1 ( intuition) and System 2 ( reasoned and rational) decision-making styles. This paper has significant implications for entrepreneurs, entrepreneurship support organizations, such as incubators and accelerators, and entrepreneurship educators, all of whom are increasingly practicing and teaching this process.

Keywords: biases and heuristics, customer development, entrepreneurship, innovation, lean startup, new product development

INTRODUCTION 2011; Maurya, 2012). For purposes of this paper, the lean startup process is defined as This paper discusses a newly emerging and an approach to entrepreneurial and increasingly popular approach to innovative activities that emphasizes entrepreneurial practice known as customer placing resources into the creation of development and the “lean startup” process customer value, viewing all other activity as (Blank, 2007; Blank & Dorf, 2012; Ries, waste until a fit is found between the

21 Journal of Small Business Strategy Vol. 24, No. 2 product and the intended market. The lean need to validate this intuition with more startup relies upon a process called formal processes that make sense in the customer development, which is a method highly uncertain and time-sensitive startup of creating and testing assumptions environment. Below we briefly review regarding the end business model for the formal methods of hypothesis testing and startup. The lean startup process is rapidly review Blank’s notion of "hypothesis being adopted by university testing” and find it lacking in rigor. Using entrepreneurship programs, accelerators, the concept of System 1 (intuitive) and entrepreneurial organizations like Startup System 2 (systematic, reasoned) thinking Weekend, communities, and even the NSF (Stanovich and West, 2000), we then (2012), which is employing this suggest methods which could be integrated methodology for its Innovation Corps Sites into the customer development process to Program. However, there has been very mitigate the significant risks of these biases, little if any systematic analysis of the lean and place these in the context of startup and customer development entrepreneurial practice and teaching. methodologies. Therefore, we place customer development within the CUSTOMER DEVELOPMENT IN THE theoretical context of new product CONTEXT OF NPD development; the methods by which the entrepreneur determines the viability of the New product development over the past two innovation as well as the needs, features, decades has been heavily influenced by the and functionality for the product clearly fall “stage-gate” model, developed by Cooper under the overarching framework of new (1988, 2001). A significant benefit of the product development whether in an traditional NPD model is its structure, entrepreneurial or more stable environment. allowing companies to follow a roadmap We review customer development as an and prescriptive steps guiding their process. entrepreneurial practice within the context Yet, according to Cooper, the traditional of earlier product development models such new product development process was as Cooper’s New Product Development designed for incremental product (NPD) (Cooper 1988; Cooper 2008) and development and “... may be inappropriate Koen’s (2004) new concept model for the …” (Cooper, 2001, p. 151) when applied to fuzzy front end (FFE). breakthrough and platform projects. A limitation of the early, traditional NPD Within this theoretical framework, we turn model is the emphasis on incremental to a key issue in customer development. product improvement with a cursory focus More than earlier forms of NPD, customer on ideation and discovery. Other limitations development relies heavily on hypothesis include a defined linear process with the testing, but not in the traditional sense. implication that “looping back” to an earlier Instead, the entrepreneur is encouraged to stage denotes a gating mistake or gating survey the environment, collect error in development. In addition, product information, and form and evaluate development cycles, sped up today by educated guesses so as to make accurate competitive markets and enabled by judgments and decisions. Customer technologies such as rapid prototyping development recognizes that there may be a (even on the desktop with 3D printing for place for “intuition” within the example) and internet distribution, can be entrepreneurial process and responds to the slowed significantly by the traditional NPD 22 Journal of Small Business Strategy Vol. 24, No. 2 approach. In addition, stage gate may contrasts with the traditional, sequential actually lead to a difficulty delivering final NPD process where “looping back” may be products because each stage has higher viewed as a gating error as opposed to a costs and may take longer as the company reasoned, positive correction. moves through the process (Anderson, 1993). The traditional NPD model proposed In contrast to the traditional NPD process, by Cooper (1988) and as modified (Cooper, new concept development processes include 2001) may be viewed as a very successful a greater focus on idea generation activities attempt at a very logical and coherent such as in-depth interviews, brainstorming, methodology of launching new products. and idea management tools (e.g., Cooper emphasizes the need for customer- crowdsourcing) aimed at potential and facing activities in the earliest stages of current customers, lead users, employees, NPD, with the realization that many and other stakeholders. Early innovation projects fail because of an overemphasis of FFE activities are often disorganized, the technical tasks over the unpredictable, and unstructured, as opposed marketing/business-oriented tasks during to the later phases of new product what he calls the “homework” or development which are typically a more predevelopment stage (Cooper 2011; structured and formalized process. Koen Cooper 2013; Cooper, 2013a). As Edgett defines the FFE as all activities that come (2011) notes, activities taken before the before product development (Stage 3) of the formal design and development of the five steps of the traditional NPD process. product play a key role in determining Likewise, Brentani and Reid (2012) define success or failure. Cooper (2014) himself FFE as “the time and activity prior to an has recognized the need for a more flexible organization’s first screen of a new product process and has recently proposed “the idea, “ (p. 70), implying the FFE “ends” at Triple A System” of an adaptive, agile and the point of traditional “product accelerated modification to Stage Gate (p. development” activities. In other words, it 21). can be argued that NPD and FFE processes are product-centric and downplay the There is an abundance of research showing ensuing business development activities, how greater emphasis on the front-end taking for granted that these processes are activities (meeting the needs of customers) embedded in a larger company’s already explains why some products are more existing structure. Blank (2007) on the other successful than others (Henard & hand carries his customer development Szymanski 2001). In studying a number of approach all through the new product industries, Koen (2004) observed that development and launch process, arguing communication and decision making is that any functional systematization (e.g., circular, nonlinear, and non-sequential, “company building”) occurs and continues rather than following the linear model of after product launch, and that customer NPD. The new concept model (NCD) development is essentially never truly developed by Koen (2004) was designed to “completed.” emphasize that ideas are thought to flow, circulate, and iterate between and among Table 1 describes some key differences five elements: opportunity identification, between traditional new product opportunity analysis, idea generation, idea development (NPD), the fuzzy front end selection, and concept definition. This (FFE) and as described by Koen et al. 23 Journal of Small Business Strategy Vol. 24, No. 2 Table 1: Differences between Traditional NPD, FFE, and CD

New Product Fuzzy Front End Customer Development (NPD) (FFE) Development (CD) Nature of Work Disciplined and goal- Experimental, often Iterative, with oriented with a project chaotic. “Eureka” continual influx plan moments. of new Can schedule work— information but not invention processing hypotheses Commercialization High degree of certainty Unpredictable or Higher degree at conclusion uncertain of certainty after completion of CD process Funding Date Budgeted Variable — in the Only for beginning phases minimum viable many projects may product until be “bootlegged,” business and while others will sales model need funding to developed proceed Revenue Predictable, with Often uncertain, with Must know Expectations increasing certainty, a great deal of sales and analysis, and speculation pricing model documentation as the for first stage product release date gets introduction closer. Activity Multifunction product Individuals and team All members and/or process conducting research involved in development team to minimize risk and extensive optimize potential “outside of the building,” largely 1:1 customer contact Measure of Progress Strengthened Minimum Milestone concept viable product; achievement product/market fit Expenses Increase with each stage Increase with each Revenue can stage begin after MVP identified Decision Process Go/No Go Kill stages Indeterminate Pivots to new directions

24 Journal of Small Business Strategy Vol. 24, No. 2 (2002) and Koen (2004), and customer Proponents of customer development assert development as described by Blank (2007). that the greatest risk for failure for any innovative product is not development of Customer Development (CD) the product but the lack of product-market Over the past several years, a new approach fit. Good fit drives growth. Poor fit can to the fuzzy front end, loosely grouped manifest itself in a lack of customers, and/or under the rubric of “customer difficulty in identifying customers, and/or development,” has been advocated and the higher costs involved in finding promoted by Blank (2007) and others, such customers for the innovation. Methods of as Blank and Dorf (2013), Cooper and mitigating this risk of poor fit are of Vlaskovits (2013), and Maurya (2012). The considerable value to the effort. As such the customer development model considers four focus is on the early stages of innovation. interlocking and circular stages: 1) Proponents of a structured CD approach customer discovery, a focus on (Reis, 2012; Blank & Dorf, 2012) understanding customer problems and recommend a significant degree of needs, 2) customer validation, the “certainty” in the product-market fit before identification of a scalable and repeatable the company pursues building a formalized, sales model, 3) customer creation, creating complete business plan. Mullins (2013) also and driving end user demand, and 4) argues for significant customer company building, the transitioning of the development activity prior even to the organization from one designed for learning launch of a business. As well as arguing for and discovery to efficient execution. A more traditional analysis of factors such as unique feature is that, following the initial competition and market conditions, he “hunches” of the entrepreneurial team proposes a process of customer regarding their business model, customer interviewing very similar to customer development is almost entirely a process of development approaches. direct contact with customers and others outside the company for knowledge Another key feature of CD is product acquisition and hypothesis testing (Blank & simplicity, often called the “minimum Dorf, 2012). All assumptions are to be viable product.” In this model, the product challenged through the first phase of CD. A is a set of “minimal requirements,” which product should be launched (“put into the meet the needs of the core group of early hands of customers”) as soon as possible to adopters or users. Also, cash burn is kept to increase the level of feedback. As Blank a minimum until creating and driving end (2007, p. 21) notes, “In essence, Customer user demand (customer creation), and Discovery and Customer Validation unlike the traditional NPD model, there are corroborate your business model. no kill gates, only continual iteration and Completing these first two steps verifies reintroduction. Using the principles of the your market, locates your customers, tests lean startup (Ries, 2011), Blank & Dorf the perceived value of your product, (2012) argue for shipping product to early identifies the economic buyer, establishes adopters rapidly and then following up with your pricing and channel strategy, and customer driven improvements, creating an checks out your sales cycle and process.” iterative product development process. The Only then, according to Blank, does the lean startup methodology allows for company move to creating the full business significant changes throughout the process, plan and company development. often called “pivots,” (Ries, 2011). 25 Journal of Small Business Strategy Vol. 24, No. 2 DelVecchio, White and Phelan (2014) note this can be done in a completely iterative that with traditional NPD, “If customer process (such as for a website that is feedback indicates a major redesign is upgraded constantly) or in a step-by-step required, the product will likely cease process, such as for a physical product moving on to the next stage and the where prototypes must be tested regularly. gatekeeper will “kill” the development of For Blank, hypothesis generation is the product.” (p. 7) essentially a founding team process based upon intuition into the market, beginning CUSTOMER DISCOVERY AND with educated guesses about the nature of HYPOTHESIS TESTING the problem(s) and the target market the Proponents of customer discovery company proposes to address. emphasize hypothesis development and testing as an integral part of the customer Hypothesis testing follows these intuitions development process. Below we review about the business model and is conducted Blank’s interpretation of “hypothesis by what Blank (2007) calls “getting out of testing,” followed by a brief discussion of the building” – in other words, talking to more formal models of hypothesis testing customers, users, and experts, to determine (frequentist and Bayesian). Next we make whether the problem, which the startup an argument that a more attainable goal is to posits, is indeed significant and what the focus on the frailties of entrepreneurial scope and characteristics of that problem decision-making, and then consider ways to may be. The hypothesis evaluation step at improve it. this stage is again one of team review and discussion, a group process. The next step is A key process in customer development is a continuation of the first, wherein a the testing of hypotheses about the prototype or first iteration of product, what problems faced by customers, the minimum Ries (2011) deems the “minimum viable feature set of the product, the product,” is exposed again to the customer product/market fit, product improvements, and hypotheses regarding its value are again etc. Hypothesis development is concerned tested. For Blank and other adherents of with how we determine what we initially customer development, this process does think is “true” about the world and, given not end, as each new decision involves appropriate information, what then appears another set of hypotheses that must be to be actually “true” about the world. The generated, tested, and evaluated. Each outcome of such testing guides decision tested hypothesis yields a decision, a course making. Proponents of CD define a three- of action. For Blank, this process is almost stage process, which includes a first stage of entirely derived from subjective information generating hypotheses, followed by testing obtained by direct one-to-one contact with the hypotheses, and finally, using the results potential customers, suppliers, partners, etc. of testing to evaluate the hypothesis. The challenge for the entrepreneur, as Blank As noted, Blank’s (2007) CD process is (2013) suggests, is to generate and test built on a continual cycle of hypothesis hypotheses rapidly and frequently, and testing, decisions, and corrections, which evaluate and reevaluate results before lends itself well to situations of highly coming to the decision-making stage and novel product introductions where there is identifying a course of action. Depending direct contact with potential consumers. In on the type of new product or new business, this way, entrepreneurial decision-making 26 Journal of Small Business Strategy Vol. 24, No. 2 may indeed be viewed as a form of experts, etc., to determine whether what the scientific process of discovery and learning startup sees as a problem is indeed one, and rather than a purely intuitive process of what the scope and characteristics of that recognition and alertness to market problem may be. As we will see later, this conditions. (Harper, 1999). Blank’s form of informal hypothesis testing is prone customer development hypothesis to certain biases and heuristics which can development and evaluation is a Bayesian impair its accuracy. decision-making process, as proposed by There are two general types of hypothesis Fischhoff and Beyth-Marom (1983). That testing methods: frequentist and Bayesian. is, one develops hypotheses (prior belief) The classical frequentist-based hypothesis based upon experience or intuition, testing is a formal statistical process; identifies data sources useful to evaluate however, within the social context of hypotheses, observes or gathers data to entrepreneurial activities, classical evaluate the truth of competing hypotheses, hypothesis testing is problematic given aggregates data into an overall appraisal of limited information, subjective opinion, and likelihood of a hypothesis being accurate, the need for rapid decisions. On the other and selects a course of action based upon hand, Bayesian inference is a normative that evaluation. The ultimate goal is to method in the sense of prescribing how make good decisions. Yet, as is well known, hypotheses should be evaluated given prior true Bayesian decision making requires information and new information. The significant amounts of data, which the Bayesian model is better suited to the entrepreneurial environment may not analysis of subjective information and provide, nor the entrepreneur be able to opinion, which makes it much more implement. relevant to entrepreneurial decision-making

and customer development. Nevertheless, Hypothesis testing in scientific efforts is the data requirements are tedious and thus grounded in rigorous statistical thinking. appear to be impractical relative to decision Formal models for hypothesis testing are making in the entrepreneurial context. The logical and systematic processes. The complications of elementary Bayesian essence of hypothesis testing is concerned decision making are discussed in Lindley with how one decides whether there is a (1985). Therefore, instead of focusing on match between what we initially think is hypothesis testing, either classical or true and what the evidence (data) shows is Bayesian, we focus on a more attainable not true. That is, is the evidence strong goal, reducing the biases inherent in enough to reject the corresponding null entrepreneurial decision making, especially hypothesis? In practice, effective decision in the context of customer development. making follows from “truth” as determined by statements of hypotheses (educated Risks of Customer Development from guesses), the testing (evaluation) of Biases and Heuristics hypotheses, and the verification of To set the stage for reducing in hypotheses. However, Blank’s (2007) decision making, we now examine some of entrepreneurial hypothesis testing methods the most significant biases and risks in are informal and tend to rely on intuitive customer development hypothesis testing thinking as noted above and discussed and then, using the concepts of system below. Blank (2007) refers to “hypothesis thinking (Stanovich & West, 2000), we testing” as talking to customers, users, examine research findings relevant to 27 Journal of Small Business Strategy Vol. 24, No. 2 reducing the bias in decision making. We large organizations. Schade and Koellinger then propose ways in which the practitioner (2007) review a detailed list of perceptual of customer development in the biases and heuristics in general that affect entrepreneurial context can improve entrepreneurial decisions, including whether decision-making. to start a business. Several of these biases and heuristics seem to carry the most risk Entrepreneurs tend to be overly active, face for the customer development process: time constraints, and hence, tend to rely on intuition. According to Stanovich and . In its most common West (2000), intuition as a basis for manifestation, the data that an entrepreneur decision making is fast, automatic, gathers may be biased if she looks only to effortless, implicit, and emotional (referred friends, colleagues, and known sources for to as System 1). System 2 refers to testing her hypotheses. Blank argues that reasoning, which is slower, conscious, “getting out of the building” is a hedge effortful, explicit, and logical. Levels of against this bias, but many entrepreneurs System 2 thinking include unstructured, still gravitate to comfortable and clinical, and assisted (e.g., training). An confirmatory sources. (Holcomb et al., excellent discussion of the implications of 2009) Stanovich and West’s (2000) work is given in Stanovich (2010) and in Kahneman Representativeness bias. In the dynamic (2011). Entrepreneurs often lack important and uncertain startup environment, it is information regarding a decision, fail to perhaps natural to generalize from small, notice available information, and face time non-random samples of data. Yet, to the and cost constraints; hence, they tend to rely extent that this bias interacts with the on intuitive System 1 thinking. But, reliance selection bias noted above, the validity of on System 1 thinking, intuition, has certain customer development information gathered inherent weaknesses which may result in can be severely compromised. poor decisions. As Kahnemann (2011) notes, expert intuition is most trustworthy in Acquiescence bias. Related to selection an environment that is regular and bias, this represents the tendency of predictable, and where the expert has had respondents to give the answers they think sufficient practice to learn the regularities of the entrepreneur wants to hear, rather than this environment, scarcely descriptive of the their unvarnished opinion. This is often startup situation. referred to in the startup world as “not wanting to tell someone their baby is ugly.” Given that many entrepreneurial activities rely upon subjective and sparse information, . People tend to favor or this decision making is prone to significant interpret information in a way that confirms errors in judgment. The presence of such their prior beliefs. If we “believe” that a forms of error in inquiry is well problem exists, we will listen for any documented (Tversky & Kahnemann, 1974; evidence that “confirms” this belief and Holcomb, Ireland, Holmes & Hitt, 2009). ignore all else. Busenitz & Barney (1997) established specific differences in two biases Overconfidence bias. “Overconfidence is (overconfidence and representativeness) the tendency for people to overestimate between entrepreneurs and managers in their knowledge, abilities and the precision 28 Journal of Small Business Strategy Vol. 24, No. 2 of their information” (Bhandari & Deaves, thrives on intuitive (System 1) 2006, p. 5). In many ways, overconfidence entrepreneurs – without them there would is essential for the entrepreneur to be able to be no startups. Purely relying upon this act in an environment of such uncertainty intuitive and error-prone approach is too (Koellinger, Minniti & Schade, 2007), but often a recipe for failure; yet the prior when overconfidence leads to the blocking “System 2” methodologies used in out of new evidence or alternative innovation development, such as the perspectives, it can be detrimental to the traditional NPD process, may be customer development process. inappropriate substitutes for the entrepreneur due to time and information . Kahneman (2011) notes that constraints, and do not take into the account optimism is an essential characteristic of the the highly uncertain nature of the entrepreneurial mindset but that it can lead entrepreneurial environment. Customer to a strong tendency to ignore the relevant development seems to address this gap by evidence. Cooper, Woo & Dunkelberg offering a framework and techniques for (1988) found entrepreneurs indeed testing the hypotheses that the startup has experienced extreme optimism no matter created through its more intuitive processes. the degree to which they were likely to Nevertheless, even moving to a “hypothesis succeed based on objective factors such as testing” approach in the startup does not experience and the nature of the new remove the risk that these very same biases business. In fact, optimism may be will manifest themselves in the sources of detrimental in the decisions of what kind of information, the type of information sought, risk to take on, but essential in the actual and the interpretation of that information. In implementation of the business once these the following section, we suggest certain decisions are made. For the startup approaches that may help to mitigate the entrepreneur, placing himself in the risks of System 1 biases in the customer appropriate environment and with a group development process. Table 2 cross of teammates who will help temper this references these bias-mitigating techniques optimism bias in the early stages can against the specific biases inherent in mitigate the risk of poor decisions. Slater, customer development. Mohr, and Sengupta (2013) note that an organizational culture which emphasizes There is significant research that shows that team interaction; however informal, leads to System 1 thinking can be improved and that more successful radical product innovation System 2 thinking can be encouraged. For capability. For innovators, this means example, a review of research by Fischhoff “getting outside one’s head” and allowing (1982) on four strategies as potential all input, even that critical of the idea. solutions for biased decision making found extended training and feedback, coaching, Reducing Bias in Customer Development and other interventions to improve Processes judgment. In specific, they found training An entire body of work has evolved over and feedback to be superior to (1) offering the last 20 years to illuminate the warnings about the possibility of bias; (2) differences between intuitive (System 1) describing the direction of a bias; and (3) thinking and System 2, a more rational, providing small doses of feedback. logical and calculating approach. As we However, extended training with feedback have noted above, the startup environment only produced only moderate improvements 29 Journal of Small Business Strategy Vol. 24, No. 2 Table 2: Bias Mitigation Techniques in Customer Development

Bias Description Mitigation techniques

Selection bias seeking information from founding team members pair up in “friendly" confirmatory sources “getting out of the building” and resulting in unrepresentative check each other’s assumptions at of the target market(s) all times careful attention to selection of interview targets

Representativeness bias generalizing from small, non- conducting customer development random samples of data and/or interviews as an iterative and information from respondents continuous process, checking who do not represent the target previous generalizations against market(s) new data

Acquiescence bias respondents’ tendency to give the carefully structuring customer answers they believe the development interviews to avoid entrepreneur wants to hear “yes/no" answers; not asking respondents to speculate on future behavior, but focusing on past and current behavior

Confirmation Bias interpreting information to maintaining open-ended interview confirm prior beliefs discussion focused on problems, not proposed solution improper linear model development to simulate data-driven decision where actual measurement data is impractical if not impossible

Overconfidence bias overestimating the knowledge engaging mentors and advisors to and precision of customer provide an unbiased perspective suggestions and/or the on the information gathered and entrepreneurs information to create an environment of System 2 thinking around the startup premortem exercises looking at potential causes of failure rather than assuming success; a belief that only the paranoid survive

Optimism bias the entrepreneur’s belief that analogical reasoning - engaging in he/she is unlikely to experience “reference class forecast" negative outcomes or fail activities comparing to other similar startups"consider the opposite” activities to force more critical thinking

Some general bias mitigation activities include: • locating in an environment with other startups and advisors (incubator, co-working) where external feedback is readily available • undergoing training in customer development activities to strengthen information-gathering skills • conducting all customer development activities as a team • engaging a mentor or advisor to play the devil’s advocate role in all customer development activities

30 Journal of Small Business Strategy Vol. 24, No. 2 in decision making. The training research provides a lucid discussion of this research. reviewed by Fischoff (1982) may be viewed To do this requires the entrepreneur to be as encouraging System 2 type thinking. The open to negative input, not an easy task. training research reviewed by Fischoff (1982) may be viewed as encouraging Encouraging people to “consider the System 2 type thinking. In the startup opposite” of whatever decision they are environment, team members can provide about to make tends to reduce errors in checks and balances, in a sense offering to judgment caused by various biases: each other the initial training and feedback overconfidence, the , and that can prevent against the injection of bias anchoring (Larrick, 2004; Mussweiler, into the customer development process. In Strack, & Pfeiffer, 2000). Reduced error in fact, Blank (2005) insists that all members judgment has also been achieved by having of the founding team, regardless of groups rather than individuals make discipline, take part in the “out of the decisions, training individuals in formal building” hypothesis testing activities in reasoning, and making people accountable order to eliminate the natural tendency for for their decisions (Larrick, 2004; Lerner & engineers to make their assumptions, Tetlock, 1999). Thus, while the startup team business types to make theirs, and the result is most accountable for decisions during the to be a compromise between opinions rather customer development activities, having an than some form of “fact,” a strongly held outside set of advisors, or an advisory board prior belief. can insert a level of bias-reduction into the process. There are other mechanisms that tend to encourage System 2 thinking. Kahneman Analogical reasoning research has and Lovallo (1993) discuss attempting to examined how System 2 thinking can be take an outsider’s perspective, that is, trying leveraged to reduce System 1 errors. to remove oneself mentally from a specific Analogical reasoning can be used to reduce situation or to consider the class of bounds on awareness (see Bazerman & decisions to which the current problem Chugh 2005). Stemming from work by belongs. For example, an entrepreneur Thompson, Gentner, and Loewenstein could look at similar type startups to get (2000), Idson, Chugh, Bereby-Meyer, what Flyvbjerg (2009) calls a “reference Moran, Grosskopf, and Bazerman (2004) class forecast” of how much money it took found people who were encouraged to to get to launch, how long until comprehend common principles underlying profitability, etc. Taking an outsider’s seemingly unrelated tasks improved in their perspective has been shown to reduce ability to discover solutions in a different decision makers’ overconfidence about their task that relied on the same set of knowledge (Gigerenzer, Hoffrage, & underlying principles. For example, an Kleinbölting, 1991), the time it would take entrepreneurial team can study the success them to complete a task (Kahneman & or failure of a startup in a separate but Lovallo, 1993), and their odds of related field to open up their awareness to entrepreneurial success (Cooper et al., contradictory rather than only confirmatory 1988). Decision makers may also be able to evidence. improve their judgments by asking a genuine outsider for his or her view Recent research on joint-versus-separate regarding a decision. Kahneman (2013) decision making indicates that people can 31 Journal of Small Business Strategy Vol. 24, No. 2 move from intuitive System 1 thinking decision in the planning stage, and then is toward the rational System 2 thinking if asked to fast forward a year or two and they contemplate and then choose between imagine that the decision has failed multiple options simultaneously rather disastrously. By proactively discussing the doing the same separately for each option. hypothetical reasons for this failure, the Research by Bazerman, White, and team can counteract the overconfidence Loewenstein (1995) suggests that choice bias. options should be evaluated jointly rather than sequentially. For example, suppose the One of the most successful System 2 team is considering the match between three decision-making mechanisms is the different product embodiments with three improper linear model research associated different target markets. Their research with Robyn Dawes. Dawes and Corrigan shows that evaluating the elements of the (1974), in their seminal article, report set jointly, collectively at one point in time, simple linear models to be superior to is superior to their evaluation sequentially at clinical-type judgment and to simulate a different points in time. Consistent with this System 2 t ype of decision making. Dawes work, Bazerman, Loewenstein, and White and Corrigan (1974) report that simple (1992) have found people exhibit more linear models work well with input willpower when they weigh their choices variables that have a conditionally jointly rather than separately. That is, a monotonic1 relationship with the output. collective decision has more resolve than an Numerous studies report that linear models aggregation of individual decisions. produce predictions that are superior to those of experts across an impressive array The entrepreneurial environment itself can of domains, see Hastie and Dawes (2010). be staged to encourage better decisions Moreover, Dawes (1971) has shown that given System 1 thinking. Thaler and even improper linear models outperform Sunstein’s (2003) work on “choice clinical judgments. The remarkable architects” suggests that mentors, implication of the above research shows accelerators, and incubators can design that clinical and intuitive System 1 thinking situations so as to encourage better choices, can be transformed into rational System 2 given known decision biases. Having an by 1) identifying potential predictors of an environment around the entrepreneurial outcome, and then 2) providing a guess at team of people who can question the input the weights in a linear model. The weights from the hypothesis testing and provide could be as simple as: -1, 0, or +1, for a neutral feedback can serve as a de facto negative, neutral, or positive relationship or “System 2” overlay upon the process. In be a simple rating system: 0,1,2,3,4 addition, the entrepreneurs in these followed by the identification of the programs can be trained, and supported, in algebraic sign (+,-) denoting the reducing bias in their customer relationship of the variable with the development processes, and be put into outcome. cross-team situations that encourage analogical reasoning to expand bounds of awareness. An example of a technique that can be employed is the premortem method 1 suggested by Klein (2007). In essence, the A sequence {a(n)} such that either (1) a(i+1)>=a(i) for every i >=1, or (2) entrepreneurial team “makes” a key a(i+1)<=a(i) for every i>=1. 32 Journal of Small Business Strategy Vol. 24, No. 2 Linear regression produces optimal weights; standards for determining which however, Dawes (1971) has shown that babies were in trouble, and the linear models with non-optimal weights are formula is credited for an important also superior to intuitive (System 1) contribution to reducing infant judgments. This holds promise for the mortality. The Apgar test is still creation of models wherein there is little used every day in every delivery possibility of fully accurate measurement. room.” When error is introduced through measurement rather than the item being The predictive accuracy of the improper measured, deviations from optimal linear model, the Apgar score, has weighting do not make much practical subsequently been validated against difference in the quality of decisions. Good extensive outcome data (Moster, Lie, predictions may be obtained by selecting Irgens, Bjerkedal & Markestad, 2001; variables that have some validity for Carter, McNabb & Merenstein, 1998). predicting the outcome. An improper linear model with subjective weights guessed at The amazing success of subjectively by the user is likely to be just as accurate in defined weighting schemes has an predicting new cases as is ordinary least important practical implication for the squares regression. The classic example is entrepreneur as it is possible to develop a the Apgar score for new born babies, as useful predictive model without any prior described by Kahneman (2011, p. 277). data collection such as that required for building a regression model. For example, a “One day over breakfast, a medical startup can develop a simple and useful resident asked how Dr. Apgar would improper linear model by creating a set of make a systematic assessment of a key variables that the entrepreneurial team newborn. “That’s easy,” she replied. feels is valuable to measure, providing a “You would do it lik e this.” Apgar subjective weighting system to these jotted down five variables (heart variables, and then using the customer rate, respiration, reflex, muscle tone, development process to assign scores to and color) and three scores (0, 1, or each of these. A core framework for this 2, depending on the robustness of would be the Business Model Canvas each sign). Realizing that she might (Osterwalder & Pigneur, 2010) which have made a break through that any breaks down business models into nine delivery room could implement, graphically connected segments. The Apgar began rating infants by this startup might determine, for example, that rule one minute after they were the key variables from the Business Model born. A baby with a total score of 8 Canvas are the potential revenue models, or above was likely to be pink, the customer segments, and the presence of squirming, crying, grimacing, with a key partners in the distribution channel. By pulse of 100 or more-in good shape. simultaneously evaluating each of its A baby with a score of 4 or below possible alternatives here, and assigning was probably bluish, flaccid, each a subjective “score,” the passive, with a slow or weak pulse- entrepreneurial team has essentially created in need of immediate intervention.” a predictive improper linear model. It is Applying Apgar’s score, the staff in critical that these evaluations be conducted delivery rooms finally had consistent in a simultaneous rather than sequential 33 Journal of Small Business Strategy Vol. 24, No. 2 manner; in other words, the process suffers that isn’t a part of one or another of the if the company eliminates one possibility earlier new product development before moving on to the next instead of frameworks, but CD’s emphasis on full scoring them all. As noted, even though team participation, continuing face-to-face these items and their weights are user- customer contact, and reliance on “out of defined and non-optimal, a decision based the building” work have the potential of upon this process brings non-biased System eliminating some of the biases inherent in 2 thinking into the process. the innovation process. On the other hand, customer development has its own risks that DISCUSSION, CONCLUSIONS, AND need to be acknowledged and addressed. FUTURE RESEARCH For example, there can be a tendency, especially for the first-time entrepreneur, to The review and study of customer become more focused on the activity of development and lean launch customer development than the outcomes, methodologies is important to innovation in essence to rely too much on fine tuning and entrepreneurial activities, as these these results at the expense of their processes are becoming well instantiated at intuition. In other words, customer the practitioner level. The speed required development is no more inoculated from for successful innovation will not abate, and “paralysis by analysis” than any other NPD thus, both academics and practitioners will processes. Furthermore, there is a key role continue to seek more effective frameworks for actual data collection and analysis in the and processes. The concept of lean startups, early stages of the startup (Croll & driven by such tools as customer Yoskovitz, 2013) through the introduction development, promises to remain an of key metrics and testing. Customer important part of the entrepreneurial development can provide the direction for approach. As noted, the National Science choosing these metrics, so that assumptions Foundation recently adopted the Customer and hypotheses can be tested empirically Development approach to the wherever possible. commercialization of NSF funded innovation through its i-Corps program As noted, educators are increasingly relying (NSF, 2012), and many leading universities on lean startup and customer development have restructured their innovation and activities for college and even high school entrepreneurship education curriculum level entrepreneurship courses. This around this methodology ( c.f. Stanford, provides a unique opportunity to educate Berkeley, MIT). Larger companies are their students about entrepreneurial biases, starting to explore variations of this process and to prepare them with the tools and for “intrapreneurship” activities as well techniques to minimize biases in both (Karlsson & Nordström, 2012), in industries information gathering and decision-making. as diverse as manufacturing (Blank, 2013) Based upon the findings of Fischhoff and healthcare (Silva & Nascimento, 2013.) (1982), this training can be effective and The process of customer development is will ideally translate to later entrepreneurial intended to not only accelerate the process activities. The same can apply to of innovation (the FFE) but also reduce the entrepreneurial support groups such as risk of error by continual iteration on the incubators and accelerators and, in this case, product-market fit before significant preparing their mentors and advisors to be investment. There is little in this process alert for these entrepreneurial biases can 34 Journal of Small Business Strategy Vol. 24, No. 2 encourage deeper System 2 approaches. REFERENCES Even using different companies in the incubator as both a referent class for Anderson, R. "Can stage-gate systems comparison as well as a friendly “check and deliver the goods?" Financial balance” can take best advantage of the Executive. (1993): 34. group setting. Bazerman, M. H. & Chugh, D. (2005). Several interesting research directions also Bounded awareness: Focusing failures emerge from this review. The prevalence of in negotiation. In L. Thompson (Ed.), the Business Model Canvas and Lean Frontiers of Social : Canvas methodologies could be studied to Negotiation. (pp.7-26) East Sussex, identify which subsets of questions in UK: Psychology Press. customer development are most relevant for product-market fit. Is there a specific order Bazerman, M. H., Loewenstein, G. F., & in which the customer development White, S. B. (1992). Reversals of hypothesis testing model increases the preference in allocation decisions: chances of success, and, if so, can Judging an alternative versus choosing innovators and their partners (such as among alternatives. Administrative venture capital firms and business Science Quarterly, 37(2), 220-240. accelerators) create specific programs to take advantage of this? For example, is it Bazerman, M.H., White, S.B., & more effective to talk to a group of potential Loewenstein, G.F. (1995). Perceptions customers in one field at a time, or to test of Fairness in Interpersonal and several customer segments simultaneously Individual Choice Situations. Current with regard to bias reduction? In addition, Directions in Psychological Science, 4, the use of simple linear models with 39-43. customer development and lean startup offers the framework to validate the Bhandari, G., & Deaves, R. (2006). The evaluation methods. Just as the Apgar score, demographics of overconfidence. The an improper linear model, was validated Journal of Behavioral Finance, 7 (1), 5- against subsequent data, testing the 11. improper linear models inherent in the customer development process against a set Blank, S.G.(2013). Why the Lean Start-Up of established outcome measures would Changes Everything. Harvard Business provide another means of reducing bias Review, 91(5), 64-+. from the process while still recognizing the intuitive side of the entrepreneurial process. Blank, S. G., & Dorf, B. (2012). The Future research could focus on the Startup Owner's Manual: The Step-By- development and testing of such improper Step Guide For Building a Great linear models around a tool often used in Company. K&S Ranch, Incorporated. the lean startup process such as the Business Model Canvas. Blank, S.G. (2007). Four Steps to the Epiphany: Successful Strategies for Products that Win, Palo Alto, CA: Cafepress.

35 Journal of Small Business Strategy Vol. 24, No. 2 Brentani, U., & Reid, S. E. (2012). The Cooper, R.G. 2013 New Products: What Fuzzy Front End of Discontinuous Separates the Winners From the Losers Innovation: Insights for Research and and What Drives Success? In K. Kahn, Management. Journal of Product (Ed.), PDMA Handbook of New Innovation Management, 29(1), 70-87. Product Development 3rd Edition (pp.3-34), Somerset, NJ: Wiley. Busenitz, L. W., & Barney, J. B. (1997). Differences between entrepreneurs and Cooper, R. G. 2013 (a). Invited Article: managers in large organizations: Biases Where Are All the Breakthrough New and heuristics in strategic decision- Products?: Using Portfolio making. Journal of Business Venturing, Management to Boost Innovation. 12(1), 9-30. Research-Technology Management, 56(5), 25-33. Carter, B. S., McNabb, F., & Merenstein, G. B. (1998). Prospective validation of a Cooper, R. G. (2014). Invited Article: What’s Next?: After Stage-Gate. scoring system for predicting neonatal Research-Technology Management, morbidity after acute perinatal 57(1), 20-31. asphyxia. The Journal of Pediatrics, 132(4), 619-623. Croll, A., & Yoskovitz, B. (2013). Lean Analytics: Use Data to Build a Better Cooper, A. C., Woo, C. Y., & Dunkelberg, Startup Faster. Sebastopol, CA: W. C. (1988). Entrepreneurs' perceived O'Reilly. chances for success. Journal of Business Venturing, 3(2), 97-108. Dawes, R. M. (1971). A case study of graduate admissions: Application of Cooper, B., & Vlaskovits, P. (2013). The three principles of human decision Lean Entrepreneur: How Visionaries making. American Psychologist, 26(2), Create Products, Innovate with New 180-188. Ventures, and Disrupt Markets. Hoboken, NJ: John Wiley & Sons. Dawes, R. M., & Corrigan, B. (1974). Linear models in decision making. Cooper, R. G. 1988. “The New Product Psychological Bulletin, 81(2), 95. Process: A Decision Guide for Management,” Journal of Marketing DelVecchio, J., White, F., & Phelan, S., Management, 3(3): 238-255. Tools for Innovation Management: A Comparison of Lean Startup and The Cooper, R. G. 2001. Winning at New Stage Gate System, USASBE Products: Accelerating the Process Proceedings, 2014, In Press. from Idea to Launch, 3rd edition, Cambridge, MA: Perseus. Edgett, S. J. (2011). New Product Development: Process Benchmarks and Cooper, R.G. 2011. Winning at New Performance Metrics. Ancaster, Products: Creating Value Through Ontario, Canada: Stage-Gate Innovation. 4th Edition, New York: International. Basic Books.

36 Journal of Small Business Strategy Vol. 24, No. 2 Fischhoff, B. (1982). . In D. Entrepreneurship Theory and Practice, Kahneman, P. Slovic, & A. Tversky 33(1), 167-192. (Eds.), Judgment Under Uncertainty: Heuristics and Biases. Cambridge: Idson, L. C., Chugh, D., Bereby-Meyer, Y., Cambridge University Press: 422 – Moran, S., Grosskopf, B., & Bazerman, 444. M. (2004) Overcoming focusing failures in competitive environments. Fischhoff, B., & Beyth-Marom, R. (1983). Journal of Behavioral Decision Hypothesis evaluation from a Bayesian Making, 17(3), 159-172. perspective. Psychological review, 90(3), 239. Kahneman, D. (2011). Thinking, Fast and Slow. New York: Macmillan. Flyvbjerg, B. (2009). Survival of the unfittest: why the worst infrastructure Kahneman, D., & Lovallo, D. (1993). Timid gets built—and what we can do about choices and bold forecasts: A cognitive it. Oxford Review of Economic Policy, perspective on risk and risk taking. 25(3), 344-367. Management Science, 39, 17-31.

Gigerenzer, G., Hoffrage, U., & Karlsson, J., & Nordström, J. (2012). Kleinbölting, H. (1991). Probabilistic Implementing Lean Startup mental models: a Brunswikian theory Methodology in Large Firms-Benefits, of confidence. Psychological review, Challenges and Solutions. 98(4), 506. (Unpublished Thesis). Chalmers University of Technology, Gothenburg, Harper, D. A. (1999). How Entrepreneurs Sweden. Learn: A Popperian Apprach and Its Limitations. Handelshøjskolen i Karniouchina, E.V. 2011. “Are Virtual København. Markets Efficient Predictors of New Product Success? The Case of the Hastie, R., & Dawes, R. M. (Eds.). (2010). Hollywood Stock Exchange, “ Journal Rational Choice In An Uncertain of Product Innovation Management, World: The Psychology Of Judgment 28(4): 470–484. And Decision Making. Sage. Klein, G. (2007). Performing a Project Henard, D. H. and D. M. Szymanski. 2001. Premortem. Harvard Business Review, “Why Some New Products Are More 85(9), 18-19. Successful Than Others, Journal of Marketing Research ,” 38( 3) August: Koellinger, P., Minniti, M., & Schade, C. 362-375. (2007). “I think I can, I think I can”: Overconfidence and entrepreneurial Holcomb, T. R., Ireland, R. D., Holmes Jr, behavior. Journal of Economic R. M., & Hitt, M. A. (2009). Psychology,28(4), 502-527. Architecture of entrepreneurial learning: exploring the link among Koen, P., Ajamian, G., Burkart, R., Clamen, heuristics, knowledge, and action. A., Davidson, J., D'Amore, R., ... & Wagner, K. (2001). Providing clarity 37 Journal of Small Business Strategy Vol. 24, No. 2 and a common language to the" fuzzy subsequent death and cerebral palsy: a front end". Research-Technology population-based study in term infants. Management, 44(2), 46-55. The Journal of pediatrics, 138(6), 798- 803. Koen, P.A. G. Ajamian, S. Boyce, A.Clamen, E. Fisher, S. Fountoulakis, Mullins, J.W. (2013). The New Business A. Johnson P. Puri, R. Seibert. 2002. Road Test: What Entrepreneurs and “Fuzzy-Front End: Effective Methods, Executives Should Do Before Tools and Techniques,” In: P. Launching a Lean Start-Up. London: Belliveau, A Griffen and S. Sorermeyer Pearson Financial Times. (Eds.) PDMA Toolbook for New Product Development, (pp. 2-35), New Mussweiler, T., Strack, F., & Pfeiffer, T. York: Wiley. (2000). Overcoming the inevitable anchoring effect: Considering the Koen, P.A. 2004. "Fuzzy Front End for opposite compensates for selective Incremental, Platform and accessibility. Personality and Social Breakthrough Products and Services", Psychology Bulletin, 26, 1142- 1150. In K. Kahn, G. Castellion and A Griffen (Eds.) PDMA Handbook (pp. Osterwalder, A., & Pigneur, Y. (2010). 81-91) New York: Wiley. Business model generation: a handbook for visionaries, game changers, and Larrick, R. P. (2004). Debiasing. In D. J. challengers. Hoboken, NJ: Wiley. Koehler & N. Harvey (Eds.), Blackwell Handbook of Judgment and Decision Ries, E. 2011. The Lean Startup, New Making. Oxford, England: Blackwell York: Crown Publishing Group. Publishers. Schade, C., & Koellinger, P. (2007). Lindley. D. V. (1985), Making Decisions, Heuristics, biases, and the behavior of New York, Wiley. Lerner, J. S., & entrepreneurs. In M. Minniti (Ed.) Tetlock, P. E. (1999). Accounting for Entrepreneurship, the Engine of the effects of accountability. Growth, Vol. 1, (pp. 141-163) Psychological Bulletin, 125(2), 255- Westport, CT: Praeger. 275. Silva, M. B., & Nascimento, M. A. Lean Maurya, A. (2012). Running Lean: Iterate Startup Applied In Healthcare: A from Plan A to a Plan That Works. Viable Methodology for Continuous O'Reilly Media. Improvement in the Development of New Products and Services. Retrieved Minniti, Zacharakis, Spinelli, Rice, & from Habbershon, Entrepreneurship: The http://www.cti.gov.br/mcpl2013/papers Engine of Growth, Greenwood _MMCP13/0054/0054_FI.pdf. Publishing Group, 2007. Slater, S. F., Mohr, J. J., & Sengupta, S. Moster, D., Lie, R. T., Irgens, L. M., (2013). Radical Product Innovation Bjerkedal, T., & Markestad, T. (2001). Capability: Literature Review, The association of Apgar score with Synthesis, and Illustrative Research 38 Journal of Small Business Strategy Vol. 24, No. 2 Propositions. Journal of Product Jonathan L. York, Ph.D. is an associate Innovation Management, 31, (3). professor of entrepreneurship in the Orfalea College of Business at Cal Poly. Dr. York is Stanovich, K. E., & West, R. F. (2000). also the Founding Faculty Director of the Individual differences in reasoning: Cal Poly Center for Innovation & Implications for the rationality debate. Entrepreneurship, and is a former serial Behavioral & Brain Sciences, 23, 645- entrepreneur whose research interests are 665. now centered in entrepreneurial search processes and entrepreneurial cognition. He Stanovich, Keith (2010), Rationality and has a Bachelor's Degree from Yale the Reflective Mind, Oxford University and a Ph.D. from Michigan University Press, USA. State.

Jeffrey E. Danes, Ph.D. Michigan State Thaler, R. H., & Sunstein, C. R. (2008). University, is professor of Marketing Nudge. New Haven: Yale University Analytics at Cal Poly in the Orfalea College Press. of Business and works at the intersection of psychology, probability, and behavioral Thompson, L., Gentner, D., & Loewenstein, economics. His expertise is judgment under J. (2000). Avoiding Missed uncertainty. He seeks to solve problems in Opportunities in Managerial Life: product pricing, branding, new product Analogical Training More Powerful development, and data-driven decision Than Individual Case Training. making. Organizational Behavior & Human Decision Processes, 82(1), 60-75.

Tversky, A., & Kahneman, D. (1974). Judgment under uncertainty: Heuristics and biases. Science, 185(4157), 1124- 1131.

39 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.