Serendipity and strategy in rapid innovation T. M. A. Fink∗†, M. Reevesz, R. Palmaz and R. S. Farry yLondon Institute for Mathematical Sciences, Mayfair, London W1K 2XF, UK ∗Centre National de la Recherche Scientifique, Paris, France zBCG Henderson Institute, The Boston Consulting Group, New York, USA Innovation is to organizations what evolution is to organisms: it while, Bob chooses pieces such as axels, wheels, and small base is how organisations adapt to changes in the environment and plates that he noticed are common in more complex models, improve [1]. Governments, institutions and firms that innovate even though he is not able to use them straightaway to produce are more likely to prosper and stand the test of time; those new toys. We call this a far-sighted strategy. that fail to do so fall behind their competitors and succumb Who wins. At the end of the day, who will have innovated to market and environmental change [2, 3]. Yet despite steady the most? That is, who will have built the most new toys? We advances in our understanding of evolution, what drives inno- find that, in the beginning, Alice will lead the way, surging vation remains elusive [1, 4]. On the one hand, organizations ahead with her impatient strategy. But as the game progresses, invest heavily in systematic strategies to drive innovation [5{ fate will appear to shift. Bob's early moves will begin to look 8]. On the other, historical analysis and individual experience serendipitous when he is able to assemble a complex fire truck suggest that serendipity plays a significant role in the discovery from his choice of initially useless axels and wheels. It will seem process [9{11]. To unify these two perspectives, we analyzed that he was lucky, but we will soon see that he effectively cre- the mathematics of innovation as a search process for viable ated his own serendipity. What about you? Picking components designs across a universe of building blocks. We then tested on a hunch, you will have built the fewest toys. Your friends had our insights using historical data from language, gastronomy an information-enabled strategy, while you relied on chance. and technology. By measuring the number of makeable designs Spectrum of strategies. What can we learn from this? If in- as we acquire more components, we observed that the relative novation is a search process, then your component choices to- usefulness of different components is not fixed, but cross each day matter greatly in terms of the options they will open up other over time. When these crossovers are unanticipated, they to you tomorrow. Do you pick components that quickly form appear to be the result of serendipity. But when we can predict simple products and give you a return now, or do you choose crossovers ahead of time, they offer an opportunity to strate- those components that give you a higher future option value? gically increase the growth of our product space. Thus we find By understanding innovation as a search for designs across a that the serendipitous and strategic visions of innovation can universe of components, we made a surprising discovery. Infor- be viewed as different manifestations of the same thing: the mation about the unfolding process of innovation can be used changing importance of component building blocks over time. to form an advantageous innovation strategy. But there is no Lego game. Let's illustrate the idea using Lego bricks. Think one superior strategy. As we shall see, the optimal strategy de- back to your childhood days. You're in a room with two friends pends on time|how far along the innovation process we have Bob and Alice, playing with a big box of Lego bricks|say, a advanced|and the sector|some sectors contain more oppor- fire station set. All three of you have the same goal: to build as tunities for strategic advantage than others. many new toys as possible. As you continue to play, each of you Components and products. Just like the Lego toys are made searches through the box and chooses those bricks that you be- up of distinct kinds of bricks, we take products to be made up lieve will help you reach this goal. Let's now suppose each player of distinct components. A component can be an object, like a approaches this differently. Your approach is to follow your gut, touch screen, but it can also be a skill, like using Python, or a arbitrarily selecting bricks that look intriguing. Alice uses what routine, like customer registration. Only certain combinations we call a short-sighted strategy, carefully picking Lego men and of components form products, according to some predetermined their firefighting hats to immediately make simple toys. Mean- universal recipe book of products. Examples of products and 104 R cayenne ) Rails ) ) cocoa 104 jQuery UI Language F Gastronomy 100 Technology 1000 lime 50 X Sauce Labs usefulness 1000 arXiv:1608.01900v4 [physics.soc-ph] 17 Mar 2017 usefulness ( usefulness ( ( 100 10 100 5 10 10 1 0.5 Makeable words Makeable recipes 1 1 Makeable software 0 2 4 6 8 10 12 14 16 18 20 22 24 26 0 127 254 381 0 331 662 993 Acquired letters Acquired ingredients Acquired development tools 1st R 1st cayenne 1st Rails 2nd F 2nd cocoa 2nd jQuery UI Rank 3rd X Rank 3rd lime Rank 3rd Sauce Labs FIG. 1: Products, components and usefulness. (Top) We studied products and components from three sectors. In language, the products are 79,258 English words and the components are the 26 letters. In gastronomy, the products are 56,498 recipes from the databases allrecipes.com, epicurious.com, and menupan.com [12] and the components are 381 ingredients. In technology, the products are 1158 software products catalogued by stackshare.io and the components are 993 development tools used to make them. (Bottom) The usefulness of a component is the number of products we can make that contain it. We find that the relative usefulness of a component depends on how many other components have already been acquired. For each sector, we show the usefulness of three typical components: averaged at each stage over all possible choices of the other acquired components and|for gastronomy|for a particular random order of component acquisition (points). 2 the components used to make them are shown in Fig. 1. Now it cannot decrease. We write uα(n) to indicate this dependence suppose that we possess a basket of distinct components, which on n: uα(n) is the usefulness of α given possession of α and we can combine in different ways to make products. We have n − 1 other components, the combined set of components being more than enough copies of each component for our needs, so n. Averaging over all choices of the n−1 other components from we do not have to worry about running out. There are N possi- the N − 1 that are possible gives the mean usefulness, uα(n). ble component types in total, but at any given stage n we only Usefulness experiment. To measure the usefulness of different have n of these N possible building blocks. At every stage, we components as the innovation process unfolds and we acquire pick a new type of component to add to our basket. more components, we did the following experiments. Using data Usefulness. The usefulness of a component is the number of from each of our three sectors, we put a given component α into products we can make that contain it [13]. In other words, the an empty basket, and then added, one component at a time, usefulness uα of some component α is how many more products the remaining N − 1 other components, measuring the useful- we can make with α in our basket than without α in our basket. ness of α at every step. We averaged uα(n) over all possible As we gather more components, uα increases or stays the same; orders in which to add the N − 1 components to obtain uα(n). (We explain how in SI B.) We repeated this process for all of E A the components α. Typical results from these experiments are I R shown in Fig. 1. We find that the mean usefulnesses of different N components cross each other as the number of components in T 6 O S SMALL KITCHEN BIG KITCHEN L C 127 ingredients: almond to fenugreek 381 ingredients: almond to zucchini U Recipe D complexity M 13 21 Language P H 597 recipes in total 56,498 recipes in total G 17 Y B 13 F V 20 K 9 W Z 5 X J Q A 1 B 26 600 Recipes 0 0 Recipes 6000 egg wheat butter onion 21 garlic milk 89 recipes contain cocoa 4801 recipes contain cocoa vegetable_oil cream 17 tomato olive_oil 95 black_pepper pepper 13 vanilla cayenne vinegar 9 cane_molasses bell_pepper cinnamon parsley 5 chicken Gastronomy 190 lemon_juice beef cocoa C 1 D corn bread scallion 100 Recipes with cocoa 0 0 Recipes with cocoa 1000 mustard ginger basil celery Cocoa is more useful than cayenne Cayenne is more useful than cocoa 286 carrot potato chicken_broth 21 yeast rice mushroom 43 recipes contain cayenne 7950 recipes cheese 17 soy_sauce contain cayenne cumin oregano 381 13 Google Analytics GitHub 9 jQuery nginx Bootstrap Slack 5 JavaScript New Relic Redis Google Apps E 1 F 248 Amazon S3 Amazon EC2 Git 100 Recipes with cayenne 0 0 Recipes with cayenne 1000 AngularJS Node.js MySQL Amazon CloudFront FIG. 3: Why crossovers happen. On the right is a big kitchen with Trello Rails 381 ingredients.