1 The Problem of Emergence by John F. Padgett and Walter W. Powell Introductory chapter to The Emergence of Organizations and Markets Princeton University Press forthcoming, Spring 2012 December 23, 2009 revised: January 12, 2010 revised: August 1, 2011 revised: November 15, 2011 2 Chapter 1 The Problem of Emergence John F. Padgett and Walter W. Powell Organizational Novelty Darwin’s question about the origin of species is worth posing and exploring as much in the social sciences as it was in biology. Human organizations, like living organisms, have evolved throughout history, with new organizational forms emerging and transforming in various settings: new types of banks and banking in the history of capitalism; new types of research organizations and research in the history of science; new types of political organizations and nations in the history of state formation. All of these examples are discussed in this book. The histories of economies and polities are littered with new organizational forms that never existed before. In biological language, this emergence of new organizational forms is the puzzle of speciation. We economists, political scientists, and sociologists have many theories about how to choose alternatives, once these swim into our field of vision. But our theories have little to say about the invention of new alternatives in the first place. New ideas, new practices, new organizational forms, new people must enter from off the stage of our imaginary before our analyses can begin. Darwin asked the fundamental question, but our concepts are like those of Darwin before Mendel and Watson and Crick. We understand selection and equilibrium, but we do not understand the emergence of what we choose or of who we are. Our analytical shears are sharp, but the life forces pushing things up to be trimmed elude us. 3 Novelty almost by definition is hard to understand. Something is not genuinely new if it already exists in our current practice or imagination. The terms innovation and invention, as we use them in this book, mean the construction of something neither present nor anticipated by anyone in the population. We do not mean that planned incremental improvement on what already exists is not possible—quite the opposite. This type of learning occurs far more often than does the production of genuine novelty. The conundrum for both researchers and participants is that logical cognition, no matter how useful for refinement and improvement, is unlikely to be a fundamental process for generating novelty, because logic can only use axioms that are already there. The literature on “organizational innovation” is voluminous, but that literature largely focuses on learning,1 search,2 and diffusion,3 and often uses patents as indicators.4 The term innovation in organization theory refers to products and ideas, never to the emergence of organizational actors per se.5 Social science studies processes of innovation, so defined, but mainly by abstracting from the content of innovation itself.6 Lest this limitation be mistaken for criticism, it is important to remember that Darwin himself never truly answered his own question about the origin of species. He “only” analyzed the natural selection of populations of organisms within species, once species existed. Some parts of social science with an evolutionary sympathy 1 See, for example, March (1991), Zander and Kogut (1995), Szulanski (1996), Argote (1999). 2 See, for example, Levinthal and March (1981), Hansen (1999), Rivkin and Siggelkow (2003). 3 See, for example, Strang and Meyer (1993), Guillén (1994), Davis and Greve (1997), Simmons, Dobbin and Garrett (2006). 4 See for example, Mansfield (1986), Griliches (1990), Jaffe, Trajtenberg and Henderson (1993), Owen-Smith and Powell (2004), Fleming and Sorenson (2004). 5 Numerous scholars have lamented that the origins of institutions have been largely opaque to social scientists. Kreps (1990: 530) has remarked that whereas the economics literature emphasizes the effects of institutions, it “leaves open the question, where did institutions come from?” In an assessment of the sociological literature, Barley and Tolbert (1997) underscored the neglect of how institutional arrangements are created. More recently, in a comprehensive review of organizations research, Greenwood et al. (2008: 26) conclude that “institutional studies have not been overly concerned with how institutions arise.” 6 Fortunately there are some valuable exceptions: Hughes (1983), Bijker, Hughes and Pinch (1987), Latour (1988), Hutchins (1995), Galison (1997), Sewell (2005). 4 have absorbed Darwin,7 but natural (or artificial) selection alone does not solve his puzzle of speciation. Besides this introductory chapter and a coda, this book contains fourteen historical case studies of the emergence of organizations and markets, plus three modeling chapters that apply concepts from biochemistry to social evolution. The case studies are divided into three clusters: four case studies on the European co-evolution of early capitalism and state formation, four case studies on Communist economic reform and transition, and six chapters about technologically advanced capitalism and science. These case studies, discussed below, were selected because all of them contain instances of the historical emergence of organizational novelty. Some chapters also discuss failed emergence, as a control group. Not all of the chapters involve speciation in the radical sense of new to human history, but nearly half of them do. All involve speciation in the sense of organizational novelty in the context of the population under study. The three modeling chapters in part 1 extract the foundational concept of autocatalysis from the existing chemistry literature on the origins of life and then apply this concept, through agent-based computer models, first to the self-organization of economic production and second to the evolution of primitive language and communication. These simple, biochemically inspired models are in no way rich enough to capture the phenomena or the array of emergence mechanisms observed in the historical case studies.8 But they do provide an analytical framework for specifying with some precision the social science problem of emergence. In this volume, inductive histories and deductive models are viewed as complementary (not competitive) research strategies, both being dedicated to the discovery of social processes of organizational genesis and emergence. 7 For example, Simon (1969), Nelson and Winter (1982), Hannan and Freeman (1989). 8 Colyvas and Maroulis in chapter 16, however, explicitly develop autocatalyic models for their biotechnology case. 5 Organizational genesis does not mean virgin birth. All new organizational forms, no matter how radically new, are combinations and permutations of what was there before. Transformations are what make them novel. Evolution, therefore, is not teleological progress toward some ahistorical (and often egocentric) ideal. It is a thick and tangled bush of branchings, recombinations, transformations, and sequential path-dependent trajectories, just as Darwin said it was. Invention “in the wild” cannot be understood through abstracting away from concrete social context, because inventions are permutations of that context. Historical path dependency does not imply that there are no transformational principles at the base of endless open-ended generation. Scientific prediction in open-ended, creative systems such as life is not the specification of a fixed-point equilibrium. It is the description of processual mechanisms of genesis and selection in sufficient detail to be capable, in the rich interactive context of the study system, of specifying a limited number of possible histories. This is the biology, not the physics, view of science. A barrier, however, inhibits social science investigation into processes of organizational emergence or speciation. Most social science proceeds according to the logic of methodological individualism. That is, the analyst takes as given some constitutive features of the hypothesized individual or actor (typically preferences, beliefs, and resources), and then derives aggregate or behavioral conclusions from them. “Actors” are objects imbued with boundaries, purposes and choices, whose teleological behavior is explained thereby. Useful as this approach is for many purposes, it creates in our understanding a black hole of genesis. To assume axiomatically that real people are actors makes them logically impenetrable to the theories built upon them. No theory can derive its own axioms. The problem is not that the social science concept of actor is not useful. The problem is that the atomic conception of actor precludes investigation into the 6 construction and emergence of the real people and organizations that we refer to by that abstraction. The whole question of where novelty in actors comes from, so central to any theory of evolution, never arises in the first place. In this book, we take the following as our mantra: In the short run, actors create relations; in the long run, relations create actors. The difference between methodological individualism and social constructivism is not for us a matter of religion; it is a matter of time scale. In the short run, all objects—physical, biological, or social—appear fixed, atomic.9 But in the long run, all objects evolve, that is, emerge, transform, and disappear. To understand the genesis of objects, we argue, requires a relational and historical turn