Organization, Self-Organization, Autonomy and Emergence: Status and Challenges
Total Page:16
File Type:pdf, Size:1020Kb
Organization, Self-Organization, Autonomy and Emergence: Status and Challenges Sven Brueckner 1 Hans Czap 2 1 New Vectors LLC 3520 Green Court, Suite 250, Ann Arbor, MI 48105-1579, USA Email: [email protected] http://www.altarum.net/~sbrueckner 2 University of Trier, FB IV Business Information Systems I D-54286 Trier, Germany Email: [email protected] http://www.wi.uni-trier.de/ Abstract: Development of IT-systems in application react and adapt autonomously to changing requirements. domains is facing an ever-growing complexity resulting from Therefore, approaches that rely on the fundamental principles a continuous increase in dynamics of processes, applications- of self-organization and autonomy are growing in acceptance. and run-time environments and scaling. The impact of this Following such approaches, software system functionality trend is amplified by the lack of central control structures. As is no longer explicitly designed into its component processes a consequence, controlling this complexity and dynamics is but emerges from lower-level interactions that are one of the most challenging requirements of today’s system purposefully unaware of the system-wide behavior. engineers. Furthermore, organization, the meaningful progression of The lack of a central control instance immediately raises local sensing, processing and action, is achieved by the the need for software systems which can react autonomously system components themselves at runtime and in response to to changing environmental requirements and conditions. the current state of the environment and the problem that is to Therefore, a new paradigm is necessary how to build be solved, rather than being enforced from the “outside” software systems changing radically the way one is used to through design or external control. think about systems and its development: self-organization, While such self-organizing and emergent software systems autonomy and emergence are the concepts that have to be are very attractive for their inherent flexibility, robustness build into future systems. and graceful degradation under component failure – to name This paper shows first steps in order to arrive at a theory of just a few commonly cited attributes – the design, self-organization, autonomy and emergence and provides implementation, validation and evaluation of such systems some of the fundamental principles that should be followed remain major challenges for researchers and system for the design of truly adaptive software systems. developers. Typical questions and issues revolving around Keywords: systems theory, organization, self-organization, such approaches are: emergence, autonomy. What principles govern the self-organization of autonomous components and the emergence of desired 1. Introduction system-level features? How to break down the system into interacting This section shows the basic principles of self-organization in components? natural and artificial systems. The components identified will How to design inter-component interactions and be explained in the following chapters. The aim is to show, component-internal reasoning to achieve the desired how organizations are able to change their inherent structures system-level functions? respectively how to apply structural learning as a self- How to define a static and dynamic environment that organizing process. supports and nourishes the goal-driven operation of The ever-growing complexity of today’s IT-supported the system components? processes in business, government, military or entertainment, How to design methods for human operators to seriously challenges traditional approaches to IT-application interact and guide such systems? design, implementation, deployment, and management. How to define interfaces for external IT systems to Today’s software systems are increasingly decentralized, interact with such a system for mutual benefit? distributed, and dynamic, just as the problem domains in How to implement robust decentralized and which they are operating are taking on these characteristics distributed systems? too. How to understand their behavior under a wide range To deal with this growing software and domain of operational parameters? complexity and the increasing dynamics of application- and How to formally or even empirically validate and run-time-environments, software systems must be able to evaluate the emergence of the desired system-level functions? run-time strongly relate to definition (ii). Our focus, How to deploy, maintain, and update the components therefore, lies on the concepts (ii) and (iii) of an organization. of such a complex system? In management-theory organizational settings are How to certify correct functioning also in malicious characterized more specific by the five ‘structural environments dimensions’ respectively ‘structural variables’ [9]. These In the following, we discuss various such issues in more dimensions immediately relate to architectural design- detail to provide context to the various contributions of this problems during conceptual phase of systems development: special journal edition. Specialization. Depending on the kind of problems to be solved by the system, specific capabilities of processing units 2. Theory of Organization, Self-Organization, must be assigned. In general, having specific purpose Autonomy and Emergence processing units will result in high performance but low reliability. If a high degree of reliability and flexibility is One of the major challenges of the new approach to intended the components of the system should show engineering complex software system by harnessing overlapping capabilities. principles such as self-organization or emergence is the lack Coordination. Different coordination measures relevant for of a formal theory that can underpin such systems, whether the systems design consist of ‘coordination by algorithms’, engineered or “naturally” grown. The interdisciplinary ‘coordination by direct communication’ and ‘self- science of Complex Adaptive Systems (CAS) offers some coordination’. ‘Coordination by algorithms’ is very efficient useful concepts and ideas that are drawn from a wide variety in static environments, where the problems to be solved are of fields, such as physics, chemistry, biology, economics, or well-known in advance. In distributed processing one social science. But, in general, our ability to formally model, generally prefers ‘direct communication’ or ‘self- analyze and evaluate such systems in their structure or coordination’. dynamics is still very limited. Before presenting a formal In the case of ‘direct communication’ the client, who definition, a more phenomenal understanding of the concept requests a problem solution, must know the specific of an organization and its properties is essential. capabilities of the servers in question. Thus, it’s a problem of information availability at run-time. The well known 2.1 Organizations and Organizational Dimensions contract-net-protocol is an example of ‘coordination by direct communication’. The concept ‘organization’ has different meanings. Mainly, In the case of ‘self-coordination’ a group of processing four versions are distinguished: units, respectively a subsystem is confronted with the problem to be solved. The processing units decide by their (i) An organization A is a set of elements, A = {a1, .., an}. This definition is in correspondence with the formal own, which part of the problem will be solved by which unit. definition of a system, but is very unspecific. As it is The blackboard –paradigm, where a task is announced to the with universal definitions, they do not help much in public and processing units declare readiness to solve the differentiating an organization from any other system. problem, is one of the most popular forms of self- Therefore, in the following this version is disregarded. coordination. As is known of blackboard-architecture (ii) An organization can be understood as the process of communication likely becomes a bottleneck and limits organizing. scalability. This is generally true for self-coordination (iii) An organization can be seen as the instrument or mean to approaches, if coordination uses a central device. As is shown achieve specific goals. In this context one talks of the in the following, stigmergy provides means for self- instrumental character of an organization. The concepts coordination by changing environmental variables, thus (ii) and (iii) are essential when talking about self- avoiding bottlenecks of hierarchical or other centralized organization, since, as is later shown, self-organization is approaches. understood as the process of adapting an organization by Configuration. Configuration relates to the organizational internal forces to a specific requirement or goal. structure, i.e. the division of labor according to the tasks, (iv) Finally, an organization can be identified with the result responsibilities, obligations and accountability of members of of organizing, i.e. an organization is seen as a social the organization. Design decisions address the number of institution. hierarchy levels, the span of control, definition of roles etc. Organizations and Distributed Processing have many The dimensional variables coordination and configuration common characteristics. The understanding of the are not independent each other. Typically, by the definition of fundamental properties of organizations helps in roles and correspondent responsibilities the need for understanding basic