An IntemaUonsl Journal computers l mathematics Mth q~k.~km, PERGAMON Computers and Mathematics with Applications 37 (1999) 23-34 Distributed Dynamic Programming Using Concurrent Object-Orientedness with Actors Visualized by High-Level Petri Nets B. MIKOLAJCZAK Computer and Information Science Department University of Massachusetts Dartmouth North Dartmouth, MA 02747, U.S.A. bolek~cis, umassd, edu J. T. RUMBUT, JR. Naval Undersea Warfare Center, Architecture and Computer System Division Newport, RI 02841, U.S.A. rumbut©ada, npt. nuwc. navy. mil Abstract--This paper deals with application of concurrent object-oriented programming with Actors to solve dynamic programming problems in a distributed computing environment. This area of research is often called distributed artificial intelligence. Using a dynamic programming example of chained matrix multiplication, a method of managing dynamic programming searches in a distributed programming environment with Actors is presented. Distributed computations with Actors are vi- sualized by means of Time-Varying Automata (for cases with no intra-actor concurrency) or using a class of high-level nets called Hierarchical Colored Petri Nets (for cases with intra~actor concurrency). Design and implementation features of the specific Actor-based programming environment, using a concurrent extension of C++, are also discussed. (~ 1999 Elsevier Science Ltd. All rights reserved. geywords--Distributed dynamic programming, Distributed artificial intelligence, Concurrent objeet-orientedness, Actors, High-level Petri nets, Visualization. 1. INTRODUCTION In [1] Andreoli et al. have shown, using the problem of finding the shortest path in a weighted acyclic network as an example, that the search technique of dynamic programming can be im- plemented in a form of multiagent computation by the interaction of cooperating and competing agents. We extend their work by applying the following methodologies to dynamic programming optimization problems. The first author during this research has been supported by the grant IPA #N66604-4160-1411 from Naval Undersea Warfare Center, Newport, RI. A preliminary version of this paper has been presented during the International Workshop on Intelligent Systems and Innovative Computations--The 6 th Bellman Continuum, August 1-2, 1994, Hachioji, Tokyo, Japan. This paper is in final form and no version of it will be submitted for publication elsewhere. Both authors appreciate continuous guidance and encouragement from our direct supervisors: B. Navin from Architecture and Computer System Division and B. Green, chairman of the Computer and Information Science Department at the University of Massachusetts Dartmouth. Authors would like to express their thanks to graduate student C. Mello for his contribution to implementation of the original version of distributed system with Actors during his Master Project in 1993--1994. 0898-1221/99/$ - see front matter. © 1999 Elsevier Science Ltd. All rights reserved. Typeset by ,4¢~4S-TEX PII: S0898-1221 (99)00139-X 24 B. MIKOLAJCZAK AND J. T. RUMBUT, JR. (i) Concurrent Object-Oriented Programming (COOP) paradigm with Actors based on the original idea by Hewitt [2] and as presented in detail in Agha's book [3]; this approach attempts to reflect inherent concurrency of systems and distributed nature of processes and contributes to better operational and formal understanding of distributed computing systems. Actors compared with agents [1] may be inherently concurrent, may be equipped in intelligenceof arbitrary granularity, may have capabilities of self-modification, and self- procreation. (2) Time-Varying Automata [4] and Hierarchical Colored Petri Nets [5] to model structure and dynamics of Actor programs, extending Agha's application of predicate-transition nets and Sami and Vidal-Naquet's [6] application of Colored Petri Nets to visualization and simulation of important structural properties (deadlock, fairness, nondeterminism) of Actor programs. (3) Distributed Actor-based algorithm for the chained matrix multiplication problem as de- scribed in its sequential version in [7] as an illustrationof the method of solving dynamic programming optimization problems in distributed computing environment. (4) Describing COOP environment in C++ as designed and implemented in [8]. The approach presented in this paper is mostly conceptual and language-independent. This means that from an implementation point of view a substantial effort is still needed to make this methodology practically available, especially from performance (speedup and efficiency) and reliability points of views. We show that the search techniques of dynamic programming models are a very regular form of Actor computations which can be characterized by strict cooperation and competition of Actors in a dynamically changing distributed computing environment. 2. DISTRIBUTED DYNAMIC PROGRAMMING WITH ACTORS Open Distributed Systems There exist two categories of distributed systems: weakly and strongly distributed [9]. A system is weakly distributed if its modules know the names of other modules. It is strongly distributed if its modules cannot directly name other modules. In a strongly distributed system a given module knows only the names of its own communication ports. The names of modules with which it communicates are stored as data in its communication ports, and may be viewed as pointers to ports of other modules that cannot be dereferenced to determine the name of nonlocal ports. Therefore a module communicates by sending a message, containing data, to its local port which takes over a task of sending it to its destination. Object-oriented languages and also Actor-based languages are weakly distributed systems, i.e., Actors know names of some other Actors. One of the main properties of an open distributed system is its capability to dynamically modify configuration of software or hardware components without destroying the functionality of the system. These systems should consist of many computational entities (fine-grain parallelism) which will cooperate/compete with one another in solving complex problems. Entities of such systems should be able to communicate with entities outside of the system, i.e., some previously unknown entity may be able to communicate with other entities in the system. Distributed systems are distinguished by the presence of three essential features of parallel systems: causality, conflict, and concurrency. One can understand such complex systems by imposing structure on them and by equipping them with a method for abstracting from details. Conceptually this means also to be able to decide how far one has to go, in terms of space, time, and logical proximity, to identify all the entities of the system interacting with the entity of interest. The primary goal of considering such systems is expected improvement of performance; the second goal is practical connectivity of existing heterogeneous computing resources to solve challenging computing problems, and the third is modularity of design and related extendability of systems. Distributed Dynamic Programming 25 Distributed Systems with Actors A model of computation, based on the notion of Actors, can be used as a foundation of dis- tributed open systems. Actors are "active" objects which execute simultaneously and communi- cate asynchronously with each other via message passing. Actors are independent, self-contained, interactive components of a computing system. Actors may provide inherent concurrency of an object, encapsulation of local intelligence, and allow an object state to be directly expressed and manipulated. Therefore Actor-based languages combine, to a certain extent, procedural, func- tional, and object-oriented programming paradigms. This allows for programs which are easily understandable and which avoid unnecessary sequencing. Each Actor has a conceptual location, its mail address, and a behavior. An Actor's acquain- tances are all of the Actors whose mail addresses it knows. Actors communicate through messages. Preservation of message order is not guaranteed because of potential heterogeneity of computers involved (speed of processing) and because of adaptive routing strategies applied. However, mes- sages sent are guaranteed to be received with an unbounded but finite delay. This requirement satisfies the weak fairness property of the Actor system. State change in Actors is specified using replacement behaviors. Each time an Actor processes a message, it also computes its behavior in response to the next message it may process. The behavior of each individual Actor is character- ized by a set of well-defined, observable states and a set of transitions over these states. An Actor has three options to express its behavior: to process the next message and at the same time to preserve its current behavior, to change behavior to a new one, and to create a new Actor. The change in Actor's behavior may represent a single change of state variables (member fields) or it may represent changes in the operations (member functions) which are carried out in response to messages. At any instance of time every Actor can be only in one of the two "major" states: the state of message processing or state of accepting new message. Only when being in state of message processing can an Actor change its behavior. When being in the state of accepting a new message, an Actor is sensitive only to incoming external messages storing them in its mailbox. A program in Actor-based programming languages