1 Introduction The importance of game and simulation applications grows everyday, as does the need for animated agents Non-linear dynamical that operate autonomously in these environments. These agents must be able to exhibit certain low- and system approach to high-level behaviors and interact with the environ- behavior modeling ment. Various approaches for modeling behavior and movement decisions have been investigated, among them the pioneering work of Reynolds (Reynolds 1987) and that of others (Haumann and Parent 1988; Wilhelms 1990; Renault et al. 1990; Bates et al. Siome Goldenstein, 1992; Noser and Thalmann 1993; Reynolds 1993; Edward Large, Ko et al. 1994; Tu and Terzopoulos 1994; Noser et al. Dimitris Metaxas 1995). These include, kinematic, dynamic, learning, and artificial intelligence-based approaches. AI techniques for behavior generation, such as (Hau- VAST Lab., Center for Human Modeling and mann and Parent 1988; Lethebridge and Ware 1989), Simulation, Computer and Information Science Department, University of Pennsylvania, 200 South generally require complex inferencing mechanisms. 33rd Street, Philadelphia, PA, USA 19104-6389 This may require considerable computational re- e-mail: {siome,large,dnm}@graphics.cis.upenn.edu sources, raising the question of scalability of such systems, as the number of independent agents and behaviors grows, and each agent has a different set of goals and behavioral directives. In addition, agents We present a dynamic systems approach to must be able to interact with real-time moving ob- modeling and generating low-level behav- jects that might either contribute to or compromise iors for autonomous agents. Such behaviors the final goal. Other approaches to this problem em- include real-time target tracking and obsta- ploy learning, perception, and dynamic techniques cle avoidance in time-varying environments. (Ridsdale 1990; Tu and Terzopouluos 1994; Noser The novelty of the method lies on the inte- et al. 1995; Grzeszczuk and Terzopouluos 1995), gration of distinct non-linear dynamic sys- while computational geometry techniques have been tems to model the agent’s interaction with employed in rather restricted environments (Wilfong the environment. An angular velocity con- 1988). trol dynamic system guides the agent’s di- Dynamic system approaches to this problem have rection angle, while another dynamic system mostly been explored at the level of control the- selects the environmental input that will be ory (Cohen 1992; Liu et al. 1994; Kokkevis et al. used in the control system. The agent inter- 1995). This methodology restricts the type of be- acts with the environment through its knowl- haviors that can be simulated since it does not cope edge of the position of stationary and mov- well with dynamically changing environments and ing objects. In our system agents automati- its computational complexity prohibits its use in cally avoid stationary and moving obstacles real-time simulations. Others discuss the useful- to reach the desired target(s). This approach ness of different layers of control and behaviors to allows us to prove the stability conditions achieve better results (Kurlander and Ling 1995; that result in a principled methodology for Magnenat-Thalmann and Thalman 1995; Blumberg the computation of the system’s dynamic pa- and Galyean 1995; Perlin and Goldberg 1996) rameters. We present a variety of real-time In this paper, we investigate and develop an alterna- simulations that illustrate the power of our tive methodology that has its roots in behavior-based approach. robotics (e.g., (Braitenberg 1989; Brooks 1991)) and is based on a novel way of combining differential Key words: Digital agents – Game anima- equations exhibiting particular behaviors (Steinhage tions – Low-level behaviors – Motion plan- and Schöner 1997; Large et al. 1999). Using this ning – Dynamical systems type of approach, Schöner and colleagues (Schöner and Dose 1992; Schöner et al. 1996) have devel- The Visual Computer (1999) 15:349–364 c Springer-Verlag 1999 350 S. Goldenstein: Non-linear dynamical system approach to behavior modeling oped a dynamical system for robot path planning and porarily disregard a target if there is an unsurpassable control. In this system a set of behavioral variables, moving or stationary obstacle immediately between namely heading direction and velocity, define a state it and the target. Like a human, it will first focus on space in which the dynamics of robot behavior is avoiding the obstacle, and then refocus on the target. described. Path planning is governed by a nonlinear In this paper we also prove an important result on dynamical system that generates a time course of the properties of the parameters of the task constraint the behavioral variables. The system dynamics are system which determine the number of behaviors specified as a nonlinear vector field, while the task modeled. These properties are associated with the that the agent will execute depends upon the task stability2 of the task constraint system. We also pro- constraints. Task constraints, such as obstacle avoid- vide an inductive proof that extends the above result ance and target tracking, are modeled as component to any number of dimensions in the task constraint forces which define attractors and repellers for the system. This result allows the principled design of dynamical system. The individual constraint contri- complex systems with large number of behaviors. butions are weighted and then added together into Our system allows single and multiple target track- a single vector field, which determines the observed ing in the presence of multiple static and moving behavior. Next, a second dynamical system is used obstacles, and scales well with the number of behav- to compute these weights. This dynamical system, iors. Obstacles are processed in a local fashion based the task level system, operates at a faster time scale on their relative location to the agent and the tar- than the movement level (Large et al. 1999). Qualita- get. In our current implementation and without loss tively different behaviors are modeled as fixed points of generality the agents are memoryless and reactive of this dynamical system, and the environment deter- in nature. Depending on the situation (emergence of mines the values of the parameters. As the perceptual new obstacles and/or targets) their movement can be information changes, parameters change, causing bi- discontinuous. furcations in the task-level system. During the course This paper is organized as follows. In Sects. 2 and of the simulation, one fixed point loses stability and 3 we present the dynamic formulation of the sys- another becomes stable, modeling the decision to tem. Section 4 describes the task constraint system cease executing one behavior and to execute another responsible for the generation of the behaviors. We instead. devise an inductive proof for the restrictions on this Here, we adapt the above methodology to model and system’s parameters (stability analysis) that will re- simulate autonomous low-level dynamic behaviors sult in the desired behaviors. Finally, we present which can be used in applications ranging from vir- a series of real-time simulations involving complex tual environments to games. We use (see also our interactions between the agents, the environment and preliminary results, in (Goldenstein et al. 1998)) a set the targets. of time-varying differential equations that control the heading angle and forward speed of a given dig- ital autonomous agent. Based on a principled com- 2 Movement dynamics and low-level bination of these equations we create a whole set of relatively complex low-level behaviors such as ob- behavior stacle avoidance and target tracking. To avoid unsta- ble fixed points in the differential equations1 we add Our methodology consists of the combination of two a Gaussian noise term in each equation. Using this distinct dynamic systems to model the movement system, decisions are made online and do not require and behavior of each autonomous agent. The first any previous memory, training, or global planning. system controls the movement of the agent. The state The set of targets and obstacles can change during space of this system is two-dimensional, the first pa- the course of the simulation, since the agent is able rameter represents the heading direction, while the to make smart local decisions based on its current other specifies its velocity (In a three-dimensional global knowledge of the dynamic environment in space there would be two heading angles and one which it is situated. For example, an agent will tem- forward velocity). The second system controls the 1 In differential equation terminology a fixed point is a point 2 The stability is related to the local convergence properties of where the derivative of the vector field is zero the system around a fixed point S. Goldenstein: Non-linear dynamical system approach to behavior modeling 351 agent’s movement decision-making, i.e., its behav- from the sum of contributions from all obstacles. We ior. The state space of this system is the space of the then extend the formulations to account for multiple agent’s behaviors. The parameter values of the state groups. vector components determine which elements of the Using (1) an attractor is defined as environment (e.g., obstacles, targets) will be used in φ˙ = =− (φ − ψ) , the calculation of the agent’s movement and there- ftar a sin (2) fore behavior. ψ Each autonomous agent’s movement is described in where is the angle of the target’s location relative a polar coordinates. It consists of a heading direction φ to the agent’s location and is a constant parame- v ter. Figure 1 shows a plot of (2), assuming that the and a forward velocity . The heading angle is con- ψ = trolled by a one-dimensional non-linear dynamical target is in front of the agent ( 0). The negative slope around the fixed point (in this case, the origin) system, which consists of repellers placed in the sub- φ˙ tended angle of the obstacles, and attractors in the is what gives the attractor property, forcing to be subtended angles of targets (see Sect.