A Survey of Behavior Trees in Robotics and AI Matteo Iovinoa,b, Edvards Scukinsa,c, Jonathan Styruda,d, Petter Ögrena and Christian Smitha aDivision of Robotics, Perception and Learning, Royal Institute of Technology (KTH), Sweden bABB Future Labs, hosted at ABB Corporate Research, AI Lab, Sweden cSAAB Aeronautics, Sweden dABB Robotics, Sweden ARTICLEINFO Abstract Keywords: Behavior Trees (BTs) were invented as a tool to enable modular AI in computer games, Behavior Trees but have received an increasing amount of attention in the robotics community in the Robotics last decade. With rising demands on agent AI complexity, game programmers found Artificial Intelligence that the Finite State Machines (FSM) that they used scaled poorly and were difficult to extend, adapt and reuse. In BTs, the state transition logic is not dispersed across the individual states, but organized in a hierarchical tree structure, with the states as leaves. This has a significant effect on modularity, which in turn simplifies both synthesis and analysis by humans and algorithms alike. These advantages are needed not only in game AI design, but also in robotics, as is evident from the research being done. In this paper we present a comprehensive survey of the topic of BTs in Artificial Intelligence and Robotic applications. The existing literature is described and categorized based on methods, application areas and contributions, and the paper is concluded with a list of open research challenges. Dialogue ? games Platform FPS —> —> —> games games Wander Player is Has low Player is RTS Evade Find aid ? Game AI Attacking? health? visible? games Fire arrow at Swing sword Manipulation Taunt player player at player Mobile ground Domain Figure 2: An example of a BT for a First Person Shooter robots Robotic AI (FPS) game, adapted from [97]. UAVs Behavior Trees Other controller maps a state to an action. We illustrate the robots Reinforcement Learning idea with a simple example of a BT, see Figure2. The Design method actions are illustrated by grey boxes ("Evade", "Find Learning by aid", "Fire arrow at player", "Swing sword at player", demonstration Manual "Taunt player", and "Wander"), while the state of the Design Planning world is analyzed in the conditions, illustrated by white ovals ("Player is Attacking?", "Has low health?", and arXiv:2005.05842v2 [cs.RO] 13 May 2020 Figure 1: Overview of the topics covered in this survey. "Player is visible?"). We begin by listing a number of key ideas behind the design of BTs, while a detailed description of the execution is presented in Section 1.2. 1. Introduction In this paper we survey the area of Behavior Trees • There is explicit support for the idea of task hier- (BTs) in AI and robotics applications.1 A BT describes archies, where one task is made up of a number a policy, or controller, of an agent such as a robot of subtasks, which in turn have subtasks. In Fig- or a virtual game character. Formally, the policy or ure2, Fighting is done either with a bow or a sword, and fighting with a bow in turn could in- [email protected] (M. Iovino); clude grasping an arrow, placing it on the string, [email protected] (E. Scukins); [email protected] (J. Styrud); [email protected] (P. Ögren); [email protected] (C. Smith) pulling the string, and then releasing it. 1Note that there is a different concept with the same name, used to handle functional requirements of a system, see e.g. [43], • There is explicit support for Sequences, sets of which will not be addressed here. The term is also used in [70] tasks where each task is dependent on the suc- to denote a general hierarchical structure of behaviors, different cessful completion of the previous, such as first from the topic of this paper. Iovino et al.: Preprint submitted to Elsevier Page 1 of 22 A Survey of Behavior Trees in Robotics and AI grasping an arrow and then moving it. Thus, the ticked node. A node is executed if, and only if, it the next item in a Sequence is only started upon receives Ticks. The child immediately returns Running success of the previous one. to the parent, if its execution is under way, Success if it has achieved its goal, or Failure otherwise. • There is explicit support for Fallbacks, sets of In the classical formulation, there exist three main tasks where each task represents a different way categories of control flow nodes (Sequence, Fallback and of achieving the same goal, such as fighting with Parallel) and two categories of execution nodes (Action either a bow or a sword. Thus, the next item and Condition), see Table1. in a Fallback is only started upon Failure of the Sequences are used when some actions, or condi- previous one. tion checks, are meant to be carried out in sequence, • There is support for reactivity, where more im- and when the Success of one action is needed for the portant tasks interrupt less important ones, such execution of the next. The Sequence node routes the as disengaging from a fight and finding first aid Ticks to its children from the left until it finds a child when health is low. that returns either Failure or Running, then it returns Failure or Running accordingly to its own parent, see • Finally, modularity is improved by the fact that Algorithm1. It returns Success if and only if all its each task, on each level in the hierarchy, has the children return Success. Note that when a child re- same interface. It returns either success, failure, turns Running or Failure, the Sequence node does not or running, which hides a lot of implementation route the Ticks to the next child (if any). In the case details, while often being enough to decide what of Running, the child is allowed to control the robot, to do next. whereas in the case of Failure, a completely different action might be executed, or no action at all in the case All items above are different aspects of the rules where the entire BT returns Failure. The symbol of the for switching from one task to another. By encoding Sequence node is a box containing the label “→”. this task switching logic in the hierarchical structure of the non-leaves of a tree, having basic actions and Algorithm 1: Pseudocode of a Sequence node conditions at the leaves, a surprisingly modular and with N children flexible structure is created. 1 for i } 1 to N do Below we first give a brief description of the history 2 childStatus } Tick(child(i)) of BTs, and then a more detailed description of their 3 if childStatus = running then core workings. 4 return running 1.1. A Brief History of BTs 5 else if childStatus = failure then BTs were first conceived by programmers of com- 6 return failure puter games. Since important ideas were shared on 7 return success only partially documented blog posts and conference presentations, it is somewhat unclear who first proposed the key ideas, but important milestones were definitely Fallbacks2 are used when a set of actions represent passed through the work of Michael Mateas and Andrew alternative ways of achieving a similar goal. Thus, as Stern [102], and Damian Isla [71]. The ideas were then shown in Algorithm2, the Fallback node routes the spread and refined in the community over a number of Ticks to its children from the left until it finds a child years, with the first journal paper on BTs appearing in that returns either Success or Running, then it returns [47]. The transition from Game AI into robotics was Success or Running accordingly to its own parent. It even later, and independently described in [114] and [4]. returns Failure if and only if all its children return Failure. Note that when a child returns Running or 1.2. Formulation of the core parts of a BT Success, the Fallback node does not route the Ticks to A BT is a directed tree where we apply the standard the next child (if any). The symbol of the Fallback node meanings of root, child, parent, and leaf nodes. The leaf is a box containing the label “?”. nodes are called execution nodes and the non-leaf nodes Parallel nodes tick all the children simultaneously. are called control flow nodes. Graphically, the BT is Then, as shown in Algorithm3, if M out of the N either drawn with the root to the far left and children children return Success, then so does the parallel node. to the right, or with the root on top and children below. If more than N*M return Failure, thus rendering success We will use the latter convention, see the example in impossible, it returns Failure. If none of the conditions Figure2. above are met, it returns running. The symbol of the The execution of a BT starts from the root node, that Parallel node is a box containing the label “Д. generates signals called Ticks with a given frequency. 2Fallback nodes are sometimes also referred to as selector or These signals enable the execution of a node and are priority selector nodes. then propagated to one or several of the children of Iovino et al.: Preprint submitted to Elsevier Page 2 of 22 A Survey of Behavior Trees in Robotics and AI Table 1 The five node types of a BT. Node type Symbol Succeeds Fails Running Sequence → If all children succeed If one child fails If one child returns running Fallback ? If one child succeeds If all children fail If one child returns running Parallel Ð If g M children succeed If > N * M children fail else Action shaded box Upon completion When impossible to complete During completion Condition white oval If true If false Never 3 4 Algorithm 2: Pseudocode of a Fallback node with To do this, the Google Scholar , Scopus , and Clar- 5 N children ivate Web of Science databases were searched with the keyword "Behavior Tree" and alternative spelling 1 for i } 1 to N do "Behaviour Tree".