1153 Multimedia Contents 46. Simultaneous Localization Simultaneoand Mapping u Part E | 46 Cyrill Stachniss, John J. Leonard, Sebastian Thrun 46.1 SLAM: Problem Definition..................... 1154 This chapter provides a comprehensive intro- 46.1.1 Mathematical Basis ................... 1154 duction in to the simultaneous localization and 46.1.2 Example: SLAM mapping problem, better known in its abbreviated in Landmark Worlds .................. 1155 form as SLAM. SLAM addresses the main percep- 46.1.3 Taxonomy of the SLAM Problem .. 1156 tion problem of a robot navigating an unknown environment. While navigating the environment, 46.2 The Three Main SLAM Paradigms ........... 1157 the robot seeks to acquire a map thereof, and 46.2.1 Extended Kalman Filters ............ 1157 at the same time it wishes to localize itself us- 46.2.2 Particle Methods ....................... 1159 46.2.3 Graph-Based ing its map. The use of SLAM problems can be Optimization Techniques............ 1162 motivated in two different ways: one might be in- 46.2.4 Relation of Paradigms................ 1166 terested in detailed environment models, or one might seek to maintain an accurate sense of a mo- 46.3 Visual and RGB-D SLAM ........................ 1166 bile robot’s location. SLAM serves both of these 46.4 Conclusion and Future Challenges ........ 1169 purposes. Video-References......................................... 1170 We review the three major paradigms from which many published methods for SLAM are de- References................................................... 1171 rived: (1) the extended Kalman filter (EKF); (2) particle filtering; and (3) graph optimization. We also review recent work in three-dimensional (3-D) tance-sensors (RGB-D), and close with a discussion SLAM using visual and red green blue dis- of open research problems in robotic mapping. This chapter provides a comprehensive introduction tured, and of limited size. Robustly mapping unstruc- into one of the key enabling technologies of mobile tured, dynamic, and large-scale environments in an on- robot navigation: simultaneous localization and map- line fashion remains largely an open research problem. ping, or in short SLAM. SLAM addresses the problem The historical roots of methods that can be applied of acquiring a spatial map of an environment while si- to address the SLAM problem can be traced back to multaneously localizing the robot relative to this model. Gauss [46.1], who is largely credited for inventing the The SLAM problem is generally regarded as one of the least squares method. In the Twentieth Century, a num- most important problems in the pursuit of building truly ber of fields outside robotics have studied the making autonomous mobile robots. It is of great practical im- of environment models from a moving sensor platform, portance; if a robust, general-purpose solution to SLAM most notably in photogrammetry [46.2–4]andcom- can be found, then many new applications of mobile puter vision [46.5]. Strongly related problems in these robotics will become possible. fields are bundle adjustment and structure from mo- While the problem is deceptively easy to state, it tion. SLAM builds on this work, often extending the presents many challenges, despite significant progress basic paradigms into more scalable algorithms. Mod- made in this area. At present, we have robust methods ern SLAM systems often view the estimation problem for mapping environments that are mainly static, struc- as solving a sparse graph of constraints and applying 1154 Part E Moving in the Environment nonlinear optimization to compute the map and the tra- chosen by the practitioner will depend on a number of jectory of the robot. As we strive to enable long-lived factors, such as the desired map resolution, the update autonomous robots, an emerging challenge is to handle time, and the nature of the features in the map, and so massive sensor data streams. on. Nevertheless, the three methods discussed in this This chapter begins with a definition of the SLAM chapter cover the major paradigms in this field. problem, which shall include a brief taxonomy of dif- For more a detailed treatment of SLAM, we refer ferent versions of the problem. The centerpiece of this the reader to Durrant-Whyte and Bailey [46.6, 7], who Part E | 46.1 chapter is a layman introduction into the three major provide an in-depth tutorial for SLAM, Grisetti et al. paradigms in this field, and the various extensions that for a tutorial on graph-based SLAM [46.8], and Thrun exist. As the reader will quickly recognize, there is no et al., which dedicates a number of chapters to the topic single best solution to the SLAM method. The method of SLAM [46.9]. 46.1 SLAM: Problem Definition The SLAM problem is defined as follows: A mobile surements are noisy, and path integration techniques robot roams an unknown environment, starting at an inevitably diverge from the truth. initial location x0. Its motion is uncertain, making it Finally, the robot senses objects in the environment. gradually more difficult to determine its current pose in Let m denote the true map of the environment. The global coordinates. As it roams, the robot can sense its environment may be comprised of landmarks, objects, environment with a noisy sensor. The SLAM problem surfaces, etc., and m describes their locations. The en- is the problem of building a map of the environment vironment map m is often assumed to be time-invariant, while simultaneously determining the robot’s position i. e., static. relative to this map given noisy data. The robot measurements establish information be- tween features in m and the robot location xt.Ifwe, 46.1.1 Mathematical Basis without loss of generality, assume that the robot takes exactly one measurement at each point in time, the se- Formally, SLAM is best described in probabilistic ter- quence of measurements is given as minology. Let us denote time by t, and the robot location by x . For mobile robots on a flat ground, x is t t Z Dfz ; z ; z ;:::;z g : (46.3) usually a three-dimensional vector, comprising its two- T 1 2 3 T dimensional (2-D) coordinate in the plane plus a single rotational value for its orientation. The sequence of lo- Figure 46.1 illustrates the variables involved in the cations, or path, is then given as SLAM problem. It shows the sequence of locations and sensor measurements, and the causal relationships be- XT Dfx0; x1; x2;:::;xT g : (46.1) tween these variables. This diagram represents a graph- Here T is some terminal time (T might be 1). The x initial location 0 often serves as a point of reference x x x for the estimation algorithm; other positions cannot be t –1 t t+1 sensed. Odometry provides relative information between ut –1 ut ut+1 two consecutive locations. Let ut denote the odometry that characterized the motion between time t 1and zt –1 zt zt+1 time t; such data might be obtained from the robot’s wheel encoders or from the controls given to those mo- tors. Then the sequence m UT Dfu1; u2; u3 :::;uT g (46.2) Fig. 46.1 Graphical model of the SLAM problem. Arcs in- characterizes the relative motion of the robot. For noise- dicate causal relationships, and shaded nodes are directly free motion, UT would be sufficient to recover the poses observable to the robot. In SLAM, the robot seeks to re- from the initial location x0. However, odometry mea- cover the unobservable variables Simultaneous Localization and Mapping 46.1 SLAM: Problem Definition 1155 ical model for SLAM. It is useful in understanding the When building 2-D maps, point-landmarks may cor- dependencies in the problem at hand. respond to door posts and corners of rooms, which, The SLAM problem is now the problem of recov- when projected into a 2-D map are characterized by ering a model of the world m and the sequence of a point coordinate. In a 2-D world, each point-landmark robot locations XT from the odometry and measurement is characterized by two coordinate values. Hence the data. The literature distinguishes two main forms of the world is a vector of size 2N,whereN is the number SLAM problem, which are both of equal practical im- of point-landmarks in the world. In a commonly stud- Part E | 46.1 portance. One is known as the full SLAM problem:it ied setting, the robot can sense three things: the relative involves estimating the posterior over the entire robot range to nearby landmarks, their relative bearing, and path together with the map the identity of these landmarks. The range and bearing may be noisy, but in the most simple case the identity . ; ; /: p XT m j ZT UT (46.4) of the sensed landmarks is known perfectly. Determin- ing the identity of the sensed landmarks is also known Written in this way, the full SLAM problem is the as the data association problem. In practice, it is one of problem of calculating the joint posterior probability the most difficult problems in SLAM. over X and m from the available data. Notice that the T To model the above described setup, one begins variables right of the conditioning bar are all directly with defining the exact, noise-free measurement func- observable to the robot, whereas those on the left are tion. The measurement function h describes the work- the ones that we want. As we shall see, algorithms for ings of the sensors: it accepts as input a description of the full SLAM problem are often batch, that is, they the environment m and a robot location x ,anditcom- process all data at the same time. t putes the measurement The second, equally important SLAM problem is the online SLAM problem.