Cmrobobits: Creating an Intelligent AIBO Robot

AI Magazine Volume 27 Number 1 (2006) (© AAAI) Articles CMRoboBits: Creating an Intelligent AIBO Robot

Manuela M. Veloso, Paul E. Rybski, Scott Lenser, Sonia Chernova, and Douglas Vail

■ CMRoboBits is a course offered at Carnegie Mellon eas of perception, cognition, and action (illus- University that introduces students to all the con- trated in ﬁgure 1) and use the Sony AIBO robots cepts needed to create a complete intelligent robot. to help the students understand in depth the In particular, the course focuses on the areas of per- issues involved in developing such capabilities ception, cognition, and action by using the Sony AIBO robot as the focus for the programming as- in a robot. signments. This course shows how an AIBO and its The course has a two-hour weekly lecture software resources make it possible for students to and a one-hour weekly lab session. The course- investigate and work with an unusually broad va- work consists of weekly homework assign- riety of AI topics within a single semester. While ments and a larger ﬁnal project. The homework material presented in this article describes using AI- assignments include written questions about BOs as the primary platform, the concepts present- the underlying concepts and algorithms as well ed in the course are not unique to the AIBO and as programming tasks for the students to im- can be applied on different kinds of robotic hardware. plement on the AIBO robots. Evaluation is based on the students’ written answers, as well as their level of accomplishment on the programming tasks. All course materials, including student solutions to assignments, are made ince 1997, we have researched teams of soccer robots using the Sony AIBO ERS-210 available on the web. robots as the robot platform (Veloso and Our goal is for our course materials to be S used by other universities in their robotics and Uther 1999; Veloso et al. 2000; Lenser, Bruce, and Veloso 2001a; Lenser, Bruce, and Veloso AI courses. In this article, we present the list of 2001b; Uther et al. 2002). Our experience runs topics that were covered in the lectures and in- across several generations of these four-legged clude examples of homework assignments as robots, and we have met with increasing suc- well as the rationale behind them. This curricu- cess every year. In the fall of 2003, we created a lum shows how the AIBO and its software re- new course building upon our research experi- sources make it possible for students to investi- ence with the AIBO robots. We have since re- gate and work with an unusually broad variety fined the course and taught it again in 2004. of AI topics within a one-semester course. The course, which we entitled CMRoboBits: The Sony AIBO is a commercially available Creating an Intelligent AIBO Robot, introduces robot that is relatively inexpensive as research students to all the concepts needed to create a robots go (on the order of US$2,000). While complete intelligent robot. We focus on the ar- purchasing large quantities of these robots may

Perception Cognition Action

Sensors Actuators

External World

Figure 1. The Modules Used in the Complete Robot.

be beyond the available budget for smaller in- positive effects, namely facilitating our re- stitutions, the robots could be purchased in search work and easing students’ transition in- small quantities for use in special topics courses to that research. and independent research projects. Additional- The AIBO, shown in figure 2, is a remarkable ly, commercially available pedagogical elec- piece of commercially available robotic hard- tronics and robotics kits from other companies ware. An AIBO has 15 degrees of freedom are rapidly approaching the CPU performance (DOF) in its legs and head, a color CCD camera capabilities of the AIBOs. Coupled with inex- that can process images at 25–30 frames/sec- pensive cameras and actuators, such kits could ond, a three-axis accelerometer for body pose very easily be used in a complementary fashion estimation, buttons on its back, head, and foot- to the courseware described in this article. pads, LEDs for visual debugging, and a wireless Ethernet (802.11b) card for interrobot and The Goals of the Course host-computer communication. Students program AIBOs using a free SDK called OPEN-R,1 and the Schedule which lets them compile control code on a The main goal of the course is to learn how to workstation with a MIPS cross-compiler (avail- create an intelligent robot, using the AIBO plat- able for GNU Linux, Microsoft Windows, and form as a concrete example. We want the stu- Mac OS X). The AIBO’s low cost allows an in- dents to understand how to program the robots structor to purchase several of them for the to perform tasks. We view a robot as a complete price of a more traditional research robotic intelligent agent, in the sense that it includes platform. perception, cognition, and action. Our aim This 15-week course contains several main through the course is to demystify robot pro- components, as we present below. In the first gramming so that it becomes clear and accessi- version of the course in the fall of 2003, we fol- ble to all of our students. lowed these main components. In the second A parallel goal of particular interest to us is version in the fall of 2004, we put significantly to bridge from our research code in robot soc- more emphasis on behaviors, multirobot coop- cer to modular code that can be used for any eration, and learning. Although all the compo- general robot task. We aim to provide course nents are relevant to the goals of an AI/robotics materials that are modular and well structured course like ours, it is easy to get lost in the low- so that instructors at other universities can use level details of perception and action, for which the materials in their own courses. We have there may be other courses specifically address- found that reorganizing and cleaning up our ing these topics. Instead the focus of our course robot soccer code has had several additional is the complete intelligent agent, hence the em-

68 AI MAGAZINE Articles

Figure 2. Annotated AIBO ERS-210. phasis on cognition and learning. We always teractions between the robot and its environ- teach perception and action, but we balance the ment and are equivalent to input and output topics toward cognition. The following descrip- operators in computer programming. This tions outline how we strike this balance. component of the course introduces students to the idea of acting in the face of uncertainty. Behaviors Unlike traditional programming where input This course familiarizes the students with the values are completely known, robots must per- concept of behaviors for robot control. Every form with only limited, noisy knowledge of component, from sensors to localization, is cast their environment. Additionally, robots must in the framework of how a mobile robot can use cope with noise and uncertainty in the effects those techniques in its behaviors. We reintro- of their actions; motors do not always perform duce behaviors at several times in the course, the requested movements, and factors such as since behaviors are the basic components of vir- friction and slip are difficult to take into ac- tually any robot task. Initially, we introduce ﬁ- count when predicting the outcome of actions. nite-state machines and incrementally address Students are introduced to the idea of uncer- more complex behavioral structures, such as hi- tainty, which is central to robot programming. erarchical behaviors and planning. Figure 3 de- Figure 4 shows an example plot of the three-ax- picts a decomposition of several parallel and se- is accelerometer data that students can use to quential behaviors used in class. determine what “state” the robot is in. Sensors and Actuators Motion Robots perceive the world using their sensors, The AIBO robots offer an interesting and chal- and they affect their environment with their lenging platform for exploring robot motion actuators. Sensors and actuators mediate all in- (ﬁgure 5). AIBOs differ from most educational

SPRING 2006 69 Articles

Decompositional Lifted Straight Sequential Behavior

AIBO back AIBO lifted AIBO lifted AIBO tilted on ground

AIBO back on ground Tilted On Ground Behavior Behavior AIBO tilted

Tilted left Tilted right

SET LED SET LED WALK TROT RUN MIDDLE-LEFT MIDDLE-RIGHT

Figure 3. Description of a Behavior Hierarchy

robots because they are a legged platform with image capture hardware, as well as real time 15 degrees of freedom (DOF) in their head and color segmentation, and camera calibration. legs. Each of the four legs has three DOF, and The final topics include higher-level concepts the head has pan, tilt, and roll joints. In addi- such as object recognition from the color-seg- tion to these major joints, students can actuate mented images, including weeding out false the tail, mouth, ears, and eye LEDs to create positives (figure 6). Students also learn how more expressive behaviors. In this unit, we in- kinematics ties back to vision for calculating troduce students to the ideas of forward and in- the real world position of objects in the vision verse kinematics. We also include a practical in- frames. Figure 6 shows an example frame of troduction to our motion system on the AIBO. postprocessed AIBO video. This image illus- We describe our parameterized walk engine, trates the output of the AIBO’s real-time color- which uses approximately 50 numeric parame- segmentation algorithm. ters to specify an entire gait for the robot. These parameters include factors such as robot body Localization height, body angle, lift heights for each leg, In order to act effectively, a robot often needs and timings. We also introduce students to the to know its location in the environment. Local- idea of frame-based motion where all joint an- ization becomes an essential component that gles are specified for a few key frames and the interacts with perception, decision making, robot interpolates between them. This type of and motion. This unit introduces the ideas of motion is useful for scripting kicking motions probabilistic localization beginning with the for soccer, dance motions, climbing, and other basic ideas of probabilistic localization and in- predefined motions. cluding different methods of representing lo- cale belief, such as Kalman filters and particle Vision filters. We also cover more advanced localiza- The AIBO robots use vision as their primary tion challenges, such as recovering from errors sensor. Color images in the YUV color space ar- in motion models (for example, the kidnapped rive at a frame rate of 25Hz. The vision unit of robot problem) through sensor-based resam- the course acquaints students with the basics of pling algorithms, and study various trade-offs robot visual processing. Students briefly learn that may be made between computational cost about the YUV color space commonly used by and resource consumption.

70 AI MAGAZINE Articles

Multirobot Cooperation

Once the students understand how to program 1.5 a single AIBO to do interesting behaviors, we gsensor data - laying on left, upright, then on right side teach them how to use the AIBO’s onboard 802.11b wireless Ethernet system (figure 7). 1

es) Y accel This feature allows the robots to communicate i t i among themselves. We teach the students how 0.5 av to solve problems with multirobot behaviors, r discuss the challenges, and present several ap- X accel

on (g 0 proaches for multirobot communication and i at coordination, including market-based ap- r e

l Z accel proaches, and cognitive architectures, such as -0.5 the skills-tactics-plays architecture. Acce Learning -1 Throughout the course we explain how robot programming involves a considerable amount -1.5 of parameter tuning in a variety of algorithms. 40 42 44 46 48 50 52 Time (seconds) We introduce learning approaches for motion optimization, control learning, and team play adaptation to an opponent team. Learning is a theme that we have taught in the lectures, but Figure 4. Three-Axis Accelerometer Signature for a Robot students do not currently investigate it deeply That Starts on Its Left Side, Is Rotated to an Upright Position, in their programming assignments. In future it- and Then Is Rotated to Its Right Side. erations of the course, we plan to make learning a more significant component of students’ robot programming assignments. matics engine for performing complex motions with the AIBO’s limbs; (3) a parameterized walk engine for specifying different trot gaits that CMRoboBits the AIBO can use to translate and rotate in any The CMRoboBits code distribution provides all direction; (4) a Monte-Carlo localization sys- the software necessary for complete control of tem (particle filter) using landmarks at known an AIBO. The default code distribution con- positions to track the robot’s pose and orienta- tains all of the functionality necessary to run tion; (5) a world modeling system for tracking simple behaviors (such as chasing a ball). Stu- the positions of detected objects over time; (6) dents are able to use the existing codebase for an interrobot communication system for shar- their homework assignments and typically ing world state with other robots; and (7) write behaviors that make use of the preexist- stream-based debugging tools for viewing de- ing systems. For homework assignments that bug text strings and frames of color-threshold- expand on a particular aspect of the robot’s ca- ed images in real time or from log files. pabilities (such as vision homework assignments), the students are expected to augment Homework Assignments the existing CMRoboBits code with the new functionality. In this section, we briefly describe the ratio- The software includes a number of specific nale, requirements, and grading of the course capabilities, such as a behavior engine for spec- homework assignments. Students are given one ifying complex robot actions. Behaviors can week to complete each assignment. We suggest run in parallel in a priority-based scheme, and that the students work in groups of two or each behavior can invoke a tree of subbehav- three, taking into account the number of stu- iors. It also includes a real-time low-level color- dents relative to the number of robots and the segmentation algorithm for identifying objects weekly homework timing. Assignments are due by specific color types. This also includes the at the beginning of the lab period each week. tools necessary for learning the segmentation This routine allows either a demonstration ses- based on labeled training images. Other capa- sion at the beginning of the lab or a review ses- bilities include (1) a high-level vision object- sion where the teaching assistant reviews stu- recognition system for identifying and return- dents’ code and watches the robots to diagnose ing the three-dimensional positions of specific problems. Helping students debug problems re- objects such as colored markers, balls, and even quires the availability of both the robots and other AIBOs; (2) a frame-based inverse kine- source code.

SPRING 2006 71 Articles

Dynamic Walking Motion Static Frame-Based Motion

Walk Parameters Motion Frames

Walk Engine Frame Interpolator

Figure 5. The Two Motion Control Systems in the CMRoboBits Code Base.

All the homework assignments in the course missions using a dropbox system. This simple include several written questions that cover the first assignment allows us to assess how quickly lecture material as well as the robot program- the students familiarize themselves with the ming assignment. This homework structure al- software and robots. We then tune the other lows us to address both theory and practice. We homework assignments accordingly. discuss the programming part of the home- Basic Sensors. We follow with a homework to work assignments, while the complete ques- familiarize the students with the sensors on the tions can be viewed online at our course web robot. The background section covers how to site. subscribe to sensor messages, specifically, data from the robot’s accelerometer and the touch Introductory Homework Assignments sensors on its feet. The homework sets up a se- We designed a set of initial homework assign- ries of specific sensor-actuator behaviors that ments that successfully familiarizes the the robot needs to perform exactly. In particu- students with the robot’s software develop- lar, LEDs need to be set on the robot’s face in ment environment, core hardware, and simple different situations, including when the robot motion behaviors. has a foot in contact with the ground and Development. Our first homework serves as when the robot is lifted off the floor. LEDs also an introduction to the development environ- need to be set to display whether the robot is ment and brings students up to speed on how leveled, tilted toward its left side, or tilted to its to access the source code from our repository, right. compile the code using the OPENR SDK (freely This assignment gives the students practical available from Sony), and copy the final pro- experience with the sense-think-act loop. They grams to memory sticks for use with an AIBO. must read (noisy) sensor data from the robot, This first homework assignment teaches the determine which actions to take based on this students how to operate the robot’s software in- sensor data, and finally send commands to the frastructure. The homework also shows stu- robot to perform these actions. This sequence is dents how to select which behavior runs using repeated with a frequency of 25 Hz on the ro- our framework and allows us to test code sub- bot (the frame rate of the camera). The home-

72 AI MAGAZINE Articles work focuses exclusively on the basic sensors and does not require use of the vision-processing system. Robot Motion Homework Robot motion involves a substantial amount of trial and error. In this motion homework, students learn how to build up to a complete motion. We provide our walk engine, which im- plements the needed forward and inverse kinematics and sets up a parameter-based inter- face for motion variations. The assignment is broken down into two parts. In the first part, students create a set of walk parameters to describe a gait. The walk engine uses a set of 51 parameters to generate the actual trajectory that the end of each foot follows over the course of a single step. The parameters include limits on how high each foot can rise above the ground, the desired angle of the robot’s body, and other similar factors. Finding an effective walk is an optimization in this 51-dimensional parameter space. The walk Figure 6. An Example Frame of Video from the engine parameters are not completely indepen- AIBO’s Camera after Color-Segmentation Postprocessing. dent, which makes finding an optimal set of parameters extremely difficult. Typically we op- timize for speed and stability, although other factors such as a walk with a high body height are possible. We challenge the students to develop a learning approach, as extra credit, as an alternative to the incremental, trial and error experimentation for parameter setting. While this extra credit offers a natural connection with machine learning, we were surprised to find an interesting secondary benefit to hand- tuning gait parameters. Doing so greatly in- creased students’ understanding of and famil- iarity with the robots. The homework also tests frame-based motion. The students are to create a new motion using our key-frame animation–based approach. We ask for specific motions, such as a motion that makes the robot perform a complete rollover and then climb back onto its feet. The students learn how to convert between the positions of the robot’s limbs in space and the corresponding angles of the robot’s joints in their own coordinate frame. Since rolling over Figure 7. Students Learn Some of the Challenges with Programming is a dynamic activity that depends on building Behaviors for Cooperative Multirobot Tasks. up momentum and moving different legs in concert, the students also learn how to coordinate different joints simultaneously. Success gram a complete intelligent robot. We focus on with the frame-based portion of the assign- the high-level vision perception and behaviors, ment requires an incremental, experimenta- but working with real robotic hardware also in- tion-based approach. cludes calibrating sensors for use in specified environments. Our homework assignments ad- Vision Homework Assignments dress these low-level and high-level vision-pro- The purpose of our course is to teach the stu- cessing challenges. dents the knowledge that they need to pro- Calibrating Vision. The CMRoboBits vision

SPRING 2006 73 Articles

Figure 8. The Egocentric Local Obstacle Model Where the Robot Is in the Center of the Grid. Left. Obstacles and freespace are represented as samples from the samples; black = freespace, white = obstacles. Right. Occupancy grid generated from samples; black = freespace, white = obstacles. Scanlines from the visual perceptual system are parsed for colors that are freespace and obstacles (according to the color-segmentation algorithm). The scanlines are sampled and a collection of points is added to the local model’s database, as shown in the ﬁgure on the left. These points have a ﬁnite lifetime (approximately two seconds) before being forgotten. These points can be turned into a more traditional grid-based occupancy grid by summing the contribution of each of the freespace and obstacle points in that grid.

system uses a color-segmentation algorithm to segmentation, they must program the AIBO to semantically label pixel colors with specific perform higher-level tasks such as object recog- meanings (for example, orange is a ball and nition. Students program their AIBOs to detect green is a playing ﬁeld). These labels are stored a bright orange ball, a colored bull’s-eye, a in a lookup table that assigns labels to every small scooter, a two-color sign, and a tower pixel in the full YUV color space. The students built from colored cylinders using color-seg- must train the lookup table from captured im- mented images. We chose the objects so that age data. the students would have to face many of the In this assignment, students use the robot’s challenges caused by ambiguous vision infor- camera to capture still images from the envi- mation. For instance, the wheels of the scooter ronment and then save those images to a log are the same shade of orange as the ball, so the file that is transferred to a workstation. The students must consider shape and relative size. students use standard image-editing software The students have to recognize the orientation to assign semantic labels to each of the colors of the two-colored sign (vertical, horizontal, in the images. As an example, they assign the and diagonal), so they have to consider what ball label to the parts of the image that include the image looks like from various camera an- a ball. These labeled images serve as training gles. data for a supervised learning algorithm that learns a mapping between YUV color values Single-Robot Behavior and symbolic color values. One important as- Homework Assignments pect of this assignment is to give students an In the third set of homework assignments we appreciation of how much work is required to focus on single-robot behaviors. The main goal obtain a usable vision calibration. is to teach the students how to connect percep- Object Recognition. Once students under- tion, cognition, and action in more complex stand low-level vision concepts such as color tasks. The homework assignments require the

74 AI MAGAZINE Articles students to process perceptual data, effectively select actions, and move in different environments. The students build upon their skills Initialization from previous homework assignments to solve Moderatorrandomlydetermines a sequence of length n these problems. drawn from a set of m possiblecolors(withrepeated colors Maze Traversal. In this maze-traversal assign- allowed) ment, students use a (provided) egocentric spatial world model to track regions of free space around the robot. Figure 8 illustrates data cap- Play tured from the egocentric spatial world model. Students create a behavior to traverse a convo- Player guesses a sequence luted mazelike path. The students have to con- Moderator tellsplayerhow many colorsare: sider how to aim the camera on the robot’s Correct and inthe rightposition Correct but inthewrong position head to ensure that the local spatial model con- Continue until theplayerhas deduced and guessed the hidden tained accurate and up-to-date information at sequence from the mn initial possibilities all times. The path is not a true maze, as it has no dead ends, but the robots have to navigate through several turns without touching the Theoriginal game rules and an on-line playableversion is walls. available here: Localization. Students create a behavior that http://www.kongtechnology.com/index.asp?im=mastermind allows the robots to avoid a large square in the center of a carpet. The robot starts in a home position and calculates (and memorizes) its position. The robot is manually picked up and Figure 9. Rules for the Game of Mastermind moved to a new position (typically on the other side of the carpet) and replaced on the car- create custom motions to move the robots into pet. The robot has to identify that it has been position over the chargers and settle them- moved so it can return to its original position. selves onto the contacts. While returning, the robot has to avoid the square in the center of the carpet by using only Multirobot Behavior its localization. The robot uses six colored Homework Assignments markers placed around the carpet for localization. Students have to reason about how to One of the major challenges of creating intelli- control the head to seek out and fixate on these gent robots is to have them coordinate with markers. Without accurate visual measure- other robots. We focus a major part of the ments of the landmarks, the robot cannot lo- course on multirobot behaviors and therefore calize properly. Localization requires more for- also develop a corresponding set of homework mal mathematical background than the rest of assignments. the material in the course. For the written com- Mastermind. This homework introduces stu- ponent of this homework, students have to dents to the concept of human/robot interac- manually calculate a posterior probability of tion and learning on a real robot. The students the robot’s position given a uniform prior dis- program their AIBOs to play a guessing game tribution of robot poses in a grid world. by which one player (either the human or the Mounting a Charging Station. Students use the AIBO) guesses a sequence of colored markers object-detection code written in previous that the other player (AIBO or human, respec- homework assignments to find a colored bull’s- tively) makes up ahead of time. The AIBO com- eye and tower beacon. These two landmarks al- municates to the human by a predefined set of low the robot to compute the distance and ori- motions. When guessing the colored sequence, entation of a charging station. The robot needs the AIBO has to reason about the patterns of to search for and then climb onto the charging the colors as well as about the clues given to it station. Once on the charging station, the ro- by the human. This homework connects more bot has to shut itself off so it can recharge. traditional AI techniques (game playing and This assignment requires the students to use state space search) with the challenges of hu- much of the material that they learned in pre- man/robot interactions. This homework is de- vious assignments and tie it together into a scribed in greater detail in the next section. unified whole. Besides having to use vision to Multirobot Ball Collection. Students write mul- find the landmarks, the students have to create tirobot behaviors that make use of interrobot a custom walk that lifts the body high enough communication. The students program a pair of to step onto the station. Students also need to AIBOs to cooperatively push colored balls into

SPRING 2006 75 Articles

a goal. There are two components to this home- Part One: AIBO as Moderator work, whereby the students first solve the task In the first part, the AIBO generates a random by not having the AIBOs communicate, but sequence of two colors that the student must rather treat each one independently, and then guess. The required steps for play are as follows: solve the task where they can use communication (802.11b wireless Ethernet) to coordinate 1. The student presses the back button to unpause the robot. the actions of the robots. After completing the behaviors, the students need to analyze the per- 2. The AIBO starts in the play position, in this formance of both the noncommunicating and case a resting position on its belly. communicating cases and to report on any dif- 3. The student presses the button under its chin ferences in performance. to start the game. The AIBO selects a random sequence and blinks both middle LEDs twice to indicate the start of game play. The AIBO will Example Assignment: also transmit the color sequence over the wire- Mastermind less network for debugging purposes. This section provides the text for an actual 4. Students choose two colored squares and homework from the fall 2004 CMRoboBits place them in front of the AIBO at a specified class. This homework gives the students an op- distance. portunity to work with vision and motions and 5. The students indicate that they are ready for requires them to reason about complex behav- the AIBO to evaluate their choice by pressing iors that involve interacting and communicat- the back head button. ing with a human in a deterministic fashion. 6. If all goes well, the AIBO will see the two balls In this homework, the students program the and respond by (a) lighting n green LEDs to il- AIBOs to play the game of Mastermind with a lustrate the number of squares that are of the human and do so with either the robot as play- correct color and in the right place (correspond- er and the human as moderator or vice versa. ing to the black pegs in Mastermind), (b) show- Mastermind is a guessing game in which the ing m red LEDs to illustrate the number of balls player must guess an ordered sequence of col- that are of the correct color but in the wrong ors defined in secret by a moderator. The mod- place (corresponding to the white pegs in Mas- erator provides the player with simple but not termind), and (c) nodding the head to signify necessarily decisive clues that the player must that its evaluation has completed. use to infer the correct colored sequence. Figure 7. The game continues with the students plac- 9 describes the rules. The following homework ing squares and requesting a response with the description mentions a program by the name AIBO’s back head button until the right se- of chokechain, which is a Linux-based console quence is observed. debugging tool that connects to an AIBO over 8. The AIBO rolls over when it observes the cor- wireless TCP/IP. Students use this program to rect sequence. view debug statements (referred to in the homework as pprintf), as well as color-thresh- Part Two: AIBO as Player olded frames of AIBO video. In this section, the AIBO takes the role of play- Introduction er. Students must implement the logic for Mas- This homework is geared towards switching be- termind formally in the AIBOs. This requires tween different behaviors (state machines in keeping some history of what the AIBO has state machines), contains an opportunity to ex- seen and the responses it received. The required periment with learning, and requires some aug- steps for play are as follows: mentation to the robot’s vision. In this home- 1. Students press the back button to unpause work, students play the game Mastermind with the robot. the AIBOs. 2. Students pick a random sequence of colors and press the button under its chin to start the Game Setup play sequence. The students play a simplified version of the game with colored squares with only two posi- 3. Students retrieve the AIBO’s guess by show- ing it different colored squares (one at a time) tions and three colors (repeats allowed) to and eliciting a position response for each choose from. The moderator picks a sequence square. Specifically, for each square of color C = while the player uses the responses from the C1, C2, C3, if the AIBO wants to use square C as moderator to guess the sequence. Students in- part of its guess, it will nod “yes” (up and down) teract with their AIBOs through simple mo- and raise its right or left paw to indicate tions, buttons, and LEDs. whether it wants the square in the left or right

76 AI MAGAZINE Articles

Figure 10. Mastermind: The AIBO on the Left Is the Moderator and the AIBO on the Right Is the Player.

position. If it wants the color in both slots, it for the player. In this picture, the player noti- will raise both paws. If it does not want the col- fies the human that it wants to place the col- or, it will shake its head “no” (left and right). ored square on the left. 4. When both colored squares have been select- ed by the AIBO, the students give feedback to the AIBO by (a) pressing the rear button on its Class Final Project head to tell it to “start input;” (b) pressing its Students in the 2003 and 2004 CMRoboBits left front footpad n times where n is the number courses completed a final project worth 30 per- of squares of the correct color in the correct po- cent of their overall grade. In 2003, the stu- sition; (c) pressing its right front footpad m dents proposed a project that demonstrated times where m is the number of squares of the correct color in the wrong position; and (d) some interesting behavior of the AIBOs, where- pressing the rear button on its head to tell it to as in 2004, the students all completed the same “stop input.” project. We feel that both approaches have specific advantages. The open-ended style of the 5. Students repeat this sequence of square 2003 project encourages creativity and allows guesses and input until the AIBO correctly guesses the sequence. Then, they press the but- students to explore what they learn and enjoy ton on the AIBO’s back to tell it that it has most in the course, while the fixed style of the guessed correctly. As a final motion, the AIBO 2004 project tests the students’ complete should roll over or do some cute thing. knowledge of the course materials.

Questions Final Project, 2003 In addition to the programming component, In this project style, the students turn in a writ- the students are expected to answer two written proposal of their project and then are ten questions regarding how they might gener- coached by the professor to ensure that they alize the game rules to an arbitrary number of can complete their project by the end of the colors and slots. course. Because this style of project is so open- Question 1: Using candidate elimination by ended, we feel that this encourages the stu- enumerating all the values can become ridicu- dents to express their creativity and do a pro- lous for large problem spaces. For arbitrary m ject that is interesting to them. Several and n, how can one perform candidate elimina- interesting project ideas are described in the tion concisely? following paragraphs and illustrated in ﬁgures Question 2: With large problems, it is desir- 11–15. able to guess those hypotheses that produce Obstacle Course. An AIBO is programmed to the most information and thereby reduce the navigate an obstacle course that consists of an total number of guesses to win. Explain how to archway, a straight corridor, a tunnel, and a choose a hypothesis from a remaining set. step. The robot is programmed to identify each The human manipulates the colored squares obstacle by viewing a colored strip on the

SPRING 2006 77 Articles

Figure 11. Student Final Project: An AIBO Navigating through an Obstacle Course. (Thanks to Katie Chang and Ling Xu.)

ground. When encountering an obstacle, the that it took. After reaching the goal, the AIBO robot performs a custom motion to move past restarts the maze at the beginning and demon- the obstacle. Students learn to use vision to strates that it always takes the same correct identify obstacles through markers and to write path through the maze to the goal. Students custom motions to surmount them. learn to use the vision system for navigation Tag. This project uses localization and inter- and obstacle avoidance as well as to program robot communication to allow two robots to their AIBOs to reason about topological maps. play a game of tag. Both robots use the six land- AIBO Dance. Two AIBOs dance in time to a marks placed around the carpet to localize custom song. This project demonstrated the themselves. They use this information to trans- wide range of possible AIBO motions. Such mo- mit their x, y locations to each other. With this tions include flipping over to walk on their shared information, the robot designated as backs, sitting and waving both front paws in “it” tries to come within a minimum distance the air, and rolling completely over. Students of the other robot while that other robot tries focus on developing custom motions for a pair to evade. The AIBOs switch roles when one of AIBOs and synchronize their motions. comes within a minimum distance of the other. Students learn how to use localization, com- Final Project, 2004 municate effective state information, and syn- In this project style, every student is required to chronize behaviors. program the robots to complete the same task. Maze Learning. An AIBO explores a maze of t- Controlling the subject material allows the in- junctions and dead ends using the robot’s visu- structor to more thoroughly test the students’ al sonar sensor module to identify distances to cumulative knowledge of the AIBOs and the the maze walls. The robot starts at one part of CMRoboBits source code. This project required the maze and explores until it ﬁnds an orange students to program two AIBOs to cooperative- ball at the end. Whenever the robot encounters ly ﬁnd their way through two identical mazes. a t-junction, it remembers the last direction Figure 15 shows example mazes.

78 AI MAGAZINE Articles

Figure 12. Student Final Project: Two AIBOs Playing Tag. (Thanks to Noah Falk and Joe Delﬁno.)

Figure 13. Student Final Project: An AIBO Learning a Maze through Exploration. (Thanks to Sylvain Paillard and Abe Wong.)

SPRING 2006 79 Articles

Figure 14. Student Final Project: Two AIBOs Dancing in Time to Music. (Thanks to Titika Sanghi and Yash Patodia.)

Figure 15. Final Project Maze in 2004.

80 AI MAGAZINE Articles

Each AIBO starts in its own maze and has to whereby the robots have the opportunity to find its way through to the end. Each maze improve their performance through experi- contains an obstacle that the AIBO must either ence. We seek to combine the best features of climb over, crawl under, or crawl through. The each of the last two final project approaches: end of the maze contains an open area filled the creativity that arose from carefully and re- with colored balls that the robots must cooper- alistically chosen student-designed projects atively empty. This project emphasizes navigat- and the breadth of skills demonstrated in the ing through the maze, overcoming obstacles cooperative maze-clearing project. We intend with motions, and cooperating through wire- to offer the same final project to all students less communication. but will allow exceptional students to propose Cumulative Skill Testing. For this final project, projects. Such project proposals must exhibit students must make use of their knowledge of the same breadth of coverage over the course both single and multirobot behaviors. Single materials before they are approved. robot behaviors include (1) navigation and obstacle avoidance; (2) visual recognition and in- terpretation of landmarks; (3) motions to climb Conclusion over obstacles; (4) visual navigation to colored We are very interested in teaching AI concepts targets. The multirobot behaviors include (1) within the context of creating a complete intel- synchronization of world model for maze nav- ligent robot. We believe that programming ro- igation; (2) negotiation and agreement on bots to be embedded in real tasks illustrates shared action for ball task. some of the most important concepts in artifi- Cooperative Navigation in the Maze. In order cial intelligence and robotics, namely, sensing to navigate the maze, the AIBOs receive clues at uncertainty, reactive and deliberative behav- each of the t-junctions as to which direction iors, and real-time communication and mo- they should turn to reach the end of the maze. tion. These clues consist of blue and pink signs that We seek to find a good balance between the the robots must interpret. If the sign tilts to the theoretical aspects of the in-class lectures and left, the robot takes the left path. If the sign tilts the hands-on labwork. We feel very strongly to the right, the robot takes the right path. If that both are required to achieve a well-round- the sign does not tilt, the robot turns around ed learning experience. As of the 2004 class, the and takes the path behind it. In order to read students did not have any in-class midterm or and interpret the signs correctly, the robots final exam in lieu of a more in-depth final pro- must face the sign head on. ject; we have decided to include them the next In order to encourage cooperation between time the course is taught. There are too many the robots, not every t-junction in a specific ro- concepts to fit into a cumulative final project. bot’s maze has a sign. However, between the A more traditional exam schedule will fill this two mazes, each t-junction has a sign. There- gap. In 2003 and 2004, the class used the ERS- fore, students program their AIBOs to commu- 210 model of the AIBO. In 2005, the students nicate with each other to share the information began using the newer AIBO ERS-7s, which as to which direction they should turn when have higher-resolution cameras, slightly differ- they arrive at the same t-junction. ent kinematics, and more processing power. Cooperative Cleaning. When both robots find It is a testament to the AIBO’s capabilities their way through the maze, they work togeth- that the course explores too many AI topics to er to remove all of the colored balls from two capture in a single final project. Indeed, we are independent open areas (each robot had its seeking to better integrate topics such as ma- own separate area). In order to remove a ball, chine learning (and perhaps there are others) both robots need to navigate a ball of the same in future offerings. We have found that, with color and stop with their head over it. If both the AIBO’s hardware and software resources, balls are the same color when the robots stop, undergraduate students can efficiently and ef- the balls are manually removed by an observ- fectively investigate the broad and growing ing human. If the robots stop over two balls of connections among AI robotics topics. different colors, the balls are reset to a new The current course materials, including area. Because the robots do not see each other’s some final project videos, are available on the activities, they must communicate with each course web page listed in table 1. other. Acknowledgements Future Final Projects We would like to thank Sony for its remarkable In 2005, we plan to develop a final project that support of our research, specifically by making includes a significant learning component the AIBO robots accessible to us since their first

SPRING 2006 81 Articles

Web Site URL Course web page for thecurrent year http://www.andrew.cmu.edu/course/15-491 Archived course materialsfrom previousyears http://www.cs.cmu.edu/~robosoccer/cmrobobits SONY OPEN-R SDK home page http://openr.aibo.com AIBOs at RoboCup http://www.robocup.org CORAL research groupwebpage http://www.cs.cmu.edu/~coral

Table 1. Web Resources for Using AIBOs and the CMRoboBits Course Materials in Education. conception in 1997. Sony has contin- Robot Soccer World Cup III. Lecture Notes in He served as cochair for the 2005 and 2006 ued its support through these years Computer Science 1856, ed. M. Veloso, E. AAAI Mobile Robot Competition and Exhi- and is very interested in following the Pagello, and H. Kitano, 766–769. Berlin: bition. Springer Verlag. impact of an AIBO-based course. We Scott Lenser graduated would also like to thank the Carnegie Manuela M. Veloso is with a B.S. in computer Mellon Computer Science Department professor of computer engineering from the for approving this new course in the science at Carnegie Mel- University of Illinois in fall of 2003 and providing the lab fa- lon University. She re- Urbana-Champaign in cilities. ceived her Ph.D. in com- December 1996. After working for Intel Corpo- Thanks to Nick Aiwazian, James puter science from ration in Portland, Ore- Bruce, and Nidhi Kalra for help with Carnegie Mellon in 1992. She received a B.S. gon, for a year and a the course and robot development. degree in electrical engi- half, he enrolled in the Ph.D. program at Thanks also to Sony for having provid- neering in 1980 and an M.Sc. in electrical Carnegie Mellon University. While at ed to us the remarkable AIBO robots and computer engineering in 1984 from Carnegie Mellon, he participated in since early 1997. the Instituto Superior Tecnico in Lisbon. Carnegie Mellon’s entry into the legged Veloso’s research focuses on teams of intel- league of the RoboCup robot soccer compe- Note ligent robots where cognition, perception, tition from 1999 to 2005, leading the 2002 1. Found at http://openr.aibo.com/. and action are seamlessly integrated to ad- world champion team. He graduated with dress dynamic environments. In fall 2003, his Ph.D. in August 2005. He is currently References building upon her extensive experience employed at iRobot Corp. as a robotics re- with artiﬁcial intelligence and robotics, she searcher. Lenser, S.; Bruce, J.; and Veloso, M. 2001a. created a new undergraduate course, CM- CMPack’00. In RoboCup 2000: Robot Soccer Sonia Chernova is a RoboBits: Creating an Intelligent AIBO Ro- World Cup IV. Lecture Notes in Computer Ph.D. student in the bot, at Carnegie Mellon to concretely teach Science 2019, ed. P. Stone, T. Balch, and G. Computer Science De- all levels of robot algorithms. Veloso is a partment at Carnegie Kraetzschmar, 623–626. Berlin: Springer Fellow of the American Association of Arti- Verlag. Mellon University. Her ficial Intelligence. She is vice president of research interests focus Lenser, S.; Bruce, J.; and Veloso, M. 2001b. the RoboCup International Federation. She on learning and adapta- CMPack: A Complete Software System for was awarded an NSF Career Award in 1995 tion in autonomous ro- Autonomous Legged Soccer Robots. In Pro- and the Allen Newell Medal for Excellence botic systems. ceedings of the Fifth International Conference in Research in 1997. Veloso is the program on Autonomous Agents, 204-211. New York: chair of the 2007 International Joint Con- Douglas Vail is a Ph.D. ACM Special Interest Group on Artificial ference on Artiﬁcial Intelligence. student in the Computer Intelligence, Association for Computing Science Department at Machinery. Paul E. Rybski is a sys- Carnegie Mellon Univer- tems scientist in the Ro- Uther, W.; Lenser, S.; Bruce, J.; Hock, M.; sity. Advised by Manuela botics Institute at and Veloso, M. 2002. CM-Pack’01: Fast Veloso, his research in- Carnegie Mellon Univer- terests include building Legged Robot Walking, Robust Localiza- sity. He received his Ph.D tion, and Team Behaviors. In RoboCup probabilistic models to and M.S. in computer detect abstract robot 2001: Robot Soccer World Cup V. Lecture science from the Univer- Notes in Computer Science 2377, ed. A. states from sensor data, multirobot teams, sity of Minnesota in and robot soccer. Birk, S. Coradeschi, and S. Tadokoro. Berlin: 2003 and 2001, respec- Springer Verlag. tively. He received an interdisciplinary B.A. Veloso, M., and Uther, W. 1999. The in mathematics and computer science from CMTrio-98 Sony Legged Robot Team. In Lawrence University. His research interests RoboCup-98: Robot Soccer World Cup II. Lec- include distributed sensing and state esti- ture Notes in Computer Science 1604, ed. mation algorithms for teams of mobile ro- M. Asada and H. Kitano, eds., 491–497. bots, robust high-level environment mod- Berlin: Springer Verlag. eling for sensor-impoverished robotic Veloso, M.; Lenser, S.; Winner, E.; and systems, and recognition of agent (hu- Bruce, J. 2000. CM-Trio-99. In RobotCup-99: man/robot) activities through observation.

82 AI MAGAZINE