Robocup 2005 Teamchaos Documentation

ROBOCUP 2005 TEAMCHAOS DOCUMENTATION

Vicente MatellánOlivera Humberto Mart´ınezBarberá Francisco Mart´ınRico David Herrero Pérez Carlos Enrique AgüeroDurán Victor Manuel GómezGómez

University of Murcia Rey Juan Carlos University Department of Information and Systems and Communications Group Communications Engineering E-28933 Madrid, Spain E-30100 Murcia, Spain

Miguel Angel Cazorla Quevedo Mar´ıaIsabel Alfonso Galipienso Alessandro Saﬃotti Antonio Bot´ıaMart´ınez Kevin LeBlanc Boyan Ivanov Bonev

University of Alicante Orebro¨ University Department of Computer Science and Center for Applied Autonomous Artiﬁcial Intelligence Sensor Systems E-03690 Alicante, Spain S-70182 Orebro,¨ Sweden

Acknowledgements

For the elaboration of this document we have used material from past team members, be them from the original TeamSweden or its follow-up TeamChaos. The list is becoming rather large, so we strongly thank all of them for their contributions and eﬀort, which has allow us to continue with the work. In particular, this document is based on previous team description papers and some conference papers, from which we have borrowed an important amount of information. Although they appear in the references, we want to explicitly cite the following papers and give extra thanks to their authors:

• A. Saﬃotti and K. LeBlanc. Active Perceptual Anchoring of Robot Behavior in a Dynamic Environment. Int. Conf. on Robotics and Automation (San Francisco, CA, 2000) pp. 3796-3802.

• P. Buschka, A. Saﬃotti, and Z. Wasik. Fuzzy Landmark-Based Localization for a Legged Robot. Int. Conf. on Intelligent Robotic Systems (IROS) Takamatsu, Japan, 2000.

• Z. Wasik and A. Saﬃotti. Robust Color Segmentation for the RoboCup Domain. Int. Conf. on Pattern Recognition (ICPR), Quebec City, CA, 2002.

• A. Saﬃotti and Z. Wasik. Using Hierarchical Fuzzy Behaviors in the RoboCup Do- main . In: C. Zhou, D. Maravall and D. Ruan, eds, Autonomous Robotic Systems, Springer, DE, 2003.

• J.P C´anovas, K. LeBlanc and A. Saﬃotti. Multi-Robot Object Localization by Fuzzy Logic. Proc. of the Int. RoboCup Symposium, Lisbon, Portugal, 2004.

TeamChaos Documentation Contents

Contents

1 Introduction 1 1.1 History ...... 1 1.2 Team Members ...... 2 1.2.1 University of Murcia, Spain ...... 2 1.2.2 Rey Juan Carlos University, Spain ...... 3 1.2.3 University of Alicante, Spain ...... 3 1.2.4 Orebro¨ University, Sweden ...... 4

2 Architecture 5 2.1 Introduction ...... 5 2.2 ThinkingCap Architecture ...... 6

3 Locomotion 9 3.1 CMD: Commander Module ...... 9 3.2 Walk...... 10 3.2.1 Implementation ...... 10 3.3 Kicks ...... 12 3.3.1 Implementation ...... 16 3.4 Optimisation of Walking ...... 18 3.4.1 The Optimisation Problem ...... 19 3.4.2 Learning algorithm ...... 24 3.4.3 Implementation ...... 28

4 Perception 31 4.1 PAM: Perceptual Anchoring Module ...... 31 4.1.1 Standard vision pipeline ...... 31 4.1.2 Experimental vision pipeline ...... 32 4.2 Color Segmentation ...... 33 4.2.1 Seed Region Growing ...... 34 4.2.2 Experiments ...... 35 4.3 Corner Detection and Classiﬁcation ...... 37 4.4 Active Perception ...... 42

i TeamChaos Documentation Contents

4.4.1 Perception-based Behaviour ...... 43 4.4.2 Active Perceptual Anchoring ...... 45 4.4.3 Implementation ...... 46 4.4.4 Experiments ...... 48

5 Self-Localization 51 5.1 Uncertainty Representation ...... 52 5.1.1 Fuzzy locations ...... 52 5.1.2 Representing the robot’s pose ...... 52 5.1.3 Representing the observations ...... 53 5.2 Fuzzy Self-Localization ...... 54 5.3 Experimental results ...... 57

6 Information Sharing 59 6.1 TCM: Team Communication Management ...... 59 6.1.1 Arquitecture ...... 60 6.1.2 Communicating with TCM ...... 64 6.1.3 Implementation ...... 67 6.2 Sharing Global Data ...... 72 6.3 Ball Fusion ...... 73

7 Behaviours 77 7.1 Low-level Behaviours ...... 77 7.1.1 Basic Behaviours ...... 77 7.1.2 Fuzzy Arbitration ...... 80 7.1.3 The Behaviour Hierarchy ...... 81 7.2 High-level Behaviours ...... 82 7.2.1 Hierarchical Finite State Machines ...... 83 7.3 Team Coordination ...... 83 7.3.1 Ball Booking ...... 84 7.3.2 Implementation ...... 84 7.4 The Players ...... 87 7.4.1 GoalKeeper ...... 87 7.4.2 Soccer Player ...... 90

A ChaosManager Tools 95 A.1 Introduction ...... 95 A.2 Vision ...... 95 A.2.1 Color Calibration ...... 96 A.2.2 Camera Conﬁguration ...... 97 A.2.3 General Conﬁguration ...... 98 A.3 Kicks ...... 99

ii TeamChaos Documentation Contents

A.4 Walk...... 100 A.5 MemoryStick Manager ...... 102 A.6 Game Monitor ...... 103 A.7 Game Controller ...... 104 A.7.1 Implementation ...... 105

B HFSM Builder 109 B.1 Using the Application ...... 109 B.1.1 Buttons and Menus ...... 110 B.1.2 Code Generation ...... 112 B.2 Building a Sample Automata ...... 113 B.3 Implementation ...... 114 B.3.1 File Format ...... 114 B.3.2 Class Diagrams ...... 115

iii TeamChaos Documentation Contents

iv TeamChaos Documentation Introduction

Chapter 1

Introduction

1.1 History

Team Chaos is a cooperative effort which involves universities from different countries (currently Sweden and Spain). Team Chaos is a follow-up of Team Sweden, which was created in 1999 and has participated in the 4-legged league of RoboCup ever since. The distributed nature of the team has made the project organization demanding but has resulted in a rewarding scientific and human cooperation experience. The scientific results of the team during these years have been very fruitful, although the competition results have not matched those, in particular in the the last two years. Because of the complexity of the software system, the code for the 2003 competition got very messy and included an important number of bugs. In addition, it did not include tools for helping in the configuration or calibration (except for a very simple tool for color segmentation). For the 2004 competition the team decided to go through a major code debugging and rewriting process. This included a number of configuration and calibration tools. Unfortunately the effort was not finished and ready for the 2004 competition. A major problem for the 2003 and 2004 was the lack of appropriate funding, including travelling. This, among other obvious problems, did not allow us to assist to local competitions, which definitively is very important for getting the system up and running for the competition. In December 2004 an important fact resulted in a major reshape of the team: the Spanish Ministry of Education and Science granted three Spanish universities in order to work in the RoboCup domain for three years. Thus, funds were available not only for the RoboCup but also for local competitions. The Team Chaos 2005 consisted on the following members:

1 TeamChaos Documentation Introduction

University Country Coordinator University of Murcia Spain Humberto Mart´ınezBarberá [email protected] University of Alicante Spain Miguel Angel Cazorla Quevedo [email protected] Rey Juan Carlos University Spain Vicente MatellánOlivera [email protected] Orebro¨ University Sweden Alessandro Saffiotti asaffi[email protected]

We had two main requirements in mind when we joined RoboCup. First, we wanted our entry to effectively address the challenges involved in managing uncertainty in the domain, where perception and execution are affected by errors and imprecision. Second, we wanted our entry to illustrate our research in autonomous robotics, and incorporate general techniques that could be reused on different robots and in different domains. While the first requirement could have been met by writing ad hoc competition software, the second one led us to develop principled solutions that drew upon our current research in robotics, and pushed it further ahead.

1.2 Team Members

1.2.1 University of Murcia, Spain

The Laboratorio de Robótica Móvil (Mobile Robotics Lab) is part of the Intelligent Sys- tems Group, University of Murcia, Spain. The lab is headed by Dr. Humberto Mart´ınez and currently counts 3 docents, 4 PhD students and several undergraduate students. There are three different research activities in the group: fundamental mobile robotics research, industrial mobile robots, and field robots. In the first case, the members of the group work in control architectures for multi-robot systems, navigation techniques (mapping and localisation), and sensor fusion. These techniques are applied in both standard indoor mobile robots and in the RoboCup domain (Sony Four-Legged League), in which the group actively participates in the competition. In the second case, the members of the group work in integrating basic research techniques into industrial AGVs in order to achieve real autonomy. The developed system (the group has designed and built both the hardware and software), called iFork, has been deployed in an agricultural company and is currently in use. In the third case, the group has been working in the development of different autonomous outdoor vehicles: an autonomous platoon of real cars, an autonomous airplane (UAV) for search and rescue operations, and an autonomous boat (ASC) for autonomous bathymetry and autonomous sampling.

2 TeamChaos Documentation Introduction

1.2.2 Rey Juan Carlos University, Spain

Rey Juan Carlos University is the youngest public university in the Madrid autonomous region. Founded in 1997 currently counts about 17,000 students in four different campuses around Madrid. The Laboratorio de Robótica (Robotics Lab) was established in 2001 and it is part of the Systems and Communications Group, that belongs to the Computer Science Depart- ment. The lab is headed by Dr. Vicente Matellán and currently counts 3 full-time teaching people, 4 PhD students, and several undergraduate students. The group main research interest is how to create autonomous mobile robots. This is a wide area with many challenging issues involved: environment estimation through sensor readings, robust control, action selection, etc. The group is focused in perception and control architectures, both for single robots or groups of robots. These architectures will let an autonomous robots exhibit intelligent behaviors, reacting to different stimuli and accomplishing their goals in a dynamic indoor environments. Another related research issue the group is involved is in the integration of vision into the robot architectures. This involves works in attention, abductive perception, etc. We work both with wheeled and legged robots. In both cases the group is primarily a robotics software group, which means that we try to use standard robotic platforms (currently Pioneer from ActivMedia and Aibo from Sony). We are also a libre software group, meaning that we support open source approaches to robotics.

1.2.3 University of Alicante, Spain

People from University of Alicante are integrated in the Robot Vision Group: RVG at the Department of Computer Science and Artificial Intelligence. The group was formed in 2001, but almost all of their members have more than 10 years of research experience. We are 5 PhD docents, 5 docents and 3 PhD students, only 3 of them working in TeamChaos. The main research interest is focused in robotics and computer vision. On the one hand, we are interested in how to exploit vision to perform tasks in mobile robots. The relative orientation of the robot may be computed through the analysis of geometric structures. Obstacle detection and avoidance may rely on 3D information estimated with stereo vision. Stereo vision is also key for building 3D maps which may be sampled by particle-filters algorithms that finally localize the robot in the environment. The effective- ness of such an approach increases significantly with the aid of visual appearance which in turn contributes to identify discriminant places or landmarks. We are interested in estimating these landmarks, incorporate them in topological maps, and inferring optimal exploration behaviors through reinforcement learning. Our previous expertise in control architectures and planning will contribute to successfully embedding visual modules in robot architectures. On the other hand, we are interested in implementing of effective and efficient computer- vision algorithms for: feature extraction and grouping (obtain a geometric structure from grouping junctions through connecting edges), clustering and segmentation (automatic

3 TeamChaos Documentation Introduction

selection of the most eﬀective set of ﬁlters for texture segmentation), recognition (either exploiting visual appearance, like PCA and ICA approaches, 3D information with match- ing strategies, or shape information through deformable templates), stereo and motion estimation (dense algorithms for disparity estimation, motion registration and tracking).

1.2.4 Orebro¨ University, Sweden

Orebro¨ University is one of the newest universities in Sweden, counting about 12,000 students. The Center for Applied Autonomous Sensor System (AASS) was created in 1998, and has seen a rapid growth, counting today about 25 members (10 PhD students, 10 senior researchers, and 5 visiting researchers). The mission of AASS is to explore the applicability of new paradigms in the analysis and design of autonomous sensor systems. These are mobile and immobile systems, which employ a vast array of sensors in order to analyse and influence a dynamic and uncertain world. These systems must operate in real time under both expected and unexpected situations, and with only a limited amount of human intervention. Since the world that these systems inhabit is complex, these systems need to employ advanced techniques for data processing, including embedded artificial intelligence, intelligent control, and neuromorphic systems. These are represented by the three interacting laboratories that compose the AASS Center: Sensor and Measurement technology, Intelligent Control, and Mobile robotics The Mobile Robotics lab is devoted to the development of techniques for autonomous operation of mobile robots in natural environments. By natural we mean real world environments that have not been specifically modified to accommodate the robots. This lab is headed by Dr. Alessandro Saffiotti, and currently counts 10 people (4 PhD students, 3 senior researchers, and 2 visiting researchers).

4 TeamChaos Documentation Architecture

Chapter 2

Architecture

2.1 Introduction

The problem of creating a robotic football team is a very difficult and challenging problem. There are several fields involved (low level locomotion, perception, location, behaviour development, communications, etc.), which should be developed for building a fully func- tional team. In addition debugging and monitoring tools are also needed. In practical terms, this means that the software project can be very large. In fact, if we execute the command sloccount[51] to our current robot’s code (that is, excluding the off-board tools), we get:

[...] Total Physical Source Lines of Code (SLOC) = 60,987 [...]

There are more than 60.000 lines of code. This fact shows that if we don’t organise and structure the project, it could be very difficult to manage. We use the Eclipse IDE for programming and debugging and CVS for sharing code and documentation. TeamChaos development is organised in three Eclipse and CVS projects: TeamChaos, ChaosDocs, and ChaosManager. TeamChaos contains all code related to the robot, Chaos- Manager is a suite of tools for calibrating, preparing memory sticks and monitoring different aspects of the robots and the games, and ChaosDocs contains all the available documentation, including team reports, team description papers, and RoboCup applications. The whole robot code is organised as Open-R [48] objects. Each object is mono-thread and the use of an unique object could be very difficult to develop and debug. Therefore, we have decomposed our code in three Open-R objects: ORRobot, ORTcm and ORGCtrl. ORRobot is the main object, which implements the full cognitive architecture and all the robotics dependent tasks. ORTcm is the communication manager and must fulfil tasks for sending and receiving data. ORGCtrl is an object entrusted to manage all instructions sent by the GameController.

5 TeamChaos Documentation Architecture

Communication is an important aspect because we can use external tools for achiev- ing laborious tasks easily. Using the ChaosManager we receive images from the robots and we can reﬁne the camera parameters and reconﬁgure the robot’s camera while the robot is running. We can teleoperate the robot for testing kicks or locomotion using communication between dogs and ChaosManager.

2.2 ThinkingCap Architecture

Each robot uses the layered architecture shown in Figure. 2.1. This is a variant of the ThinkingCap architecture, which is a framework for building autonomous robots jointly developed by Orebro¨ University and the University of Murcia [30]. We outline below the main elements of this architecture:

Other Other Robot Robot

Team Communication Communication Layer Module Messages

Local State Global State Hierarchical Global Finite Map Higher Layer State Machine

Local State Behaviours Perceptual Hierarchical Anchoring Behaviour Middle Layer Module Module

Sensor Motor Commander Lower Layer Data Commands

Figure 2.1: The variant of the ThinkingCap architecture.

• The lower layer (commander module, or CMD) provides an abstract interface to the sensori-motor functionalities of the robot. The CMD accepts abstract commands from the upper layer, and implements them in terms of actual motion of the robot eﬀectors. In particular, CMD receives set-points for the desired displacement velocity < vx, vy, vθ >, where vx, vy are the forward and lateral velocities and vθ is the angular velocity, and translates them to an appropriate walking style by controlling the individual leg joints.

• The middle layer maintains a consistent representation of the space around the robot (Perceptual Anchoring Module, or PAM), and implements a set of robust tactical behaviours (Hierarchical Behaviour Module, or HBM). The PAM acts as a short- term memory of the location of the objects around the robot: at every moment, the PAM contains an estimate of the position of these objects based on a combination of current and past observations with self-motion information. For reference, objects are named Ball, Net1 (own net), Net2 (opponent net), and LM1-LM6 (the six possible landmarks, although only four of them are currently used). The PAM is also in charge of camera control, by selecting the ﬁxation point according to the

6 TeamChaos Documentation Architecture

current perceptual needs [39]. The HBM realizes a set of navigation and ball control behaviours, and it is described in greater detail in the following sections.

• The higher layer maintains a global map of the ﬁeld (GM) and makes real-time strategic decisions based on the current situation (situation assessment and role selection is performed in the HFSM, or Hierarchical Finite State Machine). Self- localization in the GM is based on fuzzy logic [12] [22]. The HFSM implements a behaviour selection scheme based on ﬁnite state machines [25].

• Radio communication is used to exchange position and coordination information with other robots (via the TCM, or Team Communication Module).

This architecture provides effective modularisation as well as clean interfaces, making it easy to develop different parts of it. Furthermore, its distributed implementation allows the execution of each module in a computer or robot indifferently. For instance, the low level modules can be executed onboard a robot and the high level modules can be executed offboard, where some debugging tools are available. However, a distributed implementation generates serious synchronisation problems. This causes delays in decisions and robots cannot react fast enough to dynamic changes in the environment. For this reason we have favoured the implementation of a mixed mode architecture: at compilation time it is decided whether it will be a distributed version (each module is a thread, Figure 2.1) or a monolithic one (the whole architecture is a thread and the communication module is another, Figure. 2.2).

Other Robot COMMUNICATION IMPLENTATION Team Communication Module

Other Messages Robot CONTROL IMPLENTATION Local State Global State Hierarchical Global Finite Map State Machine

Local State Behaviours Perceptual Hierarchical Anchoring Behaviour Module Module

Motor Sensor Commander Data Commands

Figure 2.2: Implementation of the ThinkingCap architecture.

7 TeamChaos Documentation Architecture

8 TeamChaos Documentation Locomotion

Chapter 3

Locomotion

3.1 CMD: Commander Module

The Commander Module (CMD) is an interface to physical functions of the Aibo robot. Each robot model (ERS-210 and ERS-7) has its own particular feaures, so the way CMD interacts with the robot hardware depending on the robot model. Even though the new ERS-7 model offers a better performance (faster processor, more memory, etc), we still have an important number of ERS-210 in our labs, and thus we decided it was worth the extra effort to make them work. They are currently used for educational purposes. Basically the programs are quite compatible except for the PAM, and specially the CMD. The main differences between the two models are in the initialization part of the code, where the primitives for accessing the joints have changed and the walking algorithms, which take into account the physical aspects of the robot such as size of the limbs. We do not want to have two versions of the CMD code since it makes it very difficult two manage when doing any change or improvement and that’s the reason we focus on getting a generic code for both models. The most important feature of our CMD implementation is that this module achieve and independent way for interacting with different Aibo robot models. So an abstract architecture is implemented in this module to control robot hardware access. The CMD module implementation is divided into nine submodules, so different CMD aspects are treated in an independent submodule. Now, a brief explanation about CMD distribution is included.

Ear: treats the robot ears movement.

Gravity: controls the ground relative position of the robot, so when the robot is fallen on the carpet launch a kick for standing it up.

Kick: contains the needed structures for kicks, and controls robot kicks movements.

LedControl: manage robot leds, including face leds.

9 TeamChaos Documentation Locomotion

Look: contains the needed structures for robot head scans, and controls robot scans movements. SensorObserver: controls robot sensors (switches, accelerometers, etc). This submodule is in charge of generating proper events. StateMachine: manages the robot machine state (walking, kicking, etc) depending on the robot events. Tail: treats the robot tail movement. Walk: implements the walking style algorithm. It generates the step sequence needed to move robot legs, according the implemented walking style algorithm (based on the UNSW trapezoidal walk). include: contains common header ﬁles. Cmd: main class of the module, not include the Open-R intermodules communication. ORCmd: main class of the module, including the Open-R intermodules communication.

Fig. 3.1 and 3.2 show class diagrams of the most significant classes of the CMD module, specially those that contribute to the abstract aspect of this module. Each of this classes structures starts from an abstract class which indicates common aspects of the structure, while specific aspects are treated in the corresponding subclass of the structure. The three most significant operations of the CMD are: Sensor data reception from robot (Fig. 3.3), Head command execution demanded by PAM module (Fig. 3.4) and Velocity command execution demanded by HBM module (Fig. 3.5).

3.2 Walk

For the 2004 competition our walking styles was based on the code for parametric walk created by the University of New South Wales (UNSW) [20]. We are currently using the walk created by the Universit´ede Versailles [24] as well.

3.2.1 Implementation

The ﬁles for the Walk class are located into the Cmd directory:

/src/source/Cmd/Walk/

Following is the directory structure of the Walk class (Figure 3.6). The organization of the classes is as follows: Cmd creates a Walk object1. The Walk object creates a Locomotion object and calls its functions. See Figure 3.7. 1The word object refers to a ”C++ object”. Otherwise we would explicitly say ”Open-R object”.

10 TeamChaos Documentation Locomotion

Figure 3.1: Class diagrams. (a) LedControl (b) SensorObserver

Similarly, the Locomotion object creates and uses a Trajectory Plannig object, which is completely transparent to the Cmd and Walk objects. See Figure 3.8. The initialization of the objects is organized as shown on Figure 3.9. It is important to keep the initialization divided into Cmd::init() and Cmd::start(). These functions are executed in the appropriate order by an Open-R object. The Open-R object can be the ORRobot, when compiling for the monolithic version of the TeamChaos system, or the ORCmd, when compiling for the distributed version. The waking up of the robot happens during the following calls to the MoveJoint() function (Figure 3.10), as the Open-R object receives the messages for ReadyLegs. The requests to Walk operate as represented in the Figure 3.10. Finally, on the Figure 3.11 is represented another frequent scenario. Real speed is obtained form the Walk module because it differs from the speeds which the Cmd module stores. For example, when asking the robot to stop, in the first instant the Cmd velocity values are: vlin = 0, vlat = 0, and vrot = 0, while the robot is just beginning to decelerate. The real values have to be obtained from the Trajectory Planning. Some primitives are used by different objects. The responsibility for opening them is divided among these objects as follows:

• The SensorObserver opens all the 36 primitives related with sensors. These are listed in the ﬁle: src/source/Cmd/SensorObserver/SensorObserver7cfg.h

11 TeamChaos Documentation Locomotion

Figure 3.2: Class diagrams. (a) PWalk (b) Look

• The Walk object just opens the 12 primitives related to the eﬀectors of the legs, listed in the ﬁle: src/source/Cmd/Walk/Locomotion/ers7.h The gains associated to the joints are also set by Locomotion. On the other hand, legs’ sensors are already opened by SensorObserver.

• The rest of the eﬀector primitives are opened by other objects, i.e. Mouth, Ear, Look. These objects also set gains, when necessary.

The OPENR::SetMotorPower(opowerON) call is made by the Cmd::init() function. There are still some pending tasks, the most important ones being:

• Task: Integrate for Position Control use. Priority: middle. Diﬃculty: middle. Dependencies: Behaviours implementation. Antecedents: By now the system behaviours use Velocity Control. More sophisti- cated behaviours and control could be developed by using the Position Control.

• Task: Make the initial waking up sequence smooth. Priority: low. Diﬃculty: easy. Dependencies: none. Antecedents: During the Locomotion integration, some piece of code was posibly deleted or changed, and the initial waking was not smooth anymore.

3.3 Kicks

Kicking is a fundamental ability for soccer playing. The robot will use it to score a goal, to pass to a teammate, and in our architecture even to block an opponent shot. Basically, we deﬁne a kick as a movement sequence that the robot performs when the behavior module (HBM) activates it, and when a given set of preconditions are satisﬁed. The working schema is as follows:

1. The behavior module (HBM) sends a set of kick, selected among all the available ones, to the motion module (CMD).

12 TeamChaos Documentation Locomotion

Figure 3.3: Sensor data reception sequence diagram.

2. The CMD module activates these kicks, but it does not performs any of them till the preconditions are satisﬁed.

3. Each kick is conﬁgured with a set of preconditions regarding the relative position of the ball to the robot. The CMD module checks the preconditions for each kick. If the position of the robot and the ball satisfyies the kick conditions, the kick is performed.

Figure 3.12 shows the precondition (in blue) associated to a particular kick. If this kick is activated by HBM and the ball is in the blue area, this kick will be performed. We have classiﬁed our kicks into two categories:

• Open loop. This kind of kick is used for short kicks and they are executed in open loop mode. These kicks cannot be interrupted and they do not use any sensory information. The open-looped kicks we have currently developed are:

– LeftLegKick The ball is kicked with the left leg. – RightLegKick The ball is kicked with the right leg. – TwoLegKick The ball is kicked with both legs. – BlockKick This kick is used for blocking a ball. The robot extends both legs enlarging the blocking area.

13 TeamChaos Documentation Locomotion

Figure 3.4: Head command execution sequence diagram, invoked by PAM.

Figure 3.5: Velocity command execution sequence diagram, invoked by HBM.

– LeftHeadKick The robot uses its head for kicking the ball at its left direction. – RightHeadKick The robot uses its head for kicking the ball at its right direction. – FrontHeadKick The robot kicks the ball with its head for push the ball at its front. This kicks is a very eﬀective one. – GetUp This kick is performed when the robot detects it has fallen down.

• Closed loop In this kind of kicks, the robot is getting information from its sensors while it is performing the kick. If a condition is not satisﬁed (i.e. the ball is not close enough) during the execution, the kick is aborted. These kicks can be divided into several phases. These phases starts and stops depending on the sensory information. Currently we have just one of these kicks.

– GrabAndTurn This kick makes the robot grabs the ball and turns. First, the robot orientates to the target with the ball grabbed. Then, the robot pushes

14 TeamChaos Documentation Locomotion

Figure 3.6: Directory structure of the Walk class.

the ball. If the robot lost the ball while it turns, the kick is aborted. This situation is detected by its infrared sensor situated between both legs.

The kicks module is in CMD object. It is used to execute kicks from a kick list. Each kick is a set of columns, each column stores the angular position for each joint and each row is a movement in the sequence of the complete kick movement, as shown in Fig. 3.13. The kick execution ends when the movement sequence (the rows) is completed. Furthermore each kick has a name (a string), ID (integer), activation zones (x min, x max, y min, y max) and odometry values. An special case of kick is the parametric one, which is a kick that accepts a parameter. This parameter corresponds to the degrees that the robot must spin when executing a left turn kick or right turn kick while simultaneously grabbing the ball. Parametric kicks ﬁnish when the turn is completed or when the robot looses the ball.

15 TeamChaos Documentation

Locomotion

d l

pee aca

v u

lklk

kikik

bj bj

OS *

t t

s ec ec

u u

lii

l i

ron

ddli

ogmoen

lii

b b

OSO*

sensorsererensorserer

v v

l i

lkk

oooo

ron

d i

oosesn

l i i

t t

ocomo onocomo on

d d

oorrorsomer

lk(d bj i)

S*OS C **

WFM

t t

am ecn

: : :

dil h

ooecar

, ,

lk()

kikd d

t t

coomeromer

y y

lii(i i i)

***

t t t t t

se eocesnnn

: : :

earsars

, ,

llii() d

t t t

ge caeocespee

il il

* +

b ()

R 1

t t

opoo

h h

t t

mo o

u u

b ()

t t

aroo

i i

t t

grara

vyvy

d i()bl

enego onoo

d( idi bl)

C * *

mnsgnenoo

: :

lki(i i i )

ttt

angnnn

: : :

d()

, ,

d ( )

S OS *

U FVD

t tt

paeensorsensorensorrameecoraa

ii()

ii i()

t t

nocomo on

()

sar

i()

tt t

sar ocomo on

bj ( bj [] bj )

S OS *

t t t t

se ecss ec ec

u u u

i i( d)

A VTA

t t

omcc ro onpee

lii(i i i)

***

t t t t t

se eocesnnn

: : :

, ,

h(i i )bl

ttt t

scasn oo

: :

llii() d

t t t

ge caeocespee

d ( )

S OS *

U FVD

t tt

paeensorsensorrameecoraa

lk()lk

ge aa

d()

eaegs

dk()

eaec

d ( idi )d

O O

t t t t

geomer nsgnen omer

y u y

: :

li( idi)d

M L

t t t t

o ocaonnsgnenomer ge

u y

: :

d( d)

C *

E V V

ecrommancommanr

l i

ocomoon

sensorserer

d( d)

C *

E H H

+ 0

eccommancommanc

+ 0

b i

i() d

ensorserer ocomoon

egoon en

S L

() iid

t t

arams nomer

Figure 3.7: Class Diagram. The Walk object acts as an interface between the Cmd class and the Locomotion class. (Note: many member functions and variables which are not relevant to this documentation, have been omitted, as well as the rest of the Cmd relations).

3.3.1 Implementation

This module is composed of the ﬁles kick.h, kick.c, kickparser.c and other that include some deﬁnitions like cmdcfg.h. The kick module uses the JointData structure in cmdcfg.h. In this structure are stored the parameters that module needs to execute a kick. The structure is the following: struct JointData { int step; // current step in the move sequence int sstep; // substep within step int dir; // direction of motion, mainly for scans int status; // tracking state, for lookto bool hwait; // head joints not ready yet bool lwait; // leg joints not ready yet bool usehead; // current motion is controlling head int *data; // array of joint data double ddata[nJoints]; // current data row double last[nJoints]; // previous data row double sent[nJoints]; // last values sent bool ParametricKick; // current kick is a parametric kick int iter; // repetition number for parametric kick int count; // repetition counter for parametric kick bool exit; // exit of exec kick

16 TeamChaos Documentation

Locomotion

ocomoon

il h

secar

j j li

TT P

raraecor annng

i l l d

mar egangesrea

i l l d

sensormar egangesrea

_ __

i l l d

ttt

sarmar egangescomman

_ __

i l l d

crrenmar egangescomman

u x

_ __

i l l dfil

mar egangescommanna

_ __

l l hl i

eganges amar

iii iii

IDPID

t t

prmerme

v v

bj i bj

OS*

t t t

secoeons ec

uv u

ji bl

PID

ttt t

sarse on oo

___

ji bl

ttt t

sarsep ons oo

dbl d

commanoo sen

_ _

hdbl d

tt t

aeenoo secomman

____ _

d bl

osar oo

d bl

seqence sen oo

f bl d

seqence o oo en

_ _

ii i

t t

nseq n

_ _

tt t

cpneseq n

fff i

nmerconsn

d d

OC *

V VD

cmec ommanecoraa

d i

VIDMRID

cmecemoregon

dl i dl

OC 2*

V J V

commanae onommanae

u u

i i

regon egon

i bl

S 20

I ER 1

serson oo

__ _

i bl

I ER 7

serson oo

__ _

bji j i li

t t t

non raecor n annng

i(bj bj ddli)

S OS* *

L L M

t t t t

ocomoonseegs ecogoen

: :

i()

ocomoon

ii i()

t t

nocomoon

i()

tt t

sarocomoon

i ( )

SOS *

R FVD

eceeensorensorrameecoraa

d f (fih h j li)i

* *

TT P

t tt t t

rea parameerso roc paramcar raecorannngn

: : :

_ _ _ _ _

bf[]h) di i li dli(fil

t t

car esncrsorposnne reanee

u u

: : :

, ,

_ _ _ _

il dl()

IVAA

naesnnges

j()j li

TT P

t t

geraraecorannng

i()

oeon

i()bl

sarngoo

ff()

eper

i i f()

eonansae

i ()

eansoero

l i()

N MJ

orma oeon

ii()

nons

di ()

enonaa

di ifi( i)

S S

tt t

enonaapeccmar

d l()

rea sensoraes

iiil ii l()

t t t t

na geposoncrrenaes

uvu

__ _ _

fil ii f h ()

t t t

na geposono smoo seqence

__ _ _ _

fil ii f idiidl h f d(h bl)

t t t t

na geposono na smoo or commansoererooo seqence

vu z u

______

ii f h ()

ttt t

geneposono smoo seqence

x u

__ _ _ _

( d d hd bl)

t t tt

geseqencesesen cm aeen oo

______

id f ()

geo sensors

__ _

hdb(l i)

tt t

eaonaen process

uvu

bkb(l i)

tt t

aconaen process

uvu

i ( )

S OS&

PP P

t tt t tt

rnoeraspsconsoeras

wu wu

Figure 3.8: Class Diagram. The Locomotion class.

};

The parameter ParameterKick indicates if the kick is a parametric kick. The parameter iter stores the iteration number for completing the turn. Each iteration spins the robot 15 degrees so the total number of iteration is the division of the parameter between 15. In the parameter count appears the number of iteration to carry out. And the parameter exit indicates the end of the kick. This parameter is false by default and is changed to true when the parameter count is higher than iter or the robot loses the ball. The kicks can be called from behaviours or from the graphic interface (Kick Master)in teleoperation mode (to debug or improve the kicks). When a kick is used by behaviors, these must be used the call RunBehaviorAndKick() where the name of the kick is a parameter. The call to execute a kick from graphic interface is selecting the kick and pushing the button Run Kick a message is send to the robot as indicated in ﬁgure A.6. The message includes the selected kick ID. The call to RunBehaviorAndKick() causes a call to AddKickRequest(). The CMD module updates the active kick list with the method receiveKick(). Then the CMD,

17 TeamChaos Documentation

Locomotion

bl) d(idi

** C

oo mnsgnen

: : :

ii()

) i bj lk(d

* * S*OS C

WFM

t t

n ec am

: : : :

, ,

ocomoon

) ddli i(bj bj

* * SOS

M L L

t tt t

ogoen ocomoonseegsec

: : :

) i i lii(i

* * *

t t ttt

n n seeocesn

: : : :

, ,

ii i()

nocomoon

()

sar

b()

opoo

i()

ttt

on sarocomo :

Figure 3.9: Sequence Diagram. Creation and Initialization of the objects related to Walk. at each iteration check which kicks in the active list are applicable using the method CheckKicks(). If one of the active kicks is applicable, then the method execKick() is called. The process is illustrated in Fig. 3.14. Each time a kick is executed, the init() method is called. With this method some parameters are initialized, for example, all arrays storing the joint movement are set to zero, the parameter which indicates if the kick is a parametric kick is set to false, and so on.

3.4 Optimisation of Walking

Two important problems are related with the walking style: (i) finding a good set of parameters that provide a stable walk and high speed; and (ii) finding a good odometry function that allows an accurate localization [7, 8]. We identify two different optimization problems. One is the maximisation of the robot’s speed. As surfaces’ characteristics vary from one place to another, it would be very difficult to find an analytical solution. The other problem is improving the correspondence between commanded and achieved speed. If the difference is high, not only the robot will

18 TeamChaos Documentation

Locomotion

b d lk i j li

O C

RR WL T P

t t t

oo m a ocomoon raecorannng

: : : : :

PENR

d( d )

SO&

RLRE

t t tt

eaeegsconseaeneen

y yvv

: :

d()

eaegs

d i()bl

en egoonoo

: :

i()

oeon

d(d )

C *

EV V

ecrommancomman r

: :

lki(iii)

ttt

angnnn

: :::

i) li l(lii db

t tttt

spacepon eocconroeocesenereser

vyv yu

: :

______

i()

oeon

v :

Figure 3.10: Sequence Diagram. Requests to Walk.

d b lk i j li

O C

RR W L T P

t t t

m oo a ocomoon raecor annng

: : : : :

d llii()

t tt

ge pee caeoces

: :

l(i liiill lfill ldl l) l

OOO O O&

R R R R R

t tt t t

ge crrenemeeanaeeanaeea eea ea

uv v v v

: : : : : :

, , , ,

______

Figure 3.11: Sequence Diagram. Other requests to Walk. exhibit an strange behavior but also it will fool the localisation subsystem. The approach we propose is a parameters optimization method with an empirical evaluation function which evaluates the real speed of the robot. The parametric space has eight dimensions and the evaluation function is quite time consuming, which precludes the use of brute force space search. We apply a simulated annealing technique for learning an appropriate set of parameters and use a straight line with marks placed at fixed distances. The idea is building a ruler that can be deployed in different arenas. The real speed is measured by counting the ruler markers using the vision system. These techniques will allow the improvement of our locomotion module, which was one the most unreliable module of our system, and allow for a better positioning in the 2005 large field.

3.4.1 The Optimisation Problem

The walking style has a set of parameters (see Figures 3.15 and 3.16) which have to be tuned in order to acheive a stable and fast walk. This depends on the surface where the robot will be walking. For example, if it is a carpet the robot should lift a little bit

19 TeamChaos Documentation Locomotion

Figure 3.12: Kick area condition

Figure 3.13: Paramaters for the definition of a kick more its legs before moving them forward. Thus dragging robot’s legs is avoided because friction would probably slow down the walk. There are two different optimization problems. One of them is the maximization of the robot’s speed. As surfaces’ characteristics vary from one place to another, it would be very difficult to find an analytical solution. The solution we propose is a parameters optimization problem with an empirical evaluation function which returns the speed of the robot. The parametric space has 8 dimensions. It is a relatively big space for this problem, provided that the evaluation function has a high time cost. The other problem is improving the accuracy of the relation between required and responded speed. In some cases only one set of parameters for the entire range of speeds is not enough. Speed is divided in various intervals, and a different set of parameters is assigned to each one of them. The optimization problem for these problems is the minimization to a concrete value of speed. For example, suppose we request a 15 cm/s speed to the robot, and at the same tame the goal of the minimization algorithm is be 15 cm/s, thus aiming to minimize the difference. The evaluation function of the optimization problem is the speed of the robot while walking straight forward. There are two possible measures:

• Time spent walking from one landmark to another, to calculate average speed.

• Instant speed, wich is much more unstable.

The robot has a camera directionable by its head, so the way chosen to measure distance (and therefore speed) is vision, using artiﬁcial landmarks on the ﬂoor. Marks are placed close to each other, so the robot can calculate its “almost” instant speed.

20 TeamChaos Documentation Locomotion

Figure 3.14: Diagram with method call during kick execution

The Training Process

There are three different walking modes: Forward walking, lateral walking, and turning. The most important for us is forward walking, which is the one taken into account for the learning process. The trajectory to follow is a straight line. Its length should be at least 2 m, but it can be as long as desired. When the robot reaches the end of the line it will turn back and follow the same path in the opposite direction, until the training has finished. The white line(Figure 3.17) has little white marks (distance marks) around it, placed in pairs at the same distance, about 20 cm (this value must be correctly configured in the training software). The width of the line and the marks can be about 1 cm but other sizes will also be recognized without additional configuration. The background color (the carpet) can be any other color (green in our case). The training software must be correctly configured with the number of distance-marks and the physical distance between them.

Internally, the robot’s trajectory is always guided by white color, no matter the shape of the segmented regions. Shape is signiﬁcant when recognizing the little white marks (distance marks). A large white blob is required, with the shape of a vertical line. Then a small blob near the line will be sought. It can be at the left or at the right of the line, or at both sides (recommended to ensure detection in case of a small deviation). The separation between the central line and the small blobs will be required to be between 3 mm and 3 cm, approximately.

21 TeamChaos Documentation Locomotion HRL HFL

LSR

LSRL/R LSL

LSFL/R sh_C_l/r

Figure 3.15: The Locomotion parameters.

Only one pair of distance-marks (as well as a single one) can appear on each frame taken from the camera, as robot’s head is tilted to look almost vertically at the ﬂoor. There must be at least two frames with no distance-marks detected on them, to look for a new mark on the next frames. When lateral walking is executed, the robot’s camera sees the line horizontally, instead of vertically. The same vision process is applied for detecting marks, changing the x and the y axis.

Once blobs are contabilized instant speed can be calculated. s vi = (3.1) ti − t(i−1)

The obtained values have a very high error because of the irregularities in the walking movements. That is the reason for considering instant speed the average speed between three distance-marks. v − v v0 = i−1 i (3.2) i 2 This measure is more reliable and gets closer to the average2 speed, as shown on Figure

2Average speed is considered to be the average of the last four instant speed measures.

22 TeamChaos Documentation Locomotion

Figure 3.16: Horizontal scheme of some Locomotion parameters.

Figure 3.17: Landmarks should be similar to these ones. The lenght of the line is arbitrary, though a long one is recommended to avoid delays in the process (2 meters minimum).

3.19. Instant speed is very distorted when landmarks are lost. A ﬁlter is applied, which only returns measures that do not diﬀer too much from the average speed. Otherwise instant speed is said to be unknown. ( v0 1 v < v0 < 2v f (v0) = 2 avg avg (3.3) speed −1 otherwise

Finally, after speed is ﬁltered and averaged, the learning algorithm receives a speed estimation after every three correct measures.

The Function to Minimize

• When the goal is to achieve a maximum speed, the learning algorithm uses the equation (3.4). fmax(speed) = −speed (3.4)

23 TeamChaos Documentation Locomotion

Figure 3.18: Our point of view of the line and the distance marks. The width, height, area, and position of each blob have to be analyzed.

0.35

0.3

0.25

0.2 cm / sec 0.15

0.1

0.05

0 0 5 10 15 20 25 30 35 Number of distance−marks

Figure 3.19: Instant speed and average speed during 4 races along the line.

• When the goal is a concrete value of speed, the diﬀerence between goal and speed has to be minimized, so the equation (3.5) is:

2 fmin(speed) = (speed − goal) (3.5)

where goal is the speed to achieve.

3.4.2 Learning algorithm

Different techniques were applied to find a good set of PWalk parameters. Varying each parameter independently is the simplest one. Then a set of the best parameter values are selected. If some parameter enters in conflict with another one (causing unstability or even clashes between front and rear legs), its value is corrected manually. This method does not find the best solution because parameters are actually dependent. It could be used to find a good initial set of parameters for a hill-climbing algorithm.

Simulated Annealing [?] ﬁnds good sets of parameters after about 30 iterations, when temperature is still high and therefore these results are random. When temperature

24 TeamChaos Documentation Locomotion

descends, changes are smaller and the algorithm converges into some local minimum. It is not feasible to wait for more than a hundred iterations because between each iteration there are a few seconds of speed evaluation. The Simulated Annealing algorithm accepts every improvement of the evaluation function, and also accepts enworsments with some low probability:  f−f − old  ηT  e f > fold P (C) = (3.6)   1 f ≤ fold where P (C) is the probability to accept the change C of parameters, fold and f are the result of the evaluation function before C and after C. T is the temperature, the initial value of which is 2.0 and it decreases 10% each iteration (each iteration has 3 steps of parameters’ changes). The factor η is used to scale the evaluation function and its value is the average variation of f in each step of the algorithm. For example, for fmax (see Formula 3.4), η = 0.01, while for fmin (see Formula 3.5), η = 0.0001. The parameters pi are updated randomly, but depending on the temperature T : √ 4pi = µi T rand(−1, 1) (3.7) Here i ∈ [1, 8] represents each diﬀerent parameter out of 8. µ is used for scaling the changes according to the range of each parameter pi, and it is initialized with a value that allows each pi to be a 10% maximum of the parameter’s range. (Therefore, from the for- R(pi) mula 3.7 is deduced that µi = √ , where R(pi) is the range of values of each parameter 10 T0 and T0 is the initial temperature). The function rand(−1, 1) returns any random value in the interval (−1, +1).

Learning with multiple goals

When training multiple sets of parameters, an uniform variation of the speed must be achieved. To avoid discontinuities in the speed function, each parameter should be trained aiming a correct speed response both at the lower and upper limits. During the training process both speeds (upper and lower) are measured alternately, so each speed measure is a pair s = (slower, supper). Then the learning algorithm deals with the summatory of the two evaluation functions, respectively.

A constraint of the problem is the need of continuity between intervals. To make such a calibration we modify the learning algorithm in order to achieve simultaneously the real 0 0 speeds (vl, vu) for the upper and lower limit of the deﬁnite speed interval [vl, vu]. When the goal for the simulated annealing is a concrete value of speed, the diﬀerence between goal and speed has to be minimized. We have two goals so we alternate measures for the speeds vl and vu, being the function to minimize:

2 fmin(s1, s2) = ((s1 − vl) − (s2 − vu)) (3.8) where s1 and s2 are the speed measures, and vl and vu are the goals, as already explained.

25 TeamChaos Documentation Locomotion

Then, for n > 1 sets of parameters the calibration process would be as follows:

1. Find a parameters set by means of speed maximization. Thus the achieved maximum speed is vmax. Determine the values vn−1, vn−2 . . . v1 so that vmax > vn−1 > vn−2 > . . . > v1 > 0.

2. Optimize a parameters set Sn for the speed interval [vn−1, vmax]

3. Similarly, for each i, n > i ≥ 0 optimize a parameters set Sn for the speed interval [vi−1, vi]

Thus we try to approximate the speed responses of the lower limit of an interval Si and the upper of the next interval Si−1.

Multiple sets of parameters

When using only one set of parameters for the entire range of speeds, it could happen that for some low speeds, the speed response is different from the desired one, because of the characteristics of the surface and the movements of the legs. The solution is to train differents sets of parameters for different intervals of speed. The experiments showed that using a single set of parameters produces a discontinuity in the relation between required and obtained speed. On the other hand, when training differend sets of parameters, the accuracy could enworse. Our experiments with 1, 2, 3, and 6 sets of parameters showed that a good solution is to traing only two sets of parameters for the ranges of speed [0cm/s, 15cm/s] and [15cm/s, +∞]. (See Figures 3.20 and 3.21). For measuring the mean speed, mean error and variance, 10 different measures were taken for each speed.

Speed response with one set of parameters −4 x 10 Error of the measures with one set of parameters 6

4 0.25 Error (m/s)

2 0.2

0 0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.15 Requested Speed (m/s) −7 x 10 Variance of the speed measures with one set of parameters 7

6 Obtained Speed (m/s)

0.1 5

4 Variance 3 0.05

0 0 0 0.05 0.1 0.15 0.2 0.25 0.3 0.05 0.1 0.15 0.2 0.25 0.3 Speed (m/s) Requested Speed (m/s) Figure 3.20: Speed relation, error and variance of the error measured using a single set of parameters. There is a discontinuity near to the 15cm/s speed.

26 TeamChaos Documentation Locomotion

Speed response with two sets of parameters −4 x 10 Error of the measures with two sets of parameters 6

4 0.25 Error (m/s)

2 0.2

0 0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.15 Requested Speed (m/s) −6 x 10 Variance of the speed measures with two sets of parameters 1.2

1 Obtained Speed (m/s) 0.1 0.8

0.6 Variance 0.05 0.4

0.2

0 0 0 0.05 0.1 0.15 0.2 0.25 0.3 0.05 0.1 0.15 0.2 0.25 0.3 Speed (m/s) Requested Speed (m/s) Figure 3.21: Speed relation, error and variance of the error measured using 2 sets of parameters. There discontinuity has been overcome and the error and variance have improved.

Odometry calibration

The three different walking modes (figure 3.22)are forward walking, lateral walking, and turning. All of them are taken into account for the odometry calibration process. In other words, their speed error and variance are measured for different speeds.

Figure 3.22: The robot and the distance marks during the speed measuring process for: a) forward, b) lateral, and c) rotational walking.

Odometry calibration results must be placed in the ﬁle MS/CONF/CMD/ODO.INI, which has the following format:

#CMD Odometry initialization file #ODO_X = REQUESTED_SPEED_X OBTAINED_SPEED_X ERROR_X #13 de junio de 2005

ODO_COUNT=20

ODO_LIN_0 = 0.4000 0.320 0.00072

27 TeamChaos Documentation Locomotion

ODO_LIN_1 = 0.3800 0.315 0.0007 ODO_LIN_2 = 0.3600 0.310 0.00067 ... ODO_LIN_18= 0.0400 0.030 0.00000500000000 ODO_LIN_19= 0.0200 0.002 0.00000010000000

ODO_LAT_0 = 0.3000 0.230 0.00167830901393 ... ODO_LAT_19= 0.0150 0.003 0.00020000000000

ODO_ROT_0 = 60.000 75.00 2.3 ... ODO_ROT_19= 03.000 00.10 0.3

ODO CONUNT speciﬁes the number of discretization intervals of the entire speed range. Speeds are measured in m/s (lin and lat), and deg/s (rot).

3.4.3 Implementation

The learning algorithm and the evaluation function, as well as the control for the line following, are implemented in Java, inside the Chaos Manager. Segmentation, blobs, and walking style were already implemented in Open-R for the robot. The only Open-R code added to the already existing, is a patch for receiving blob requests and sending blobs information to the Chaos Manager.

For receiving blobs information from the robot, the same scheme as in image receiving has been followed:

1. AIBO receives an external message for Blobs request. 2. AIBO sends an external STRING message with blobs information.

For this purpose another type of External Message has been added to the TCM (Team Communication Module) object. Instead of sending a vector of floats, strings have been used to avoid unnecesary changes in the TCM. (This module will be soon reimplemented and improved). Only white blobs information is sent in the messages to the Chaos Man- ager because no more information is necessary for this experiment. On the other hand, updating PWalk parameters via UDP has also been added to the Open-R code in the robot. This is necessary for immediate changes of the parameters during the learning process. Selecting the colour to be segmented, has also been added to a configuration file into the memory stick, and also into the Chaos Manager tool. The reason is that Team Chaos

28 TeamChaos Documentation Locomotion

does not segment the white colour by default.

A comprobation for the battery current had to be added into the Cmd module. When the robot forced too much some joints, the operating system APERIOS launched an BATTERY OVER CURRENT exception which runs the shutdown routine. The code added helped the robot to automatically avoid this exception, as it monitorizes the battery current and slows down the robot’s speed when the current gets close to the dangerous threshold. Thus the joint forcing is indirectly penalized by the evaluation function (the robot’s speed) by artiﬁcially slowing down the speed. Anyway this kind of exception rarely happens on suitable surfaces.

The rest of the walk calibration code is a part of the Chaos Manager tool. The package and classes structures are represented on the Figure 3.23. There are the GUI, the evaluation function, including vision and measures, the training process, and the learning algorithms.

pwalk

training

TrainingPanel TrainingThread CanvasBlobs learning measures

LearningMethod BlobsBuffer SpeedMeasure

IndepParams OdometryMeasure SimulatedAnnealing

Maximize AdjustSpeedRelation

Figure 3.23: Package and class structure.

Figure 3.24: A subpanel of the host application, which monitorizes and controls the evaluation and learning process.

29 TeamChaos Documentation Locomotion

30 TeamChaos Documentation Perception

Chapter 4

Perception

4.1 PAM: Perceptual Anchoring Module

The locus of perception is the PAM, which acts as a short term memory of the location of the objects around the robot. At every moment, the PAM contains the best available estimates of the positions of these objects. Estimates are updated by a combination of three mechanisms: by perceptual anchoring, whenever the object is detected by vision; by odometry, whenever the robot moves; and by global information, whenever the robot re-localizes. Global information can incorporate information received from other robots (e.g. the ball position). The PAM also takes care of selective gaze control, by moving the camera according to the current perceptual needs, which are communicated by the HBM in the form of a degree of importance attached to each object in the environment. The PAM uses these degrees to guarantee that all currently needed objects are perceptually anchored as often as possible (see [39] for details). Object recognition in the PAM relies on three techniques: color segmentation based on a fast region-growing algorithm; model-based region fusion to combine color blobs into features; and knowledge-based ﬁlters to eliminate false positives. For instance, a yellow blob over a pink one are fused into a landmark; however, this landmark may be rejected if it is, for example, too high or too low relative to the ﬁeld.

4.1.1 Standard vision pipeline

Signiﬁcant improvements have been made to the PAM over the last two years. Previously, seeds for the growing algorithm were obtained by hardware color segmentation on YUV images, taken directly from the camera. Seeds are now chosen by performing software thresholding on the images, which are ﬁrst converted to HSV. This allows for a very portable and robust color segmentation, which works even in the face of changing lighting conditions.

31 TeamChaos Documentation Perception

Figure 4.1: Standard game-playing vision pipeline

The vision pipeline works as follows (see Fig. 4.1). When a new YUV image is captured, it is ﬁrst converted to HSV. Then it is segmented using software thresholding to obtain a labelled image whose pixels are used as seeds for the seed region growing method. Then, for each region type all the blobs are processed and merged when appropriate conditions apply (overlapped blobs, adjacent blobs, etc). The resulting blobs are then processed to verify constraints (size, number of pixels, aspect ratio, etc) in order to identify which type of objects they represent. Finally, those blobs that are classiﬁed as given objects are further processed to estimate the distance from the robot to the object. All the information is then aggregated into a robot centered data structure called Local Perceptual Space (LPS), which represents a local state of the robot. This data structure is used by the HBM to decide which action the robot should perform.

4.1.2 Experimental vision pipeline

In addition to the normal objects in the RoboCup domain, we also detect natural features, such as field lines and corners. Currently, we use corners comprised of the (white) field lines on the (green) carpet. Corners provide useful information, since they can be classified by their type, and they are relatively easy to track (given the small field of view of the camera). Natural feature detection is performed using three techniques: corner detection based on changes in the direction of the brightness gradient; color-based filtering for filtering out corners that aren’t on the carpet; and corner grouping for associating the detected corners with the natural features for which we are looking. While the corner based processing is still experimental (it is still a bit slow to be used for actual game playing), it is fully operational. It works as follows (see Fig. 4.2). In addition to the standard image processing, the Y channel of the captured YUV image is used to detect grey-level image corners. As we are only interested in field lines, the grey- level corners are filtered out based on the segmented HSV image. Only corners produced by white blobs are considered. Then, the resulting corners are classified depending on whether they are convex or concave. Then these corners are grouped and classified into different feature categories, corresponding to different field lines intersection types (C, T, etc). Finally, for each detected feature a distance to the robot is computed and the feature is projected into the LPS.

32 TeamChaos Documentation Perception

Figure 4.2: Corner-based experimental vision pipeline

4.2 Color Segmentation

Color segmentation is a fundamental part of object recognition in the RoboCup domain. The segmentation algorithm has to be robust, which means that it has to give the right results under nominal conditions and, without returning, give acceptable results under slightly different conditions, such as blurred images or different lighting. The main methods for color segmentation can be classified into the following approaches, sometimes used in combination [46].

• Threshold techniques rely on the assumption that pixels whose color value lies within a certain range belong to the same class or object [43]. Many teams in RoboCup use threshold techniques [21] [10] with slight modiﬁcations to increase its performance, e.g., for images that blur at object boundaries. Threshold techniques need a careful tuning of thresholds and are extremely sensitive to light conditions.

• Edge-based methods rely on the assumption that pixel values change rapidly at the edge between two regions [32]. Edge detectors such as Sobel, Canny and Susan are typically used in conjunction with some post-procedure in order to obtain the closed region boundaries. This tends to make these methods very computationally expensive.

• Region-based methods rely on the assumption that adjacent pixels in the same region have similar color value; the similarity depends on the selected homogeneity criterion, usually based on some threshold value. Seeded region growing (SRG) is a region-based technique that takes inspiration from watershed segmentation [49] and is controlled by choosing a (usually small) number of pixels, known as seeds [1]. Unfortunately, the automatic selection of the initial seeds is a rather diﬃcult task [26]. For instance, Fan and Yau [18] use color-edge extraction to obtain these seeds. By doing so, however, they inherit the problems of edge-based methods, including sensitivity to noise and to the blurring of edges, and high computational complexity.

In our current implementation we use a hybrid method for color segmentation that integrates the thresholding and the SRG methods. We use a threshold technique to generate

33 TeamChaos Documentation Perception

Figure 4.3: Problems of SRG: homogeneity criterion too strict (center) or too liberal (right) an initial set of seeds for each color of interest, and then use SRG to grow color regions from these seeds. Special provisions are included in our SRG to account for the fact that we can have many seeds inside the same region. The integration of the two methods allows us to use a conservative tuning for both of them, thus improving robustness: robustness with respect to changing lighting conditions is improved because we use a conservative thresholding, and robustness with respect to blurred edges is improved because we use a conservative homogeneity criterion in SRG. The technique, with minor variations, has been used in the 4-legged league since 2001.

4.2.1 Seed Region Growing

In a nutshell, our color segmentation algorithm works as follows. We fix a set of colors to be recognized. We use thresholds to produce a labelled image where only pixels that belong to one of these colors with high certainty are marked as such. Then, we use each labelled pixel as an initial seed for a region of that color. From each one of these seeds, we grow a blob by including adjacent pixels until we find a significant discontinuity. The so called homogeneity criterion embodies the notion of a significant discontinuity: in our case, this is a change in one of the Y, Cr or Cb values beyond a given threshold. The choice of the homogeneity criterion is critical in SRG techniques. If this criterion is too strict, the blob growing can stop prematurely because of local variations, e.g., as produced by shadows or noise. If it is too liberal, the blob can flood to an adjacent region though a weak edge, e.g., as caused by blurring. These two cases are illustrated in Fig. 4.3, where the single initial seed is indicated by a black dot (the ball outline is drawn for clarity). The thresholds used in the homogeneity criterion are 2 and 3, respectively. In our technique, we address this problem by using many seeds for each color region. When growing a blob from a given seed, we use a strict homogeneity criterion, thus reducing the risk of flooding. We then reconstruct the full region by merging all the local blobs that belong to the same region, that is, all the adjacent blobs grown from seeds of the same color. The following procedure constitutes the core of the segmentation algorithm. (N is the size of the image.) procedure FindBlobs (seeds) reset label[0:N], color[0:N], conn table blob id = 1

34 TeamChaos Documentation Perception

for each color id for each pixel p in seeds if (seeds[p] = color id) and (label[p] = null) queue = p while queue is not empty q = pop(queue) label[q] = blob id color[q] = color id for s in 4-neighbors(q) if (label[s] = null) if CheckHomegeneity(s, q) push(s, queue) else if (label[s] != blob id) if color[s] = color id add to conn table blob id = blob id + 1 return (label, color, conn table)

The FindBlob procedure operates on the following data structures: label[N], an array that associates each pixel to a region ID, or null; color[N], an array that associates each pixel to a color ID, or null; and conn table, a list of connected regions of the same color. The procedure takes the labelled image, and returns a set of blobs plus a connection table. For each desired color, the procedure scans the seeds image to identify labeled pixels (seeds) of that color that have not yet been incorporated in a blob. For each such seed, it starts a standard growing loop based on 4-connectivity. If during the growing we hit a pixel that has already been labeled as belonging to a diﬀerent blob, then we check its color. If its color is the same as the one of the current blob, we mark the two blobs as connected in the connection table. When we add entries to the connection table, we also check it for cycles and remove them. The growing stops at the blob border, as identiﬁed by the homogeneity criterium discussed above. After all the blobs in the original image have been individuated, a post-processing phase goes through the connection table and merges all the connected blobs of the same color as discussed above. In our experiments, 5 to 10 adjacent blobs were often generated for the same connected region. A larger number of blobs (up to 30) were occasionally generated for very large regions. We have incorporated in the above algorithm the computation of blob parameters like the number of pixels, center, width and height. These are updated incrementally as the blobs are grown during the main phase and fused during the post-processing phase. These parameters are needed, e.g., to estimate the position of the objects.

4.2.2 Experiments

We have performed several test to assure the quality of the vision processing and measure the performance of the diﬀerent algorithms. These algorithms depend on lighting conditions, number of objects, distance to objects, etc. In this section we show a series of test realized in an oﬃce environment with a couple of RoboCup objects: the ball and a fake

35 TeamChaos Documentation Perception

yellow net. The main idea is to compare the processing time in the real robot using the diﬀerent algorithms implemented. Thus, the ball has been placed at diﬀerent distances from the robot: (100 cm, 50 cm, and 15 cm) and very close to the camera (5 cm). The next table shows the abbreviation of the segmentation and blob methods used.

Abrev Method Name SRG Seed Region Growing LUT-T LUT Thresholding LUT-SRG LUT Region Growing BG Blob Growing BGO Blob Growing Optimised

The following table resumes the timing (in microseconds) of the diﬀerent method combinations tested so far. The conditions have been quite similar, although calibration parameters are not exactly the same (color cubes and region growing constant). It can be noticed that the best performance is achieved by the LUT-T and BGO combination, being the blob extraction the bottleneck at short distances.

Distance SRG+BG LUT-SRG+BGO LUT-T+BGO NEAR 19799 8492 4938 MEDIUM 24029 9284 5470 FAR 46005 40279 14189

Figure 4.4 shows the SRG+BG combination. It includes three segmented images of the ball at diﬀerent distances. The total processing time for each image is also shown, decomposed in the three main stages of the vision process: segmentation, blob extraction, and object recognition. The times correspond to average values of a series of six frames. This was the combination used during the RoboCup 2005 competition. Unfortunately, these tests have been performed after, mainly because lack of time. This resulted in a very poor ball control, because at short distances the CPU time needed to process a frame was in excess of 30 ms. Figures 4.5, 4.6 shows the LUT-T+BGO and LUT-SRG+BGO combinations. They include four segmented images of the ball at diﬀerent distances and the corresponding blobs. The total processing time for each image is also shown, decomposed in the three main stages of the vision process: segmentation, blob extraction, and object recognition. The times correspond to average values of a series of six frames. Clearly, these combinations outperform the SRG-BG combination. The main reason is the YUV to HSV conversion. Using a very optimized online computation it takes about 13 ms, while the look-up-table methods takes about 4 ms.

36 TeamChaos Documentation Perception

Segment=18264 Segment=21591 Segment=36429 Blobs=1073 Blobs=2120 Blobs=9386 Recognise=461 Recognise=318 Recognise=190 Total=19799 Total=24029 Total=46005 Figure 4.4: SRG + BG

Segment=3793 Segment=3924 Segment=4584 Segment=4008 Blobs=892 Blobs=1300 Blobs=6176 Blobs=9611 Recognise=253 Recognise=247 Recognise=361 Recognise=570 Total=4938 Total=5470 Total=11120 Total=14189 Figure 4.5: LUT-T (7bits) + BGO

4.3 Corner Detection and Classiﬁcation

The robot localizes relying on the extracted features, both the amount of features detected and their quality will clearly affect the process. Currently the robots rely only on colored landmarks for localization, and thus the perception process is based on color segmentation for detecting the different landmarks. Soon artificial landmarks will be removed, as it was done in other leagues. For this reason, our short term goal is to augment the current landmark based localization with information obtained from the field lines, and our long term goal is to rely only on the field lines for localisation [22]. Because of the League rules all the processing must be done on board and for practical reasons it has to be performed in real time, which prevents us from using time consuming algorithms. A typical approach for detecting straight lines in digital images is the Hough Transform and its numerous variants. The various variants have been developed to try to overcome the major drawbacks of the standard method, namely, its high time complexity

37 TeamChaos Documentation Perception

Segment=5624 Segment=7113 Segment=18807 Segment=29076 Blobs=2468 Blobs=1931 Blobs=7887 Blobs=11055 Recognise=400 Recognise=240 Recognise=144 Recognise=148 Total=8492 Total=9284 Total=26838 Total=40279 Figure 4.6: LUT-SRG (7bits) + BGO and large memory requirements. Common to all these methods is that either they may yield erroneous solutions or they have a high computational load. Instead of using the field lines as references for the self-localization, we decided to use the corners produced by the intersection of the field lines (which are white). The two main reasons for using corners is that they can be labelled (depending on the type of intersection) and they can be tracked more appropriately given the small field of view of the camera. In addition, detecting corners can be more computationally efficient than detecting lines. There are several approaches, as reported in the literature, for detecting corners. They can be broadly divided into two groups:

• Extracting edges and then searching for the corners. An edge extraction algorithm is pipelined with a curvature calculator. The points with maximum curvature (partial derivative over the image points) are selected as corners.

• Working directly on a gray-level image. A matrix of “cornerness” of the points is computed (product of gradient magnitude and the rate of change of gradient direction), and then points with a value over a given threshold are selected as corners.

We have evaluated two algorithms that work on gray-level images and detect corners depending on the brightness of the pixels, either by minimizing regions of similar brightness (SUSAN) [44] or by measuring the variance of the directions of the gradient of brightness [45]. These two methods produce corners, without taking into account the color of the regions. As we are interested in detecting ﬁeld lines corners, the detected corners are ﬁltered depending on whether they come from a white line segment or not.

• In the first method (SUSAN) [44], non-linear filtering is used to define which parts of the image are closely related to each individual pixel; each pixel has associated with

38 TeamChaos Documentation Perception

it a local image region which is of similar brightness to that pixel. The detector is based on the minimization of this local image region. The local area contains much information about the structure of the image. From the size, centroid and second moments of this local area corners and edges can be detected. This approach has many differences to the well-known methods, the most obvious being that no image derivatives are used and that no noise reduction is needed. The integrating effect of the principle, together with its non-linear response, give strong noise rejection. The local area is at a maximum when the nucleus lies in a flat region of the image surface, it falls to half of this maximum very near straight edge, and falls even further when inside a corner. It is this property which is used as the main determinant of the presence of edge and corner features.The output of this method is shown in Fig.4.7(a).

• The second method [45] is based on measuring the variance of the directions of the gradient of brightness. The probability of the event that a point belongs to the approximation of a straight segment of the isoline of brightness passing through the point being tested is computed using the technique of Bayesian estimations and used as a weight. Along every isoline, the size of the gradient of brightness remains constant. The two half-lines that form the isoline along which the size of the gradient of brightness is maximal are edges. Along each half-line that is a part of an isoline, the direction of the gradient of brightness remains constant. The directions of the gradient are diﬀerent at the points lying to the left and to the right of the corner axis. The image points at which the size of the gradient of brightness is greater than a predeﬁned threshold are considered to be candidates for corners. The candidates are then examined, the candidate at which the value of corner measure exhibits its local maximum and at which the values of angle of break, and corner appearance are greater than chosen thresholds is a corner. The output of this method is shown in Fig.4.7(b).

(a) (b)

Figure 4.7: (a) Brightness similarity based detector (SUSAN). (b) Brightness gradient based detector

The gradient based method [45] is more parametric, produces more candidate points and requires similar processing capabilities than SUSAN [44], and thus it is the one that we have selected to implement corner detection in our robots. Because of the type of camera used [47], there are many problems associated to resolution and noise. The gradient based method detects false corners in straight lines due

39 TeamChaos Documentation Perception

to pixelation. Also, false corners are detected over the field due to the high level of noise. These effects are shown in Fig. 4.8 (a). To cope with the noise problem, the image is filtered with a smoothing algorithm. Fig. 4.8(b)˙ shows the reduction in noise level, and how it eliminates false detections produced by this noise (both for straight lines and the field).

(a) (b)

Figure 4.8: Brightness gradient-based detector. (a) From raw channel. (b) From smoothed channel. The detected corners are indicated by an orange small square.

The detected corners are then filtered so that we keep only the relevant ones, those produced by the white lines over the field. For applying this color filter, we first segment the raw image. Fig. 4.9 (b) and Fig. 4.10 (b) show the results obtained using a threshold technique for a sample image. We can observe that this technique is not robust enough for labelling the pixels surrounding the required features. Thus, we have integrated thresholding with a region-based method, namely the Seed Region Growing (SRG), by using the pixels classified by the a conservative thresholding as seeds from which we grow color regions. (See [50] for details.) The resulting regions for our image are shown in Fig. 4.9 (c) and Fig. 4.10 (c). Finally, we apply a color filter for all corners obtained with the gradient- based method. We show in Fig. 4.9 (d) and Fig. 4.10 (d) the corners obtained with the gradient-based method (white) and the corners that comply with the color conditions (black).

(a) (b) (c) (d)

Figure 4.9: Feature detection. (a) RGB image. (b) Segmented image by threshold. (c) Segmented image by seeded region growing. (d) Gradient-based detected corners (white) and color-based ﬁltered corners (black). The two detected corner-features are classiﬁed as a C

Once we have obtained the desired corner pixels, these are labelled by looking at the amount of ﬁeld pixels (carpet-color) and line or band pixels (white) in a small window around the corner pixel. Corners are labelled according to the following categories.

40 TeamChaos Documentation Perception

(a) (b) (c) (d)

Figure 4.10: Feature detection. (a) RGB image. (b) Segmented image by thresholding. (c) Segmented image by seeded region growing. (d) Gradient-based detected corners (white) and color-based ﬁltered corners (black). The two detected corner-features are classiﬁed as a C and a T, respectively.

Open Corner A corner pixel surrounded by many carpet pixels, and by a number of white pixels above a threshold. As shown in Fig. 4.11(a).

Closed Corner A corner pixel surrounded by many white pixels, and by a number of carpet pixels above a threshold.As shown in Fig. 4.11(b).

Net Closed Corner A corner surrounded by many white pixels, and by a number of carpet and net (blue or yellow) pixels above a threshold. As shown in Fig. 4.11(c).

(a) (b) (c)

Figure 4.11: Corner labelling. (a) Open corner (b) Closed Corner (c) Net Closed Corner

Note that in order to classify a corner pixel, we only need to explore the pixels in its neighborhood. From these labelled corners, the following features are extracted.

Type C An Open corner nearby of a closed corner. This feature can be detected in the goal keeper area of ﬁeld. As shown in Fig. 4.12(c).

Type T-ﬁeld Two closed corner. Produced by the intersection of the walls and the inner ﬁeld lines. As shown in Fig. 4.12(c).

Type T-net A closed corner nearby of a net closed corner. Produced by the intersection of goal ﬁeld lines and the net. As shown in Fig. 4.12(c).

41 TeamChaos Documentation Perception

(a) (b) (c)

Figure 4.12: Feature types. (a) Type C (b) Type T-ﬁeld(c) Type T-net

4.4 Active Perception

In order to perform reliably in a dynamic environment, an autonomous robot needs to link, or anchor, its actions to the appropriate objects in the environment via perception. The environment being dynamic, the properties of these objects may change over time, and thus require constant perceptual attention. However, a robot typically has limited perceptual resources, both in terms of sensing and of processing power. The design of effective allocation strategies for these resources is therefore of paramount importance in dynamic and partially unpredictable environments. The literature in the field of autonomous robotics has reported many solutions to the problem of controlling perceptual effort. One common approach is to pack both the perceptual and action processes into one module (or behavior), in order to focus the perceptual effort exclusively on those features in the environment which are relevant to the current task. Early examples of this approach can be found in Arkin’s concept of affordances [2], in Malcom and Smithers’ concept of encapsulation [29], and in Brooks’ subsumption architecture [9]. Another common approach is to use information about the current task to perform active control of the sensors, in order to search for specific features. Active control may imply selecting a specific algorithm, or physically pointing a sensor in a specific direction. This approach has been pioneered by Bajcsy [3] and Ballard [4], and is an active research area in computer vision (e.g., [17, 23]). We use a technique for active perceptual anchoring in dynamic environments which combines the two approaches above. Following [38], we introduce perception-based behaviors as basic units of control which use anchors to connect behavior to the environment. An anchor is a data structure which stores information about a physical object, which is needed by the behavior, e.g., the position of a specific door to cross. We then extend the notion of an anchor, in order to use it to focus the perceptual effort. In particular, we include in each anchor a measure of how much the corresponding object is actually needed for the execution of the current task. This measure is used to selectively direct the gaze by deciding which anchor we should try to acquire at any given moment.

42 TeamChaos Documentation Perception

4.4.1 Perception-based Behaviour

In autonomous robotics, one often distinguishes between so-called reactive, or sensor- driven behavior, and proactive, or goal-driven behavior. One way to draw this distinction is to say that these two types of behavior use diﬀerent types of input data. Proactive behavior typically uses as input a combination of prior information and “proprioceptive” data, i.e., data from sensors that observe the state of the robot’s body. Reactive behavior, by contrast, typically takes as input “exteroceptive” data, i.e., data from sensors that observe the environment.

Figure 4.13: Proactive (left) vs. reactive (right) behavior.

This distinction can be seen in the example shown in Fig. 4.13. On the left, the robot is engaged in a proactive GoToNet behavior: the expected position of the net in some reference frame is given as prior information, and the robot relies on its odometry to estimate and reduce its distance from this position. On the right, the robot is using visual-feedback to reactively move in the direction in which the net is observed. Although the two behaviors may produce the same trajectory, there are important differences. The proactive behavior ignores any real-time data about the net: if this is moved, or if the odometry is wrong, the robot will reach a position which is not related to the actual goal. The reactive behavior does not suffer from this limitation, but it will be lost if the net is not perceived, e.g., owing to sensor noise or to occlusion. As it appears, proactive and reactive strategies have merits and drawbacks which are somehow complementary; it is therefore reasonable to combine them into one type of behavior which integrates both goal-based and sensor-based information. To this aim, we define a perception-based behavior as a unit of control organized as shown in Fig. 4.14. The behavior takes exteroceptive data, proprioceptive data, and prior information, and integrates them into an internal state S. In general, S integrates information about the part of the physical world which is relevant to the behavior, and which is to be used by the control law π. The pivot of this integration is the notion of an anchor. Anchors were originally introduced in [34] as “object descriptors,” and have been further investigated in [38, 16]. An anchor is a data structure in S which collects information about a specific physical object, such as the net in the above example. This information is used by the control law π to generate a control value ~u. Perceptual anchoring is responsible for initializing and updating the anchors in S as follows.

Creation: build an anchor in S for each object in the environment used by the behavior,

43 TeamChaos Documentation Perception

Figure 4.14: A perception-based behavior.

and initialize it from prior information; set the criteria against which future percepts should be matched (e.g., the color of the net).

Prediction: predict the properties of the anchor after some time has elapsed since it was last observed; this phase takes proprioceptive data, and possibly a dynamic model of the object into account.

Anchoring: match newly acquired percepts (exteroceptive data) against the anchor, and update it accordingly; a consistency check with the prior information may also be performed in this phase.

This prediction-update cycle is typical of recursive estimation [19]. Anchoring, however, is peculiar in its use of prior, abstract information for initialization and verification [16]. It is easy to see that proactive and reactive behaviors are special cases of perception- based behaviors. The difference between these types of behavior does not reside in the control law, but in the different way in which the anchors relate to the environment. For example, the two go-to-net behaviors above can be produced by a simple control law that minimizes the distance to an anchor associated with the net. In the proactive behavior, this anchor is initialized from prior information and updated by odometry; in the reactive one, it is updated by visual data. The most interesting case, however, is when we use all three types of information to update the anchor. In this case, the anchor acts as a “micro-model” of one object in the environment, which is relevant to the behavior. Through the anchor, the behavior operates in a closed loop with the environment whenever we have perceptual data, and relies on plausible assumptions when these are not available. This is particularly important for the robots used in this work, since looking at one object usually results in losing sight of others. It is useful to know how much an anchor is supported by recent perceptual data, rather than being based on prior information or predictions. Therefore we associate each anchor with a number in the [0, 1] interval, called “anchored,” which measures the level of current perceptual support. This number is set to 1 each time the anchor is updated from perceptual data, and it exponentially decreases when the anchor is updated by prediction only. The amount of the decrease depends on the reliability of the prediction. This in turns depends on the type of object (e.g., fixed or movable) and on the displacement of

44 TeamChaos Documentation Perception

the robot. For the legged robots considered in this work, the decrease was rather dramatic since odometry is extremely unreliable and since most objects are movable.

4.4.2 Active Perceptual Anchoring

Anchors provide the key to making the perceptual processes more relevant to the needs of the behavior. Since the anchors in S contain all the variables used by the π control law, the perceptual processes only need to extract the values of these variables. Moreover, since anchors are models of the needed objects, the perceptual routines can use the information in the anchors to narrow the search space; for example, in order to anchor the ball, we only need to check the camera image for orange blobs close to the ground. Focusing the perceptual processes on the anchors in S greatly simplifies perception. Execution of complex tasks, however, may require information about many objects, and therefore S may contain many anchors. The key observation here is that the control law typically needs to access different anchors at different stages of execution. For example, in the RoboCup domain the robot only needs the position of the ball and of the opponent net when it must kick the ball into the net, and it only needs the position of the landmarks when it must return to its home position in the field. To exploit this fact, we use the organization summarized in Fig. 4.15. Anchors are extended to include a value that measures, on a [0, 1] scale, how much that specific anchor is currently needed by the control law π. This value is fed by π back into S. The perceptual processes use the “needed” values to decide for which anchors they should try to extract information.

Figure 4.15: Feedback of perceptual needs.

As marked in the figure, the anchoring module may also generate sensor controls, e.g., a pan-tilt control to change the fixation point of a camera. In this respect, the “needed” measure can be effectively used to address an important problem in active gaze control: how to decide where to move the fixation point. This problem arises whenever we have one single sensor with only a limited field of view, as it is the case with the camera in the Sony robots. For each anchor a in S, we measure the importance of a as follows:

important(a) = needed(a) · (1 − anchored(a)). (4.1)

The anchor f with the highest importance is selected as the current focus of attention;

45 TeamChaos Documentation Perception

the ﬁxation point is then determined by the expected position of the object which is associated to f. It is easy to see that if there are n anchors which are needed, this selection mechanism will select and anchor all of them in a round-robin fashion. Once an object has been perceived, its “anchored” value will raise, and a new object will then be selected whose “anchored” level had decayed in the meanwhile. Equation (4.1) has a logical reading. The “anchored” and “needed” measures can be interpreted as fuzzy predicates. Then, (4.1) gives the truth value, according to the rules and connectives of fuzzy logic [33], of the fuzzy predicate “important” deﬁned as

important(a) ≡ needed(a) ∧ ¬anchored(a).

In other words, re-anchoring a becomes important if a is needed, and if it has not been anchored recently.

4.4.3 Implementation

At an abstract level, the control architecture that we have used in each robot is similar to the one sketched in Fig. 4.15. The S state contains an anchor for each object of interest. Each anchor contains the type of object, its (ρ, θ) position in robot’s polar coordinates, and the value of the “anchored” and “needed” fuzzy predicates. The updating of the position and “anchored” ﬁelds is done as discussed above. The “needed” ﬁeld is set by the π control law (in the HBM module, as described in section 7.1). Perceptual anchoring is implemented as follows. Embedded in the last step of the vision pipeline (object recognition), we check blobs against the following three object- dependent criteria, illustrated in Fig. 4.16.

Figure 4.16: A ball blob in the astray zone (left), tracking zone (center), and anchoring zone (right).

First, we reject any blob which is “astray” for its type: in the leftmost picture in Fig. 4.16, the pink band of a landmark has been incorrectly classiﬁed as a ball (orange) blob, but rejected because the ball is assumed to lay on the ground. Second, we select blobs which are in the “anchoring zone” (AZ) for their type. This is the area in the image where we can reliably measure the object’s position. For instance, a landmark blob is in its AZ if it is fully inside the image (i.e., it does not touch the image border) since we use the blob’s size to estimate distance. On the other hand, the AZ for the ball is a small fovea at the center of the image. Finally, we consider the remaining blobs to be in “tracking zone” (TZ); these should be brought into the anchoring zone using gaze control.

46 TeamChaos Documentation Perception

Gaze control is described by the ﬁnite state automaton sketched in Fig. 4.17, where rectangles denote head motion states whose transitions are activated by events, and ovals denote purely computational states whose transitions depend on the result of the computation. The meaning of each state is as follows.

Figure 4.17: State transitions for perceptual anchoring.

Select: choose an anchor f on which to focus using (4.1); if the “anchored” level of f is greater than a given threshold, then set the ﬁxation point to this position and exit via the Good Estimate transition; otherwise exit via the not-GE transition.

Scan: perform a visual scan to explore the part of the space where f could be located, e.g., the ﬁeld ground for the ball; exit when one of the following events occurs:

• a blob that matches f appears in the TZ; set the ﬁxation point to that blob’s position and exit via the f-in-TZ transition; • a blob that matches an anchor a 6= f appears in the AZ for a; exit via the nf-in-AZ transition; • the physical scan is completed; exit via the Scan Complete transition.

LookTo: move the camera to the current ﬁxation point; when:

• a blob that matches f appears in the AZ, exit via the f-in-AZ transition; • a blob that matches f appears in the TZ, set the new ﬁxation point to that blob’s position and stay in this state; • a blob that matches an anchor a 6= f appears in the AZ for a, exit via the nf-in-AZ transition; • the ﬁxation point has been achieved, and no blob that matches f is in the image, exit via the Position Achieved transition.

Anchor: measure the position of the object and update the f anchor; select a new focus.

Serendipitous anchor: an object other than f is in the AZ: measure its position and update the corresponding anchor; continue to search for f.

47 TeamChaos Documentation Perception

Lost: either “Scan” or “LookTo” have been completed without the current f being found; mark f as lost and go select a new focus.1

There is a subtlety in the “a blob that matches” conditions above. In general, how to associate regions to objects is a complex issue, which may require the use of extra- perceptual information. For example, the three opponent robots all have the same perceptual signature, and can only be distinguished by using information about their past positions. In our case, we simply associate a blob to an anchor based on its type, i.e., its color. (See [16] for a more elaborated solution to the association problem.)

4.4.4 Experiments

We present three simple experiments. In the ﬁrst experiment, the robot simply had to stand still and track the ball, which was moved by hand. This was achieved by the following rule (see section 7.1 for information on rules), which sets the “needed” value for the ball to 1:

ALWAYS NEED (Ball)

Fig 4.18 shows the result of ball tracking. The upper part plots the temporal evolution of the estimate of θ in the anchor for the ball. The circles mark the “LookTo” commands sent to the head whenever a new fixation point was determined; the dotted vertical lines mark “Scan” commands. The lower part of the figure shows the corresponding evolution of the “anchored” value. From the beginning till time 66 the ball was moved between ±60◦, at a fixed distance in front of the robot. The robot was able to track and anchor the ball at all times. At time 66 the ball was removed from the field; the robot then started a visual scan, while maintaining the previous θ value and decreasing the degree of anchoring. At time 86 the ball was laid down again, and the robot could rapidly identify and re-anchor it. The rest of the plot shows another sequence of moving the ball, stealing it, and putting it down again. The robot consistently kept the ball anchored when it was in view, and searched for it when it was out of view. In the second experiment the robot was placed in the middle of the field and heading at 80◦ from a net (Net1), and it had to track both the moving ball and the net. This was achieved by the following two rules:

ALWAYS NEED (Ball) ALWAYS NEED (Net1)

The plot in Fig. 4.19 shows the temporal evolution of the estimate of θ for the ball and net anchors, and the corresponding values for the “anchored” predicates. The ball was ﬁrst moved between ±60◦ in front of the robot, then taken out of the ﬁeld at time 43.

1A lost object will not be selected as focus as long as there are other objects which are important according to (4.1).

48 TeamChaos Documentation Perception

Figure 4.18: Tracking a moving ball.

At time 57, it was put back down and moved from left to right in front of the robot. The robot alternatively looked at the ball and at the net using the Select function discussed above, and kept both of them constantly anchored when the ball was in view. When the ball was removed from the ﬁeld, the robot alternatively made a scan for the ball, and re-anchored the net.

Figure 4.19: Tracking a moving ball and the net..

The third and final experiment illustrates how active perceptual anchoring is guided by the current perceptual needs expressed by the behaviors during execution. The robot first has to approach the ball, then align with the net and finally kick the ball. This behavior uses the following two perceptual rules:

49 TeamChaos Documentation Perception

ALWAYS NEED (Ball) IF (ballClose) THEN NEED (Net1)

The plot in Fig. 4.20 shows the evolution of (θ, ρ) for the ball anchor, and of θ for the net anchor, together with the two values for “anchored.” Until time 118 the robot was approaching the ball (as testiﬁed by the decreasing ρ) while tracking it. Since the ballClose predicate was false, the “needed” value for the net was kept at 0, and the net was never selected as the focus of attention; as a consequence, the camera was never pointed at the net, which remained fully unanchored, with θ at its default value of 0. At time 118, the truth value of ballClose, and hence the net’s “needed” value, was high enough for the net to become the focus of attention. The robot then started a visual scan to search for the net, momentarily losing sight of the ball. The net was soon seen and anchored, and the robot then pointed its camera to the expected position of the ball and re-anchored it. The robot then got close to the ball, and at time 127 it executed the alignment maneuver while using the camera to alternatively anchor the ball and the net. The alignment maneuver brought the θ of both the ball and the net close to zero. Finally, at time 137, the robot kicked the ball, which headed away from the robot and toward the net.

Figure 4.20: Switching between tracking only the ball, and tracking both ball and net.

The above behaviors are extremely simple, but they illustrate the main ingredients of active perceptual anchoring. A set of signiﬁcantly more complex behaviors were implemented for the RoboCup competition using the mechanisms described in this paper. During the games, the robots successfully tracked all and only the objects which were relevant to the task being performed.

50 TeamChaos Documentation Self-Localization

Chapter 5

Self-Localization

Self-localization in the 4-legged robot league is a challenging task: odometric information is extremely inaccurate; landmarks can only be observed sporadically since a single camera is needed for many tasks; and visual recognition is subject to unpredictable errors (e.g., mislabeling). To meet these challenges, we have developed a self-localization technique based on fuzzy logic, reported in [12]. This technique needs only qualitative motion and sensor models, can accommodate sporadic observations during normal motion, can recover from arbitrarily large errors, and has a low computational cost. The result of self-localization is used to make strategic decisions inside the HFSM, and to exchange information between robots in ﬁeld coordinates.

(a) (b) Figure 5.1: Self localization grids.

This technique, implemented in the GM module, relies on the integration of approximate position information, derived from observations of landmarks and nets, into a fuzzy position grid — see Fig. 5.1. To include own motion information, we dilate the grid using a fuzzy mathematical morphology operator. Using this technique, our robots can maintain a position estimate with an average error of approximately ±20 cm and ±10◦ during normal game situations. Localization is done continuously during normal action. Stopping the robot to re-localize is only needed occasionally (e.g. in case of major errors due to an undetected collision). The extension to our method which includes natural features as landmarks is described in [22]. The extended technique allows the robot to localize using only the nets and natural

51 TeamChaos Documentation Self-Localization

features of the environment (i.e. corners).

5.1 Uncertainty Representation

5.1.1 Fuzzy locations

Location information may be affected by different types of uncertainty, including vague- ness, imprecision, ambiguity, unreliability, and random noise. An uncertainty representation formalism to represent locational information, then, should be able to represent all of these types of uncertainty and to account for the differences between them. Fuzzy logic techniques are attractive in this respect [36]. We can represent information about the location of an object by a fuzzy subset µ of the set X of all possible positions [52, 53]. For instance, X can be a 6-D space encoding the (x, y, z) position coordinates of an object and its (θ, φ, η) orientation angles. For any x ∈ X, we read the value of µ(x) as the degree of possibility that the object is located at x given the available information. Fig. 5.2 shows an example of a fuzzy location, taken in one dimension for graphical clarity. This can be read as “the object is believed to be approximately at θ, but this belief might be wrong”. Note that the unreliability in belief is represented by a uniform bias b in the distribution, indicating that the object might be located at any other location. Total ignorance in particular can be represented by the fuzzy location µ(x) = 1 for all x ∈ X.

5.1.2 Representing the robot’s pose

Following [12], we represent fuzzy locations in a discretized format in a position grid: a tessellation of the space in which each cell is associated with a number in [0, 1] representing the degree of possibility that the object is in that cell. In our case, we use a 3D grid to represent the robot’s belief about its own pose, that is, its (x, y) position plus its orientation θ. A similar approach, based on probabilities instead of fuzzy sets, was proposed in [11]. This 3D representation has the problem of having a high computation complexity, both in time and space. To reduce complexity, we adopt the approach proposed by [12]. Instead of representing all possible orientations in the grid, we use a 2D grid to represent the (x, y) position, and associate each cell with a trapezoidal fuzzy set µx,y = (θ, ∆, α, h, b) that represents the uncertainty in the robot’s orientation. Fig. 5.2 shows this fuzzy set. The θ parameter is the center, ∆ is the width of the core, α is the slope, h is the height and b is the bias. The latter parameter is used to encode the unreliability of our belief as mentioned before.

For any given cell (x, y), µx,y can be seen as a compact representation of a possibility distribution over the cells {(x, y, θ) | θ ∈ [−π, π]} of a full 3D grid. The reduction in complexity is about two orders of magnitude with respect to a full 3D representation (assuming a angular resolution of one degree). The price to pay is the inability to handle

52 TeamChaos Documentation Self-Localization

Figure 5.2: Fuzzy set representation of an angle measurement θ.

multiple orientation hypotheses on the same (x, y) position — but we can still represent multiple hypotheses about diﬀerent positions. In our domain, this restriction is acceptable.

5.1.3 Representing the observations

An important aspect of our approach is the way to represent the uncertainty of observations. Suppose that the robot observes a given feature at time t. The observed range and bearing to the feature is represented by a vector −→r . Knowing the position of the feature in the map, this observation induces in the robot a belief about its own position in the environment. This belief will be affected by uncertainty, since there is uncertainty in the observation. In our domain, we consider three main facets of uncertainty. First, imprecision in the measurement, i.e., the dispersion of the estimated values inside an interval that includes the true value. Imprecision cannot be avoided since we start from discretized data (the camera image) with limited resolution. Second, unreliability, that is, the possibility of outliers. False measurements can originate from a false identification of the feature, or from a mislabelling. Third, ambiguity, that is, the inability to assign a unique identity to the observed feature since features (e.g., corners) are not unique. Ambiguity in observation leads to a multi-modal distribution for the robot’s position. All these facets of uncertainty can be represented using fuzzy locations. For every type of feature, we represent the belief induced a time t by an observation −→r by a possibility −→ distribution St(x, y, θ| r ) that gives, for any pose (x, y, θ), the degree of possibility that the robot is at (x, y, θ) given the observation −→r . This distribution constitutes our sensor model for that specific feature. −→ The shape of the St(x, y, θ| r ) distribution depends on the type of feature. In the case of net observations, this distribution is a circle of radius |−→r | in the (x, y) plane, blurred according to the amount of uncertainty in the range estimate. Fig. 5.3 show an example of this case. In the figure, darker cells indicate higher levels of possibility. We only show the (x, y) projection of the possibility distributions for graphical clarity. Note that the circle has a roughly trapezoidal section. The top of trapezoid (core)

53 TeamChaos Documentation Self-Localization

(a) (b)

Figure 5.3: Belief induced by the observation of a blue net (a) and a yellow net (b). The triangle marks the center of gravity of the grid map, indicating the most likely robot localization. identiﬁes those values which are fully possible. Any one of these values could equally be the real one given the inherent imprecision of the observation. The base of the trapezoid (support) identiﬁes the area where we could still possibly have meaningful values, i.e., values outside this area are impossible given the observation. In order to account for unreliability, then, we include a small uniform bias, representing the degree of possibility that the robot is “somewhere else” with respect to the measurement. −→ The St(x, y, θ| r ) distribution induced by a corner-feature observation is the union of several circles, each centered around a feature in the map, since our simple feature detector does not give us a unique ID for corners. Fig. 5.4 shows an example of this. It should be noted that the ability to handle ambiguity in a simple way is a distinct advantage of our representation. This means that we do not need to deal separately with the data association problem, but this is automatically incorporated in the fusion process (see below). Data association is one of the unsolved problems in most current self-localization techniques, and one of the most current reasons for failures.

5.2 Fuzzy Self-Localization

Our approach to feature-based self-localization extends the one proposed by Buschka et al in [12], who relied on unique artiﬁcial landmarks. Buschka’s approach combines ideas from the Markov localization approach proposed by Burgard in [11] with ideas from the fuzzy landmark-based approach technique proposed by Saﬃotti and Wesley in [42].

1 The robot’s belief about its own pose is represented by a distribution Gt on a 2 2 D possibility grid as described in the previous section. This representation allows us to represent, and track, multiple possible positions where the robot might be. When the robot is ﬁrst placed on the ﬁeld, G0 is set to 1 everywhere to represent total ignorance. This belief is then updated according to the typical predict-observe-update cycle of recursive

54 TeamChaos Documentation Self-Localization

(a) (b)

Figure 5.4: Belief induced by the observation of a feature of type C (a) and T (b). Due to symmetry of the ﬁeld, the center of gravity is close to the middle of the ﬁeld. state estimators as follows.

Predict When the robot moves, the belief state Gt−1 is updated to Gt using a model of the robot’s motion. This model performs a translation and rotation of the Gt−1 distribution according to the amount of motion, followed by a uniform blurring to account for uncertainty in the estimate of the actual motion. Fig. 5.5 show an example of this step, in which motion (translation) and uncertainty (blurring) model are applied. Observe The observation of a feature at time t is converted to a possibility distribution 1 St on the 2 2 grid using the sensor model discussed above. For each pose (x, y, θ), this distribution measures the possibility of the robot being at that pose given the observation. Fig. 5.3 and 5.4 show these possibility distribution.

Update The possibility distribution St generated by each observation at time t is used to update the belief state Gt by performing a fuzzy intersection with the current distribution in the grid at time t. The resulting distribution is then normalized.

In the Fig. 5.6 is shown the localization pipeline process, the recursive system is continously applied to mantain the robot’s pose estimation. If the robot needs to know the most likely position estimate at time t, it does so by computing the center of gravity of the distribution Gt. A reliability value for this estimate is also computed, based on the area of the region of Gt with highest possibility and on the minimum bias in the grid cells. This reliability value is used, for instance, to decide to engage in an active re-localization behavior. In practice, the predict phase is performed using tools from fuzzy image processing, like fuzzy mathematical morphology, to translate, rotate and blur the possibility distribution in the grid [5, 6]. The intuition behind this is to see the fuzzy position grid as a gray-scale image.

55 TeamChaos Documentation Self-Localization

(a) (b) (c)

Figure 5.5: Robot’s belief posisition is represented by Gt distribution (a), when robot moves a translation is applied to Gt using the robot’s motion model (b) and a blurring to account for uncertainty of motion(c).

Figure 5.6: Flowchart localization process. First, the belief state Gt is updated with the possibility distribution St(x, y|r) of an observation, then the robot moves and the odometry model B is applied to Gt, obtaining Gt+1. Then, a new perception is updated, 0 outcoming the last belief state Gt.

56 TeamChaos Documentation Self-Localization

For the update phase, we update the position grid by performing pointwise intersection of the current state Gt with the observation possibility distribution St(·|r) at each cell (x, y) of the position grid. For each cell, this intersection is performed by intersecting the trapezoid in that cell with the corresponding trapezoid generated for that cell by the observation. This process is repeated for all available observations. Intersection between trapezoids, however, is not necessarily a trapezoid. For this reason, in our implementation we actually compute the outer trapezoidal envelope of the intersection. This is a conservative approximation, in that it may over-estimate the uncertainty but it does not incur the risk of ruling out true possibilities. There are several choices for the intersection operator used in the update phase, depending on the independence assumptions that we can make about the items being combined. In our case, since the observations are independent, we use the product operator which reinforces the eﬀect of consonant observations. Our self-localization technique has nice computational properties. Updating, translat- ing, blurring, and computing the center of gravity (CoG) of the fuzzy grid are all linear in the number of cells. In the RoboCup domain we use a grid of size 36 × 54, corresponding to a resolution of 10 cm (angular resolution is unlimited since the angle is not discretized). All computations can be done in real time using the limited computational resources available on-board the AIBO robot.

5.3 Experimental results

To show how the self-localisation process works, an example generated from the goal keeper position is presented. Let’s suppose that the robot starts in its own area, more or less facing the opposite net (yellow). At this moment the robot has a belief distributed along the full field – it does not know its own location. Then the robot starts scanning its surroundings by moving its head from left to right. As soon as a feature (natural or not) is perceived, it is incorporated into the localization process. When scanning, the robot first detects a net and two features of type C (Fig.??). The localization information is shown in (Fig.5.7), where the beliefs associated to the detection are fused. The filled triangle represents the current estimate. In order to cope with the natural symmetry of the field, we use unique features, like the nets are. When the robot happens to detect the opposite net (yellow), it helps to identify in which part of the field the robot is currently in, and the fusion with the previous feature based location gives a fairly good estimate of the robot position. Since our feature detector doesn’t provide unique identifiers for corners, the possible robot positions induced by a corner observation is the union of several circles, each centered around a corner in the map. This can be seen in Fig. 5.4. It should be noted that the ability to handle this ambiguity efficiently is a primary advantage of this representation. The data association problem is automatically addressed in the fusion process (Fig. 5.7). Data association is a difficult problem, and many existing localization techniques are unable to adequately address it.

57 TeamChaos Documentation Self-Localization

(a) (b) (c)

Figure 5.7: Belief induced by the observation of (a) a net, (b) the ﬁrst feature, and (c) the second feature. Initially the position of the robot is unknown (belief distributed along the full ﬁeld).

58 TeamChaos Documentation Information Sharing

Chapter 6

Information Sharing

6.1 TCM: Team Communication Management

TCM is an Open-R object which controls the communications between an Aibo robot and external entities (other Aibo robots, remote computers, etc). It is a “communication center” which is called from other modules when they want to communicate. Therefore, the TCM module will open a set of UDP Sockets to send and receive messages. The messages will be processed regarding the receiving port. At this moment, TCM opens 4 ports, 3 of them for ChaosManager communications and the other one for external communications using the object ExternalMessage. Communications with the ChaosManager have their own protocol for commands and data. An ExternalMessage object has a set of fields describing the data source and destiny, both the source entity (Aibo or computer) and the TeamChaos module that sent it. It works as the exchange unit between entities. The communication protocol developed is stateless due to is based in UDP. It means that there are not any connection establishment and termination. Also, transmissions are not reliable and the communication protocol does not use any ack’s or retrasmissions. OPEN-R offers TCP/IP support but we have choose the UDP alternative for make lighter communicatios. Each robot has an unique IP and it is associated to his dog identifier in file TCM.ini. When someone sends a message to the identifier of a robot, the consequence will be an unicast message towars the IP joined to this identifier. We allow broadcast transmissions too. This feature is useful for send debug information that is analyzed and showed in our external debug tools (ChaosManager). A class called UDPConn has been created for the implementation of the TCM. This class provides an abstraction layer over the OPEN-R IP stack, showing to the developer an easy interface with calls as getSocket, closeSocket, send. . . . Also, a TcmClient class has been developed for hide all details of the communication inter-object mechanism tipical in OPEN-R.

59 TeamChaos Documentation Information Sharing

6.1.1 Arquitecture

The main arquitecture of the communication system is shared in two OPEN-R objects. The main object is the ORTcm and is exclusively dedicated to communicatios tasks. The second object is ORRobot which has important goals to fulﬁll (localization, perception, generate behaviors, . . . ) but has a TcmClient object instantiated. This object acts as a stub to communicate modules in ORRobot with ORTcm. Data serialization and intermodules communication are functions that solves TcmClient class. ORTcm object uses a low level auxiliary class that allow to use the communication primitives in OPEN-R IP stack in a more handy way. If oﬀers the abstraction of socket and we can open, close, send and receive in/from it. The TCM module also allows sending and receiving data with unlimited size, even bigger than the maximun size permitted by the OPEN-R IP stack. Along this document we will enter into the details of those features.

UDPConn

The UDPConn class gives a simple API to access the UDP communications provided by OPEN-R. First of all, it must be said that, as all the OPEN-R system do, the IP stack is based in events passing, what means that the work with the stack consists in sending messages to the stack and wait for the response event, that must be suitably attended. This feature pushes an object that wants to use the IP stack to be an OPEN-R object. With the UDPConn class, an OPEN-R object which wants to use UDP communications has to (we use the ORTcm implementation as example):

• Inherit from UDPConn:

class ORTcm : public OObject, public UDPConn {

• Call UDPConn’s constructor:

ORTcm::ORTcm () : UDPConn() {

• Implement notifySend y notifyReceive methods. They are called when receive and send events come from the IP stack

• Add the extra entries SendCont() y ReceiveCont() to the object’s stub.cfg ﬁle:

// Extra connections for IP Stack Extra : SendCont() Extra : ReceiveCont()

From now on, the object can open and close sockets and send and receive using them. When one of the opened sockets receive a packet, a call to notifyReceive() will be made,

60 TeamChaos Documentation Information Sharing

so the class will implement here the suitable packet processing code. The method notify- Send() will be called when a packet sending is completed. In this case, one of the possible actions could be do nothing. For the ORTcm, the notifySend() method has been used to implement the possibility of sending more data bytes than the maximum data packet permitted. A structure to store data (a byte array) sent through a socket UDP.

struct DatagramPacket { IPAddress address; /// IP Address of sender int port; /// Port of sender byte *buff; /// Data int length; /// Data Length };

When a receive event is produced in a socket, the information received from the IP stack is processed and a DatagramPacket is generated, which is passed to the notifyReceive method that will interpret the content. The method send also accepts a DatagramPacket (although there is another send method that accepts a byte array to send) Structure to represent an UDP socket.

struct DatagramSocket { /// Endpoint conection to the OPEN-R IPStack antModuleRef endpoint;

antSharedBuffer sendBuffer; /// Send Buffer for the socket byte* sendData; /// Pointer to map the send buffer

antSharedBuffer recvBuffer; /// Receive Buffer for the socket byte* recvData; /// Pointer to map the receive buffer

int localport; /// Local port connected to this socket IPAddress localaddress; /// Local addres of the socket

/// Remaining bytes to send (for send more bytes than sendSize) int toSend;

int index; /// Index of the socket boolean busy; /// TRUE if socket is occupied sending

int sendSize; /// max size of packets sent };

This structure allows us to store the needed OPEN-R information (endpoints, buﬀers, etc.) for a socket. Therefore the class using UDPConn uses this socket structure. For example, the send method receives a socket for sending it as a parameter.

61 TeamChaos Documentation Information Sharing

ExternalMessage

This class will be used as a container for information to be sent to the outer space (i.e. the part of the world out of the robot). It has been designed to abstract the data type of the information to be sent. Basically consists of a payload with the information to be sent plus some ﬁelds with the information about source and destiny (module and remote entity). The objects of this class will be converted to a byte array in order to be sent through an UDP socket and they will be converted to an ExternalMessage at the destiny.

Figure 6.1: Attributes in externalMessage class

Next, we’re going to brieﬂy describe all ﬁelds of an externalMessage object:

SourceIP: It stores the IP address of the source robot or machine.

TargetIP: It stores the IP address of the destination robot or machine.

DogIDSource: It contains the identiﬁer of the source robot (values between 1 and 4).

DogIDTarget: It contains the identiﬁer of the destination robot (values between 1 and 4 for unicast or 0 for broadcast).

ModuleIDSrc: In this ﬁeld is stored the module that triggers communication (Cmd, Gm, Hbm, Pam, ...).

ModuleIDTrg: This ﬁeld shows the destination module at target robot.

TeamColor: This atribute is for recognize the team (red or blue) of the robot sender.

DataType: Here we store the type of the payload carried (values allowed are TYPE MONITOR, TYPE GRID, TYPE BOOL, TYPE INT, TYPE STRING,... In ﬁle externalMes- sage.h are declared all data types allowed).

62 TeamChaos Documentation Information Sharing

Paylen: This ﬁeld shows the length in bytes of the payload attribute. Payload: It stores the data ready to send serialised.

To get an array of bytes from an ExternalMessage Object the method getBytes must be called. This method accepts a pointer to a byte space reserved for the array as a parameter. The method will copy the ExternalMessage in a byte array form to this space. There must be enough space to allocate the array. To ensure this, the user should call the GetTotalLen method of ExternalMessage to know the space needed. To create an ExternalMessage, there exits a constructor that accept an array of bytes. Using this, it will fill the fields of an external message with the bytes contained in the array. The constructor will allocate memory for copying the payload. To support different payload information in an abstract way, the methods SetPayload() and GetPayload() have been implemented. SetPayload() will be used to copy the information to be sent into a new ExternalMes- sage object. It accepts the type of payload and a pointer to the payload. Regarding the type of the payload, it will allocate memory to store the payload and the payload will be copied to this space. The process to add a new type of payload consists of adding a new value for the TypeID type for this payload and then implementing the calculation of the payload length for this type of payload in this method. GetPayload() will be used to extract the payload from the external message in its native type. In order to support different payloads, the method is overloaded for every type. Basically, the method accepts a pointer to a memory space for the payload and place the payload in it. For every type of payload, there is an implementation of GetPayload() which accepts a pointer to a variable of the type. So, when a new type of payload is added to the system, a new implementation of GetPayload() must be added, which receives a pointer to a variable of the new type. The code to copy the payload to the variable will be in this implementation. It will consist, basically, of a check of the type of payload to assure that it matches with the type required and then the code for copying the data to the destiny.

TcmClient

The Open-R programming paradigm is based on objects. The way those objects can share information is using a message passing mechanism. Normally, developers will need to use communication in ORRobot object, but communication primitives built in TeamChaos are in ORTcm. The C++ class TcmClient is designed for act as a communication API to developers and hide all details of passing information between ORRobot and ORTcm. The place to instantiate the TcmClient class is in the DoInit() method present in all objects. Only it is necessary to execute two instructions. We can see it in a snippet of our DoInit() in ORRobot:

//Instance a TcmClient object for communication

63 TeamChaos Documentation Information Sharing

tcmClient = TcmClient::GetInstance(); tcmClient->SetGate(subject[sbjSetTcmExtMsg]);

Of course, you must declare the object tcmClient as a pointer to a TcmClient class as we do in ORRobot.h:

//Communication TcmClient *tcmClient;

Normally, you do not need to write this instructions because there are on the code now but if you need to write a new object, you should instantiate a new TcmClient.

ORTcm

This module will work as a communication centre for the rest of modules in the Team Chaos architecture. It will be the entry point for the messages received by a robot and will be the “postman” on charge of sending messages to external receivers. In our cases, the object will open the UDP ports needed for ChaosManager control (images port for vision monitoring, control port for teleoperation and configuration port for update camera parameters and segmentation values) and one port for sending/receiving ExternalMessages. The values for this ports are configured in the file TCM.INI.

6.1.2 Communicating with TCM

In this section we are going to talk about how a module can use communications. Com- munications can be used in two diﬀerent ways, either for send and receive data.

Sending Data

Objects that wish to send data must inherit from ISerializable interface. This means that if you plan to send data, you must create a class, inherit from ISerializable and implements the two methodss Serialize() and Unserialize(). We can see an example in the class BallPosition created as example. This is the header ﬁle (BallPosition.h):

//*************************************** #ifndef _BallPosition_H #define _BallPosition_H

#include "ISerializable.h" class BallPosition : public ISerializable

64 TeamChaos Documentation Information Sharing

{ public: BallPosition(); ~BallPosition() {}

void SetX(int val) { x = val;} void SetY(int val) { y = val;} void SetAnchor(float val) {anchor = val;}

int GetX() {return x;} int GetY() {return y;} float GetAnchor() {return anchor;}

//Virtual methods to implement, inherited from ExternalMesage void Serialize(ISerializable *object, byte **stream, size_t *len); void Unserialize(byte **stream, ISerializable *object, size_t len); private: int x; int y; float anchor; };

#endif //***************************************

And now the implementation ﬁle (BallPosition.cc):

//*************************************** BallPosition::BallPosition() { this->SetX(0); this->SetY(0); this->SetAnchor(0.0); } void BallPosition::Serialize(ISerializable *object, byte **stream, size_t *len) { *len = (2 * sizeof(int) + sizeof(float)); byte *p;

int x, y; float anchor;

*stream = (byte *) malloc(*len); p = *stream;

x = ((BallPosition*)object)->GetX(); memcpy(p,&x,sizeof(int));

65 TeamChaos Documentation Information Sharing

p += sizeof(int); y = ((BallPosition*)object)->GetY(); memcpy(p,&y,sizeof(int)); p += sizeof(int); anchor = ((BallPosition*)object)->GetAnchor(); memcpy(p,&anchor,sizeof(float)); } void BallPosition::Unserialize(byte **stream, ISerializable *object, size_t len) { byte *p = *stream; int x, y; float anchor;

memcpy(&x,p,sizeof(int)); p += sizeof(int); memcpy(&y,p,sizeof(int)); p += sizeof(int); memcpy(&anchor,p,sizeof(float));

((BallPosition*)object)->SetX(x); ((BallPosition*)object)->SetY(y); ((BallPosition*)object)->SetAnchor(anchor); }

When you have wrote your class, only rests to declare and to ﬁll the object you want to send and invoke the method SendExternalMessage of the TcmClient object. For example:

BallPosition gball;

gball.SetX(10); gball.SetY(20); gball.SetAnchor(0.5);

tcmClient->SendExternalMessage(3,CMD,GM,TYPE_POS, &gball);

The first argument of SendExternalMessage is the target. At this moment, we only use values between 0 and 4. Value 0 corresponds to broadcast address (all robots and all PC’s in the same network) and the rest of values are data send to robot 1, robot 2, robot 3 and robot 4. The second and third arguments are the identifiers of the source and target module. In file ExternalMessage.h are defined all possible modules and their identifiers. The next argument is the identifier of the data sent and all different data available are defined in ExternalMessage.h too. The last parameter must be a pointer to the data. That’s all needed to send data to other robot or PC. There are not any restrictions in data size. This service is best effort and there are no guaranties that data arrives towards the destination.

66 TeamChaos Documentation Information Sharing

Receiving Data

We have developed a callback register service to make a non blocking receive primitive. Therefore, you must write a callback method that will be triggered when the suitable data arrives to your object. The next example shows a callback method: void ReceiveGlobalData (byte dogSrcID, ModuleID dstModule, size_t len, byte *stream) { BallPosition ballPos; ballPos.Unserialize(&stream,&ballPos,len);

printf("x: %d\n",ballPos.GetX()); printf("y: %d\n",ballPos.GetY()); printf("anchor: %f\n",ballPos.GetAnchor()); }

It’s necessary that all callback methods should have the same prototype (use the previous as a template and only change the name of the method). In the next example we see the register process. Gm module (part of the ORRobot object) needs to invoke the method RegisterCallback but previously it needs to obtain a reference to the TcmClient of the ORRobot object.

//Get the reference of the TcmClient tcmClient = TcmClient::GetInstance();

//Register a test method tcmClient->RegisterCallback(GM,TYPE_POS,&ReceiveGlobalData,true);

The first argument is the module into the object that have the callback method. The next argument is the identifier of the data expected. Next parameter must be a pointer to the specific callback declared before. The last parameter is not used yet and you can keep it as in the example at true.

6.1.3 Implementation

In this section we’re going to explain all details of the communication layer involving ORRobot and ORTcm objects. In order to understand those aspects easily we will begin with the send data process and then we will justify the receive phase.

Design for sending data

As we have described in previous section, applications or behaviors will invoke SendExter- nalMessage() from the TcmClient class. Inside this method there are three main tasks to

67 TeamChaos Documentation Information Sharing

Figure 6.2: Communication class diagram complete. First of all we create a new externalMessage and fill all its fields (destination, data, length, etc.). It’s important to note that data are stored serialised using the seri- alized() method which should be implemented by the object passed to the method. The second task is to push this external message into a queue. Inside this queue are stored all messages ready to send. Finally, the last goal is to communicate one message to the ORTcm object. Only if the queue has one message we will send it to the ORTcm object directly. This method is the starting point of the flux control mechanism for sending messages to ORTcm only when is needed. The first message is stored in the queue and sent to the ORTcm. However, the rests of messages will be stored but not sent until we have the confirmation of the first message has been sent. On the other side we have the second object involved: ORTcm. This object receives the first message. Due to the implementation of the SetData() and NotifyObservers() in Open-R, we always serialise again the message. In a previous step, data were serialised. Now and before passing the message from ORRobot towards ORTcm, we serialise again the complete message. This is due to SetData() makes a breadth copy and not a depth copy, so if some field is a pointer, this Open-R method can not cross over it. When ORTcm receives the message serialised, it composes an externalMessage again. Our communications library now must check if it’s necessary to split data in several fragments. We have designed a small protocol based in the message format showed in figure 6.5. We encrust four fields in each fragment. . . sourceIP is the IP address of the

68 TeamChaos Documentation Information Sharing

Figure 6.3: Communication diagram in ORRobot for sending destination. Msg. ID is an identifier of each message in a robot. Those two joined fields form an unique identifier that allow merge the fragments in destination. The last fields are the number of fragment sent in the message and the total number of fragments in data. Each fragment is sent using the send() function offered by UDPConn class. The communication model in Open-R triggers a notifySend method asynchronously when data are sent by the physical layer. In our case, only when the last fragment is sent, we must make some job. This job is to inform ORRobot that his fragment has been sent. We have decouple the communication primitives from their original size restrictions and we have hidden this details to ORRobot object. When ORRobot receives this pseudo ack, it runs the method setTcmNotifySend() from TcmClient class. This method deletes the message from the queue due that has been sent

69 TeamChaos Documentation Information Sharing

Figure 6.4: Communication diagram in ORTcm object for sending

and if there are messages pending to send, takes the next and makes another iteration of the communication cycle passing it to the ORTcm object. This process ends when the queue is empty. This design has one main goal in mind: Avoid race conditions. We must coordinate two threads (ORRobot and ORTcm) with writer and reader roles, respectively. ORobot wants to write in a shared buffer and ORTcm wants to read this shared buffer for send data. The main key of our design based in an ordered flux control mechanism is only allows to write in the shared buffer when the ORTcm is ready to accept a new message.

70 TeamChaos Documentation Information Sharing

Figure 6.5: Message format including split info in red and data (externalMessage serialised) in pink

Design for receiving data

As we explained previously, we have used a callback mechanism for receiving data. Inside tcmClient we have a two dimension matrix with all modules in one side and all data types available for sending/receiving in the other side. So we have a table with all combinations between modules and data types. When someone registers a callback, we insert the pointer to the function in the appropriate position of the register table according to the module and data type.

Figure 6.6: Example of register table with two callbacks registered

When a robot receives data, the ORTcm object inspects the split info contained in the message. The message could be a simple and unique message or could be part of several fragments that we need to merge. We keep a list of incomplete messages. The elements of the lists are records with the total amount of fragments in each message, the number of fragments received, the IP address of the source, the ID of the message and data. When a fragment is received, it’s stored in the appropriate position of the list and its control information is updated. When a message is completed, it’s delivered to ORRobot. The unique identiﬁer for each message is the join of source IP and message ID. ORRobot always invoke the triggerCallback() method of its tcmClient object when ORTcm delivers data. tcmClient launches the suitable function indexing in its register table and passing all data received. Now, normally this callback function unserialises data and uses it.

71 TeamChaos Documentation Information Sharing

6.2 Sharing Global Data

We know major limitations in dog’s sensors. Improving calibration, ﬁltering, segmentation, etc. we can get better processed images. The problems arises when robots don’t see the ball or don’t see the other dogs. If you have the best code processing images but you are lost every time or don’t see the ball, probably your team don’t play well. This section describes the global data sharing technique used in our team. In the object ORRobot there are some modules with speciﬁc goals. One of them is the Gm (Global Map) module which has between his objectives localize the dog using visual landmarks and a fuzzy Markovusing technique. Remember this module is running in ORRobot object, so it runs one step of its algorithm and lets the rest of the modules run too. Of course the time between the steps are very close for obtaining a quick a real response. In each Gm step we send towards the rest of members a data structure containing global data. Now, we have in the global data information about the ball (position and anchor) and information about the robot who sends the message (position and orientation). When a robot receives this global details, it uses it for fusion with his own data and improve the localization of these elements. As another example of communication we could see how we send and receive global data. void Gm::sendGlobalData(void) { MonitorData monitor; Position myPos, myBall;

stat_sent++; monitor.setCount(stat_sent); myPos.x = gs.getPosition()->x; myPos.y = gs.getPosition()->y; myPos.theta = gs.getPosition()->theta; monitor.setPos(myPos);

myBall.x = gs.Ball_g->x; myBall.y = gs.Ball_g->y; myBall.dtheta = 1.0 + gs.Ball_g->anchored; if (gs.Ball_g->fused == 1) myBall.dtheta = 0.0; monitor.setGball(myBall);

monitor.setLball_rho(lps->Ball_l->getRho()); monitor.setLball_theta(lps->Ball_l->getTheta()); monitor.setLball_anchor(lps->Ball_l->getAnchored());

72 TeamChaos Documentation Information Sharing

tcmClient->sendExternalMessage(0,GM,GM,*myteamis,TYPE_MONITOR, &monitor); }

In this method we ﬁll myPos and myBall objects and runs sendExternalMessage for informing all robots and PC’s listening (value 0 ). Note that we specify that we are Gm and we want to communicate with the same module at destination. In the last version we also select our team (for avoid collisions if the opponent runs our own code), the data type we send in the message and the pointer towards data. In the destination’s side, receivers only should register a callback function. Let’s see how we do it for receive global data: tcmClient->registerCallback(GM,TYPE\_MONITOR,&receiveGlobalData,true);

In the init() method of the Gm module we inform our architecture that if someone sends us a TYPE MONITOR data directed to us (GM ), we want that data will be passed to receiveGlobalData() method. Next, the following code shows a fragment of this method: void receiveGlobalData (byte dogSrcID, ModuleID dstModule, size_t len, byte *stream) { MonitorData monitor; Position pos, ball;

monitor.unserialize(&stream, &monitor, len);

[...] }

Normally, the ﬁrst part of all callbacks is unserialize data, invoking the unserialize method of a local object declared (in our example is the object monitor). In the rest of the code you must write the instructions which use data. In our case we update the prediction for ball and dog positions only if the values received are much better than local predictions.

6.3 Ball Fusion

We use radio communication to exchange information between robots about the positions of objects in the ﬁeld, in particular the ball. Information from other robots is fused using an original approach based on fuzzy logic, which is reported in [14]. In our approach we see each robot as an expert which provides unreliable information about the locations of objects in the environment. The information provided by diﬀerent robots is combined using fuzzy logic techniques, in order to achieve agreement between the robots.

73 TeamChaos Documentation Information Sharing

Figure 6.7: Schema used for information fusion.

This contrasts with most current techniques, many of which average the information provided by different robots in some way, and can incur well-known problems when information is unreliable. Our method strives to obtain a consensus between data sources, whereas most other methods try to achieve a compromise or tradeoff between sources. One of the major innovations of this approach is that it consistently maintains and propagates uncertainty about the robots own position into the estimates of object positions in the environment. Moreover, in contrast to many existing approaches, it does not require high accuracy in self-localization. The schema used for cooperative ball localization is shown in Fig. 6.7. The self position grids for both robots are shown on the left, the ball location grids are shown in the middle, and the result of the fusion of the ball grids is shown on the right. Darker cells have higher degrees of possibility. The two triangles represent the robot’s (defuzzified) estimates of their own positions. The yellow circles show the point estimates of the ball position. An example of our method can be seen in Fig. 6.8. The left window shows the ball grid resulting from the sharing of information. The three small circles near the bottom represent the point estimates of the ball position according to each robot (lighter circles) and as a result of the fusion (darker circle). The other two windows show the self- localization grids for robots 1 and 2, respectively. In this example, both robots happen to have a rather poor self-localization, which can be seen from the blurring of the two individual self-grids. Correspondingly, the individual estimates for the ball positions are relatively inaccurate, and quite different from each other. When intersecting the fuzzy sets, however, we obtain a fairly accurate fused estimate of the ball position (left window). Note that just averaging the estimates of the ball position produced by the two robots independently would result in an inaccurate estimate of the ball position.

74 TeamChaos Documentation Information Sharing

Figure 6.8: Combining two imprecise estimates into an accurate one.

75 TeamChaos Documentation Information Sharing

76 TeamChaos Documentation Behaviours

Chapter 7

Behaviours

7.1 Low-level Behaviours

Behavior-based systems are increasingly used in many robotic applications, including mobile units, manipulators, entertainment robots and humanoids. Behavior-based systems were initially developed on mobile robots, where complex tasks were achieved by combining several elementary control modules, or behaviors [9]. In most of these systems, however, arbitration between behaviors was crisp, meaning that only one behavior was executed at a time, resulting in jerky motion. In other systems, several behaviors are executed cuncurrently and their outputs are fused together, resulting in smoother motion during switching of behaviors [13, 40, 35]. The use of behavior-based system for more complex plants than a wheeled unit needs a framework which is able to handle several DOF. [27] uses fuzzy rules to control a 6 DOF arm, and [28] describes an automated mining excavator that uses concurrent behaviors. However, the complexity of these systems makes the design process very demanding: [27] uses 120 rules to perform a PickUp task, and [28] uses a neural network for behavior arbitration, thus giving up the readability of the arbitration rules. We follow an approach to building behavior-based systems that can be applied to control plants with many DOF. Complexity is addressed by a hierarchical decomposition [41] of the overall task into simple behaviors. These behaviors are encoded by small sets of fuzzy rules. In addition, fuzzy meta-rules are used to encode the way behaviors are combined together: this makes it very easy to re-conﬁgure the system for diﬀerent tasks.

7.1.1 Basic Behaviours

We deﬁne a set of basic behaviours, that is, behaviours that perform elementary types of actions, most of them common to all players independently of their role in the ﬁeld. In our domain, these include turning toward the ball, going to the ball, or moving to a given position. These behaviours constitute the basic building blocks from which more complex types of actions are obtained by hierarchical composition. The behaviours, which

77 TeamChaos Documentation Behaviours

are packaged in the HBM module, are deﬁned by way of fuzzy rules [38] [37]. The input space used by all behaviours is the local state provided by the PAM, which contains the current estimates of the position of all the objects in the ﬁeld. The output space consists of the velocity set-points < vx, vy, vθ > which are transmitted to the CMD module. An additional control variable k[] is used to indicate which kicks are applicable in the given situation. Thus behaviours are coded by using fuzzy rules of the form:

IF THEN

where predicate is a formula in fuzzy logic that evaluates a series of properties of the local state. This can be done without the need of an analytical model of the system or an interaction matrix, which may be diﬃcult to obtain for complex plants and tasks, as the RoboCup case is. It is also worth noting that the uncertainty in fuzzy system is taken in account as well. In order to give a more concrete impression of how behaviours are implemented, we show here the CutBall behaviour. For sake of clarity, all the rules shown in this section are given in a slightly simpliﬁed form with respect to the actual rules implemented in the robot. The CutBall behaviour uses three rules to control the lateral motion of the robot (action STRAFE) to locate it on the trajectory from the ball to the centre of the net. It also uses three rules to control the orientation of the robot to point the head toward the ball (action TURN ). This behaviour does not use the forward motion, and thus it is always set to zero (action GO). Finally, it controls the type of kick to be applied (action KICK ). If the ball is close enough and the robot is pointing to a safe area it tries to kick it using its both arms and the chest.

IF (posLeft) THEN STRAFE (RIGHT) IF (posAhead) THEN STRAFE (NONE) IF (posRight) THEN STRAFE (LEFT) IF (headedLeft) THEN TURN (RIGHT) IF (headedAhead) THEN TURN (NONE) IF (headedRight ) THEN TURN (LEFT) IF AND (freeToKick, ballClose) THEN KICK (FrontKick) ALWAYS GO (STAY)

The aim of this behavior is to keep the goalkeeper on the trajectory of the ball to the net. The rule conditions are evaluated from features of the LPS like Ball < ρ, θ > (distance and orientation to the ball), and produce a truth value between 0 and 1. Fig. 7.1 shows the truth value, or membership function, of the first three conditions as a function of the x position of the robot with respect to the ball-to-net trajectory. These functions are defined by the designer, and depend on how the robot should try to cut the ball. The consequent of the rules indicate which control variable should be affected, and how: the first three rules involve lateral motion of the robot, while the three following rules involve rotational motion. Each rule affects the corresponding control variable by an amount

78 TeamChaos Documentation Behaviours

Figure 7.1: Membership functions for the posLeft, posAhead and posRight conditions

that depends on the truth value of its condition: smaller or larger adjustments to the robot motion will be generated depending on how much the robot is close to the ball-to- net trajectory. Rules are evaluated and combined according to the standard Mamdani approach. Behaviours also may incorporate perceptual rules used to communicate the perceptual needs of active behaviors to the PAM (see section 4.4.2), by using fuzzy rules of the form:

IF THEN NEED