Target Tracking Control of a Mobile Robot Using a Brain Based Control Strategy

Changwon Kim and Reza Langari

IROS 2009 Contents n Previous research on Brain Emotional Learning n Introduction to the BELBIC n Mobile robot model with BELBIC n BELBIC target tracking model n BELBIC target tracking model with fuzzy clustering method n Conclusions n Future works

IROS 2009 Previous BEL Research n Cognition Science - Mowrer(1960): Two process model of learning - Rolls(1986): The mechanism of / application to the neural basis emotion - LeDoux(1995): Function of amygdala in emotional process - Balkenius & Moren(2001): Development of brain emotional learning computational Model

IROS 2009 Previous BELBIC Research n Engineering

- Lucas et al.(2004): Introduced BEL based controller to engineering - Mehrabian and Lucas(2005): Designed a robust adaptive controller via BELBIC - Chandra and Langari(2006): Analyzed the BEL based approach by using methods of theory - Shahmirzadi et al.(2006): Compared BEL with Sliding mode control - Lucas et al. (2006): Applied BELBIC to washing machine - Sheikholeslami et al.(2006): Applied BELBIC to HVAC system - Mehrabian et al.(2006): Applied BELBIC to Aerospace launching machine - Rouhani et al.(2007): Applied BELBIC to micro-heat exchanger - Jafarzadeh et al.(2008): Applied BELBIC to path tracking

IROS 2009 Brain Limbic System

n Amygdala - communicate with other cortices in limbic system - association between a stimulus and its emotional consequence - assigning a primary emotional value to each stimulus n - operates based on the difference between the perceived reward and the actual received reward n Thalamus - initiating the process of a response to stimuli, send signal amygdala and sensory cortex n Sensory cortex - distributing the incoming signals appropriately < http://www.morphonix.com/ >

IROS 2009 BELBIC (Brain Emotional Learning Based Intelligent Controller)

Learning rules ⎛⎞ ΔGSIRewA=α ⋅⋅max 0, − Sensed OFC Aii ⎜⎟∑ i Information Sensory SI Thalamus OFC ⎝⎠i Cortex ⎛⎞ ΔGSIAOCRew=β ⋅⋅ − − - OCi i⎜⎟∑∑ i i MO ⎝⎠ii + Internal signals/ Model Output AGSI iAi=i ⋅ Rew Amygdala OC G SI Amygdala iOCi=i ⋅ MO A OC =∑∑ii− ii

IROS 2009 Mobile Robot Control Strategy

Y xy, Mobile robot & control inputs ( tt) xv&= cosθ

yv&= sinθ v = δup dy  θ = ω ω = εφ v SI and Reward dx (xy, ) 22 X OR SI=( xtt− x) +( y− y) Components of Rewards 0.9 Gam*SI 0.8 Del*U Rew = γ SI +δup p Reward uSIMO 0.7 P =× 0.6

0.5

Learning rules 0.4 2 & 0.3 GAAOC=+αγδδmax{ 0,( SI−− 1) G G SI} SI 0.2 & 2 GOC=βδ{ (11− SI) G A+( δ SI−) G OC − γ} SI 0.1 0 0 10 20 30 40 50 60 70 80 90 100

IROS 2009 BELBIC Target Tracking Model

§ Target Generator - Multi targets problem § Error Analysis + - assigning a new target + Rew v BELBIC X x, y - making SI - u Target Distance SI Genarator Robot § BELBIC + / Angle - robot velocity command from SI error w and Reward Angular Vel + -

IROS 2009 BELBIC Target Tracking Model

Robot Trajectory x and y Robot Direction 12 60

10 50

x y 8 40

6 30 Angle[Deg] Position[m]

4 20

2 10

0 0 0 10 20 30 40 50 60 70 80 90 100 0 10 20 30 40 50 60 70 80 90 100 Time[sec] Time[sec]

-3 G and G x 10 A oc Robot Trajectories(Multi Targets): x vs. y 6 15

G A 4 G oc 10 A

2 5

0

Gain B 0 Y position[m] -2 D -5 C -4

-6 -10 0 10 20 30 40 50 60 70 80 90 100 0 5 10 15 20 25 30 Time[sec] X pos i t i on[ m ]

IROS 2009 BELBIC Target Tracking Model (fuzzy clustering)

n larger error needs larger robot velocity Desired robot velocity n a decelerated faster than b: deg of Cd 1 v v n d accelerated faster than c: deg of Cd2 max a b vH SI and Reward 22 SI=( xtt− x) +( y− y) Rew = µ1Cd1 + µ2Cd2 +δ up

uSIMOP =× v L c d

Learning rules e e e e e a b c d max e  GA = α max{0, µ1Cd1 + µ2Cd 2 +δ u p − GASI}SI  GOC = β{GASI − GOC SI − µ1Cd1 − µ2Cd 2 − δ u p }SI

IROS 2009 BELBIC Target Tracking Model (fuzzy clustering)

Fuzzy clustering rules Membership Function: Error Membership Function: Velocity

1 1

If error is 0 and velocity is 0, 0.8 0.8

then Cd1 is 0 and Cd2 is 0. 0.6 0.6

If error is 0 and velocity is 0.1, 0.4 0.4

then Cd1 is 0.1 and CD2 is 0. 0.2 0.2

0 0 If error is 0 and velocity is 0.2, 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1 then Cd1 is 0.2 and Cd2 is 0. Membership Function: Cd1 Membership Function: Cd2 1 1 … 0.8 0.8 If error is 1 and velocity is 0.9, 0.6 0.6 then Cd1 is 0 and Cd2 is 1. 0.4 0.4 If error is 1 and velocity is 1, 0.2 0.2 then Cd1 is 0 and Cd2 is 0. 0 0 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1

IROS 2009 BELBIC Target Tracking Model (fuzzy clustering)

§ Target Generator - Multi targets problem § Error Analysis - assigning a new target SI v x, y - making SI BELBIC X - u § Fuzzy Clustering Target Rew Genarator + Distance Fuzzy - clustering according to the error Clustering Robot and velocity / Angle error § BELBIC w Angular Vel - robot velocity command from SI + - and Reward

IROS 2009 BELBIC Target Tracking Model (fuzzy clustering)

Robot Trajectory Target at x=10 and y=12 Robot Direction 60 12

50 10

BELBIC with Clustering x 40 8 BELBIC with Clustering y BELBIC only x BELBIC only y 6 30 Angle[Deg] Position[m]

4 20

2 10

0 0 0 10 20 30 40 50 60 70 80 90 100 0 10 20 30 40 50 60 70 80 90 100 Time[sec] Time[sec]

-3 G and G x 10 A oc Robot Trajectories(Multi Targets): BELBIC with/without Clustering 8 15 BELBIC with Clustering ----- BELBIC only 6 10 A BELBIC with Clustering G A 4 BELBIC with Clustering G OC 5 BELBIC only G 2 A B BELBIC only G OC 0 0 Gain -5 D -2

-10 C -4

-6 -15

-8 -20 0 10 20 30 40 50 60 70 80 90 100 0 5 10 15 20 25 30 Time[sec]

IROS 2009 Conclusion n BELBIC mobile robot tracks the target successfully n BELBIC with fuzzy clustering method works for target tracking n BELBIC is a temporal learning method (each time the robot learns appropriate gains) n Development of a higher level learning method is needed to achieve autonomous mobile robot

IROS 2009 Future Works

High Level - Long- Term memory Multi-objective - Planning - Mapping Decision Making (AHP)

Low Level - Target tracking - Obstacle avoidance OFC High Level

Amygdala Low Level

Development of mobile robot navigation structure for Open World Model: Multi-objective Decision Making Method: Analytical Hierarchy Process (AHP)

IROS 2009

Any Questions or Comments will be appreciated

Thank you !

IROS 2009