Connectionist Architectures for Control of Manipulation Robots by Feedback·Error Learning Method

Yu goslav Journal of O perntions Research I '1 (J9V1), Number 2, 'l35-2a:l

•

CONNECTIONIST ARCHITECTURES FOR CONTROL OF MANIPULATION ROBOTS BY FEEDBACK·ERROR LEARNING METHOD

Dusko ~1. KATIC, Miomir K. VUKOURATOVll : Robotics DC//arlrll clIl , Miha ilo Pllpill lnstitnte Bclgr-ade, 11000 Beograd, Yllgo slavia

Abstract. A major objective in this paper is the applica tion of highly efficient cc nnectioul st architectures Cor fast and robust learning of dyn amic relations used ill 1" 01JOt contro l at the e xecuti ve hi erarchical level. T wo types of neural network control s t ruct u res are considered : a single-layer neural network and a multilayer p erceptrou. The proposed connectionis t learning models are applied as a Iorm of intelli gent Ieedforward robot control in the frame of decentralized control algorjtluu with feed back-error learning m ethod.T he final result of this a p proach is a trniueble r ob ot controller with e xcellent lear ning propert ies. Effi ciency and verifi cation of the proposed algorit.luus through sin mla t .ion e xamples of robot trajectory tracking is shown .

1. INTRODUCTION

The powerful development of fl exible mannfacturing systems with high and com plex demands for all com ponents of the production process, leads toward the design of intelligent m anipulation robots [I, 2). We can define these robots as au tono mous machines capa ble of learning, making decisions, fault tolerance, ana lysis, etc . Being robus t and adaptive to internal and external disturbances , problems as com pensa ting for uncertainties, diagnosing failures, identifying failed components, and recovering from errors can be successfully tackled by using these machines. Such robots also, can perform anthropomorphic tasks in a n unfamiliar of familiar working env ironment. T here has been a significant effo rt in making rob ot more intelligent by integrating advanced sensor systems such as vision, tactile sensing , etc . But, one of the major and ultimate ste ps in robotic research is the formula tion and developmen t of intelligent control algorithms whi ch can fur ther improve t he performance of robotic systems, using control st rategies generated by human intelligen t fun ctions such as percep tion, association, reasoning, generalization or learning. The basic problems for the classic model-based control of manipulation robots are the state variables-dependen cy of the robot dy namic model, the expressive D. ~1. l\:at ic, ~I.I \. . Vukobretovi c co u pling IH' t. ween robot su bsystems , coping with s truc ture d a nd u nstructured u n cerraim.ies and I ime-dependency of ro bo t pararueters. It is known t hat. no ne o f ti lt' classic robot CO lit rollers ca nno t provide d es ir able solutions 1. 0 t.lir-se problems, I ll ' c all s ~ traditional control laws are, ill most cases, based a ll models with in conr pl l'te iuform ntion and partially known o r inaccurately dt·fined parruueters . Also , the classic a lgoritlu us a re extrem ely sens iti ve to the lark o f senso r information and uu plauued events a lld unfamiliar s it. ua tio us in the robot working eu vi ron men t , The ro bot perforruauce is not able t.o capt ure and ut ili ze past ex per ience a nd available human ex pert ise, All p revio usly iueut ioned fa ct s a nd exa m p les p rovide a m ot iva tiou for robotic iutelligeut cout rol a nd em phasize t he necessi ty t hat ellicient robot ic iutel lige ut cont rol ruus t be haspd 011 lea rning , geuerulizatio n a nd self-o rga n izing ca pahil it. ies.

T he classi c adapt.i ve a nd lion-ad aptive control algorithrus com prise robot. CO Il Irol pro blem durin g ex ecu t io n o f s ing le robot trajectories without co nsid ering repet it.i ve mo t ion . 1I t.'IICl't ill terrus of learniug , a huost all rnauipulat.ion robo ts a re m emo ryl ess, III t hi s way, tit" previously a cq uire d ex perience about dynamic robot 1II0 dl'l a nd cont rol algorit luus is not a p plied ill robot. co nt rol synthesis. It. is expected that. using a training p rocess whi ch repeats a cant rol tas k a nd records tile results accum ulated ill t he ent ire pro cess would stead ily improve the perforrnnuce. Also , s ta te varia bles-dep endency o f robot dynam ics may be so lved hy learning and stor ing t he solu tion. while time-dependency of robot parnmeters requires an Oil- line lea rnin g a p p roach . If learning co nt ro l al gorithm o nce learned some m ovement, it. co uld co ntrol quite diflcr cut am] faster m ovement using genera liznt.ion properties o f learnin g a lgo rith m . hence, OIl P o f the primary goa ls ill intelligent co n trol of m a nipula ti on robots is the additio n of learning a nd gcncralizot. io n ca pa bilit ies to the classic lion-adaptive a nd adnpt.ive cont rol a lgo ri t h ms. The recent research reports a nd extensive simulation studies ca rried a n t a ll mod els containing ucural networks have demons trated a n ability to identify and control sop hist icated inanipula tion robots [3- D). From a systems theore t. ic po int o f view 1 we ca n say t. hat in ult ilayer neura l networks used in ro bot co n t rol ill 1II0 St cases rep resent, static nonlinear Illappillgs as a s pecia l part o f the pattern recogni ti o n p ro blem . III this case t.h e patterns to he recognized a re the s ig na ls o f "c hange" tha t m ap ill "con t rol action" signals aiming at. desired co ntro l go a ls. The neural network cont ro ller s hould recog nize a nd iso late sig na ls o f "c ha nge" in real-t.ime con ditions, a nd il~illg lea rning hy experience a u.l generalizu t.ion propert ies, to control efliciently system behavior. Through the truiuiug p rocess, the m odel uucer t aiuties are eliminated, and thus , a neural net work ser ves as com p-nsat iou too l ill co ntrol systems. III t his pa per, o ur purpose is p resenta tio n o f new robo t control lea rning algo rithms with fast a nd robust learni ng properties usi ng s pec-i al co n nect ionist (neu ral net wo rk] a rch itect ures. The m aj o r co ncern is t he flp plica lillll o f neural networks ill robot co n t rol a t. execu tive hierarchica l level ( 1I1lll iU II c() 1I 11'ol pro blem) for learning inverse dyna mic m odel of ro bot. mech anism ill the cas e \\"11t' re exact robot dynamics a re genera lly II Ilk 11 0\\'11 . C O Ill lf' c t i ( l ll i ~ t a rchitecture..'S fur cont rol of rnnnipnletiou robots . ..

I Also , OUt' of t.h e main ideas of this paper is to accomplish coexistence of struc turcs that are developed for non-rcccurcut single-layer and muh.ilayer networks, using th e frtuuework of a decentralized co nt rol algorithm (a well-known classic ro bot cont ro l al gorithm)[3].Several neural network models and learning schemes were recently applied to learning of robot dynamics. aile main distinction between these methods is ill the extent of the kn owled ge a bout dynamic models which is used ill design pro cedures. Some methods (siuglc-Iayer neural networks } lise in design procedure co m plete availa ble information a bout robot model [7- 9] . At the ot her side of dyna mic co nnect ionist approaches are methods (m ultilayer percep trons ) that lise "black-box" approach in design of neural network algorithms for robot dynamic cont rol [- 1- 6]. III this case neural networks ca ll be used as very general coru pu tut.iou models. The ot her iruportaut feat ures of this new proposed cont rol st ruc t ure is fast convergence properties due to fun ctional decomposition of system dynamics and new learning rules based a ll recursive est im at.ion methods, Using repetitive exec u tion of the working task and new leamiug algorithms with feedback driving torque error, redist.rihut.ion of feedforward and feedback control are accomplished, that results in fast. system response and ellicient generalization . Training and learning by proposed neural network architectures call be accomplished using off-line and on-liIll' approach. Elliciency of proposed lcaruing algorithms will be shown using data about ind ustrial robot Ui\IS-2 .

2. IWIlOT C:O NT IWL I'IWLlLEi\l AND CONNECTION IST SO LUT ION

III contoi u pornry robotic sys te ms, t here is a need for more fl ex ible and robust robot contro llers ill orde r to take fu ll advantage of t he inherent fl exibility and versat.ilit.y of runuipu lat.ion robots. The dilli cultics in solution of control problem arise from various asp ects. The o ne of the IIi 0St. import.ant problems is high nonlinearity with expressive co uplings between subsyste ms. Based 0 11 well-kn own equations of rigid body me chauics , a dynamic model of uranipulation robot ill the absence of Irict.ion and other disturbances [determ iuist ic 1I1 0del ) ca ll he written as

p = J ('1 "j , ij ,0) = 1I ('1 ,0)ij +"('I ,'i,0) ( I )

or !' = J ('I , 'i , ij , EJ ) = II('I, 0 Jij + I?C('I , 0 )'i + !I('1 (0) (2) where P Err" is the vector of driving torques of forces ; H ('1 ,0) : R " x 0 _ rr"X" is the inertial matrix of the system; " ('I, 'i,0 ) : R " x rr" x 0 -R " is the vector which includes Cent rifugal, Coriolis and gravitationa l effects; C ('1 , 0 ) : R " x 0 R'' x R " x R " is the matrix which includes Centrifuga l and Coriolis effects; !I('1 , 0 ) : R'' x e -:. R " is th e gravitational vector ; e E R "t is th e system parameter vector ; 11 is th e 1I11111hcr of c! l'!!rC'p of freedom ; ul is the number of syste m parameters. A common aud classic wav for robot cont rol represents loca l PI D regula tors • :l:J S D. 1\ 1. Katic, ~ 1. K. Vukobratovic

for each

II = li n, = - f l ' Pe - f{ De - f{ f Je til

n x rI where 11 E n ." is the cont rol input: lin) E n,1I is the feed hack co ut.rol; /\' P E Il. is the mat rix of local position feedback gnins: f{ D E R " X" is the matrix of loca l velocity feed hack gains; 1\"/ E n ." xu is the matrix of local iutegral feedba ck gains; e = 'l>: '/0 is 11 11>feed hack error (s E R"); 'l a lld '/0 arc the real and nomi nal internal coord iua tes (" E H." , '/" E H." ). However , t.his control law is 11 01 adequate for advanced iudust.rial robots wit]: t.h e requirer ucn t.s for high precision and speed ill a com plex working environme nt . The inlluen ce of co uplings between t he subsys tems is substa ut.i a l, a nd we hav« t.o include as a sol11 1ion "dynauiic" co nt rol [l O] whi ch takes the dynamic 1II 0<1 t>1 of robot ruechu uisru ill cont rol synthesis as Iorrn of feed forward CO ll i rol. 011 the basis of the above, In>ca ll apply the deceutrnlized cont rol algorithm [I ll] :

(4)

where Ill! is t.li e 1I 01l,ill ,,1 ccui.r alized f,'"drorwa,,1 control whi ch is olf- Iine syn t he sized using rll<' illt"gral robot 1I 10dei (mode l of mechanism with t.h e model of robot actuators ). However, ill procedure of cont roller design , wo ha ve to co pe wi t.h s t.r uc t ured uu cert.aint i.», [iuacrurar ies of 1II0dei pa ra m eters ) , uust.ruct.u red uncert.ain t.i es ( 1I 1l mod el led high frcqueucy dynr uuics as s t ruc t u ra l resonant modes , neglected time delays , actual or dyn amics , s.uupliug ellocts. etc.) a nd measurement noise. Also , rune-varyiug nat lift' of robot parameters a nd variability of robot tasks represent additional di ili cuh.ics for the CO ll i 1'01 system . III t his case, t lre classic non -adupt.ive a lgoru.huis a rt' Hoi robust enoug h , because these a lgo rith ms COII I)ll'IISa le only a s u ra l] part of tilt' rueutionod uucer t. a iur ies . Hence, a more suit.able approach would hl' the one usi ng a dapti ve co nt ro] t eclm iqucs. The adapt in" co ntrol t.ec h uiqucs ill robot ics ca n hp a pplil'd as a Iorru of the well-known :\ lodl'1-Refl'rt'lI fl'd Adapt ivc Cont rol ( ~ l l t AC) or Self-Tuning method [I ll], with t he possi hiliry of .uln pt at ion ill fecdforward or feedback loops. III couc lusiou, the classic adaptive contro l t.cc hniqucs rohotics an' e1 rPf tive to co m peus a te s t. ructun.. d uuccrtniuti cs, hut. ill t ile presence of senso r dat.a overload , heu rist ic senso r iuforuiat.i on , lim its a ll real-time npplicahility and wide interva l of unst ru ct. ured uuccrt.ainties , Llre a pplicatio n of adapt.iVt'° control is 11 01. sulllcieut for high-quaIiI Y perforIIiiiIIce. Therefore, a solut ion to the robot cont rol problem will likely need to co rnhiue classic a pproaches with new learning a p proaches ill order to a chieve good perfor nr auce. For t ilt' robot co nt rol l,ruble'lli and learuiug we can idelltify three Blain paradigms: a) lter.u.ive a u.dyt.i c«l met hods: b) Tabular methods; c) Con uec t ionist methods, I Conuert.iouist methods as 1II0 St. promising approach for learning control pro vidr- the uupk-ru cnt.u i.i ou tools for complex illpll i / olltpllt relations of rahat's ely naruics and ki nematics. O ne of t.h e m a in goals of dynamic learning methods is 1. 0 find solutio n for t lu- robot iuverse dynamic problem. Let us expla in t.he inverse dynamic prohlor u of ro bot cont ro l ill a com putat iona l frruuework . There a re ca usa l relat ion Iwt Wt,t.'1l ro bo t d rivi ng torq ue and till' resulting ro bot II IOV('lI lPn t coo rdi nat es. Let 1'(1 ) denote the t im e history of driv in g torque and '1 (1 ) denote the rime hist ory of tilt." robot iuu-ru al coord inates during the traj ect ory. J\ lso \ \'C' ca n denote the ca usa l relu t.i ou between P a nd 'I usi ng the fuucti onal F , i.e . F( 1'(· )) = ,,( .). If \\'(' want t.h c ro bot to t.r acks desired trajectory '/.I , t.h e problem 10 generate a dl':-:in'd driviug torque / )" which realizes 'l . is equivalent to finding a ll inverse map ping of t hc fu nct.i onal F. TIlt' connectio nist approach lIlay, ill prinr iplc, solve the pro blem of variable-coupling complexity aud st.a te-dcpendcucy of robot dynami c model, because neural networks t hrough t he process of traini ng ca ll a pprox imate in put/ oli IIHll, IIHlppillgs. III this way co nnec t ionist st ruct ure a!" part. of deceutrulizr-d fecdforwarrl co nt rol law C

:l. coxi'I CCTIO i'l lST LC,\ IWIi'I( ; STIW CTlI ltCS • AS PAllT O F FCCDFOl tW,\ltD ItOBOT COi'l T IWLLEltS

~ .l. SI NCLE- LAYEIl N EVIl AI. :" 1.1 " '0 11 1< C ONT IlOL S TIlVGT VIl E

In the first case. it is proposed that sorue, wh atever small, a priori know ledge of I ln- robot dynami cs is a lways a vai lable. "'l'l'X I "" ' l tha t single-layer ueura l nctwo rk model usin g t.11 (' a prio ri knowledge about ro bot dynamics , will signific a nt ly im prove performance of robotic system. IWIIl.:e, I he p roposed neural network m odels ca ll he regarded as exa m ples of the a ut tl lIOIJIOliS driving torque generator ( Fig . 1). This c o n n~ c t io n i s t s truc t ure is com mo nly used as part of fe"drorward controlle r in D. xt. I~,n tjc. xt.K. Vukobrarovic decentralized control a lgorit hm . In this case, the feedhack cont ro ller se rves as a rohust cont ro ller to achieve low errors and perform high-quality learuing, because the feedfo rward contro ller al one is not sufficient for a ccurate tracking.

• qd / q jQ NEURAL P Uf(r U NETWORK f R0 SO T q qcl + + / Ufb Ufb

£ r + PI D , ) . • + REGULATOR £ ) • ,

Fig. I . Deceu t rnlized cout rol with cou uectiouist Iecdbeck-error learnin g

Trainiu g all d lenrning hy proposed co nnectionis t. s t.r uc t.u re is accomplished ex elus ively in a ll-line reguue by feedhack-error learuing method ti l (Fig. I ). This m ethod is exc lusively o n- line method for robot cont ro l, but this control struct ure provides all iun-runl teacher so that the control sc he me works in an unsupervised urauuer, because we have no externa l tea cher ill this casco The adjustmcnt of the network weig hts during t he real-time control by feedback-error learning is m ore co nvenient th em ot he r learuiug structures as gcucralizcd or specia lized learning [4] . Tlu- neurul not.work model as fixed nonrecurrent siugle-layer network , generates uecessary driving torques ill robotjoints which is ex pressed as linear (weighted) S lI lII of a set. of specific li onlinea r dy namic functions: .. f'r = L wij ll;j ('1 ° ,8)" j + Wi ,"+lhi('1?,'i?,8), ; = I , . . ., ,, (:;) j=l w Iiere 1,i0 ' E n."···ISJOlIIl (I f1VlII" g torque generatorI I>y 1I ('lI r,, 1 network ; tVij - adaptive weighting factors of neural network ( IV E n:,.u+l = lV(w'J)). TIl<' o ut pu t of a rt ifi cial ncuron is hounded with liwar t hrr--hold fu nction (acti vatio n function of neuron) due to real physical const raiuts . Using equation (fJ) and Connectionist architectures for control of mauipulat.ion robots . .. 241 according to integral model of robotic sys tems. decentralized control with learning has the next form:

tl i = Aijqi - [J u rj ; - CiiPp · - 1\ Puc; - K D iie i - IUii Je, dt , i = 1, .. . , 71 , (6)

where Aii I Bu I eii are constant parameters of robot actu at ors model. Training and learning of single-layer network is accom plished using error correc ting method based On well-known DELTA rule [nJ(W idrow- Iloff least mean square algorithm):

dWij o. T = D;j E, = Dij ( Pi - Pi ) <1/ O = Dij ( Pi - t Wij Hij (I/ , eJij/ - Wi,"+1"i(g O, li ,e)) (7) ; =1 where T is t.h e learning t ime constant: E; - driving torque feedback error; Dij selected part of dynamic robot model (lIij(If, e) or " i(If ,Ii, e )). Approximately, we can ca lculate driving torque error in implicit way utilizing robot actuato rs model and position, velocity and ac celeration errors ill form : T

:l.2 . M ULTI L AY ER PERCEPTRON CONTROL STRUCTU RE

In this case, basic principles of training and learning accord ing to Fig. I are t he • same "" in the previou s case. Now, the topology of proposed connectionist struc- t ure for robot control is defined by four layer percep t ron with sy mmetric sigmoid 0 .1\'1. Katie, 1\-1. K . Vukobratovi c

functi o n as activation functi ons in ho th of hidden layers . The network has inpu t layer wit.h :1" Il l~ \l I" OIl S and ou tput layer wit h 11 neurons . This number of neurons ill approp riat.e layers is deterruiuatcd according to number of dr-gree of freed om for standard robot. co nfig ura ti o n. The activation functio n for input a nd outpu t. lay er is ident.ity Iuuct.ion. The number of neuro ns ill hidden layers is dcterruiuated by simula t.i o n ex peru uents and ex perie nce ( 121l neurons ill lirs t. hidden layer; 6 11 neuro ns ill second hidden layer). The neural network with proposed topolouy of fixed uou rccurrr-n t ruul t.il nyer network generates necessary drivi ng torques ill rc hot joints as no nlinear Ili appillg of robot desired internal coord ina tes. .... eloritics and nccelerutious: i =I , . . . , 1I . (9) where Pi E n.n is j oi nt driving torque generate d by neura l network ; wJt ndapt.ive w"illht illil fa ct o rs between neuron j ill «-t.h layer and neuron I.: ill b-th layer; II no nlinea r lIIappi ng. Acco rdin g to integ ru l m odel of robo t.i c systems, dccent.rnlized co ntrol a lgorith m with learning has the next form:

Ii i = fd IJ d , Iji , Iii , P ) - 1\ I'u ei - 1\ D i i i i - IU i i JCi

where Ii is the nonlinear m apping which describes nature of robot a ctuator model. Training and learning of prop osed co nnecti o nist s t ruc t ure can be a ccomplished usi ng well- known bnr]: propagation algorithm [11]. III the pro cess of training we • ca ll lise two ty pe of ou tpu t error for ha ck propagntion a lgorithm. T he first type of error is feed ba ck co nt rol s ig na l

; = 1. ... , 11 ( II)

where c~I' E n." is t he out put error for hack pro pagati on a lgorit hm . Bu t , in fa ct when we cons ider intcgrnl m odelling of robo t m ech anism with model of robot act uators , feedback co ntrol signal is not output error for neu ral netw o rk . Thus , we have to calculate driving torq ue error sig nal .. .. e,b·" = lJr - Pi == fl l ej+ bl t i + c l U fb i = I , .. . , lI, ( 12 )

II . . . when> pr c: It is the real robot driving torque; (l' E It," 1 b' E R" , r' E 111l I are ronsl.aut. parameter» of robot. integral model (this m o del is val id for robo t. DC ac t. uators) . Although the proposed pure or na ive neural network approach wi thout knowl edge about robot dy umuics may he promis ing , it is im portant to notice that t his a pproac h will no t he very practical because of hig h dimensio nali ty of in put-output spaces a nd long learning timc. For example, with most rnanipulat ion robo ts hav ing G d.o .L, we have 18 input varia bles a nd G o ut put varia bles. Also, the num ber of t rajecto ry pat t.er ns "l' (i n pnt a nd ou t put variables) may he very la rlle ·(I O- IOO) if we want. to cxaru ine the whole wo rki ng spa ce .Hence , for t he robot training it would be necessa ry to present u l Jl8 saulpiN; , i.e. the training hy pure connectionist Connect.ionist arvhit cctnres for control or mnuipulntion rohou, .. . I m odels would require a neural net work of irupra ct.i c«] siz(' awl unrr-asou.rhlo 111I11I ber o f rcpetiti ou cycles. Therefo re . we ca ll co nclude that t.h e nai ve con nec ti o nist approach es are o nly appfirnhle for t.h e low-rlimen siou robotic SySt. '> IIl S. The airu of proposed approach for design of co u noct.ionist rah at. co ntroller is t.h e leaving of co nt 1" 01 :-; yntllt':-; is without a priori inform at.iou about rahat dynamics. hence, it is very import.aut to li se all the available iuformat.iou about robot dyuaru ic but o nly ill geuerul a nd s pec ific Ionu, The general knowledge for t.ha t. purpose is co n veuic n t.ly in corporated int o the st.ruc t.ure of network. The way of an.aiuiug ab ove goal s is a dccoruposit.ion of robot. dynamics on sim pler robot dynamic reln t.i ons . III t.his way , instead or using single neural network, training a nd learning is a ccomplished hy several neural su bnetworks which have s im ple r input. / output relations t.hat ena bles significant. re d uc tion of learnin g lillie. III t.h is paper, ;;,3F-2SF"' ducoru posit.ion (decoru posit.ion of three-vector fun rt.ion into t.wo two-vector s ubfu uc t io ns) is proposed. Namely, we call see that. in robot dynamics seve ra l terms can be ideut.ified wh ich have a di stinctive Iuuct.ional dependency, Exactly, basic robot m odel ( I) ca ll he decomposed into two terms:

first. term 1l (" J'J )ij o r F, (" , ij ,0)

second term "(" ,'i, 0 ) o r F2(,!,'i ,0 ).

Wit.h this type of decomposition, instead of rnultilayer percept.ron with 311 , input. values , we have two multilayer perceptrons with 2" inputs and II o ut p uts for approximation of mapping FI and F2 :

p .NNI F (NN 'ab .. ) • , - ' I WjJ: , '1 d , ([.t .=1,.. . , II ( 1:3 ) p .JI,.'N2 r ( N N2"b . ) · , - ' 2 Wjk , '1 d , fJ d .=I I •• • , II (14) p .N N t + p .NN 2 • Pi •• .=I , ... , II (15)

where F't is a nonlinear mapping for first perceptrou NN I; F2 is a nonlinear map ping for second percept.ron N lV '2; P jN N I and p /V N 2 are parts of robot dynamic model gene rated by perceptron s NN 1 and N N '2 ; WffN 1f1 b and wJ\.N2f1b are weight iug fa ctors for percept rous NN I a nd NN 2; P; is driving torque at. the out p ut of connec tionist structure. The topol ogy of perceptrons N N I a nd N N2 a re determinat.ed using sim ila r activation fllll cl ~on s a nd principles as in previous case ( 1I0W we have: input layer - 2 11 neural units ; first hidden layer - 8 11 neural units: second hidden layer - 411 neural uni ts; ou tput layer - 11 neural units). Training or both per ceptrons is ac com plished sy nch ronous ly by feedback-erro r learning m ethod ( Fig. 2) The feedback error s igna l o r driving torque error signal are transferred as o ut p ut backpropagation error to bot h of perceptron outputs, The backpropagation a lgorit h m ca used a tremendous breakthrough in the con trol a pplica t ion of multilayer perceptrons . aile of the major drawbacks of this method is its slow convergence. Starting from a random initial state, the path

I 0 . :\1. 1 ~ .1.t i c . l\ 1. 1 ~ , Vukobrnrovic

/ iid ~ ' NN-I I qd f • l q + r uff U / ~ f + .J B 0 ! 0 T q • / + NN-2 qd plIO r I / U f b

• + U E .r f b PI D . , ULATOR E • + •

Filo; .1. . '':IF·'.l SF'' Decompovit icu co nuectio ni»t » t ruct u re with Ieedbuck t'!TOI" learning to the glo bal m in inunu is often s t re wn wi th local rniniuunn, ca using oscilla t io ns around ravines in the weight spare. In this paper, o ur intention is that in stead a ccclerntion of s tandard hack pro p agation a lgorir lu u using standard methods from numerical ana lysis , consider the prolrleru of adjusting the weights of internal hidden units as a problem of est.i mating parameters hy well-kn own identification method - Recursive Least Square ( R LS) method [12J. Using these methods with time-varying lea rn ing ra te y ield 10 benefits for learning sp eed and generalization ill co m parison with standard back pro pagation algorithm . The proposed new a lgorithms are based on previously defined -l-Iayer decom posed connect ionist st ruct ure ('' : If- ~ S F'') with a ppropriately defi ned parameters of network. , The rua in forward network relations ill process of training are descri bed ac cordi ng to next ex pressions:

Forward Relation» - RLS Method

s 2(k) = lV 12jl (k) ( 16)

, 1 2 o ~ ( k) = (2(k)) - 0.5, ,, = 1•... , L1• oo( k) = 1 ( 17) I + exp - ". . Cnu uect ioll i!' l nrr-hitcctures for control of lIliU liplll"t ioIlIOl wls . .. I ., 3(k) = 1\'23 02(k ) ( 18) I o~(k)= I ( ·'Ull- O." o =I ,. .. , L2 , o'o'l(k) = I ( 19) +exp -"I . .,·'(k ) = I I' '' ''' ,' ( ~' ) (20)

!I(k ) = s 4 (~. ) = l "vNI' C = I, ... , II , 1= 1, ... , II , l' = l or 2 (21)

wh ere .,2 U'), ,," (k ), s4(k ) a re the ou tput vectors for linenr partsof layers: 02( k), 03U') a re the out put vect ors o f the hidden layers; 1\' 12 = [w;,?xLl , 11' 23 = 23 11/3.' [34 I . I ' f f II -r: [W L 1+ 1 x L";!1I = (l'L 2+ 1 X'I I are tie \\'cig rung acto rs 0 t It' ayers; 1\ - = {w,I,?x Ll is th e inputs ill the network ( robot iuterual positions, veloci t ies an d accel era t ions - 111 = :1" + I ); !I(k) is output of network. TIll' aiui of es t.i m ution is to defin e opt imal values for mntrices 11' 12, 11'23 and 11' 23 using II 10dd s of linea r syste ms a ccording to cquat.io ns ( 1(j), ( 18) and (20) . In application of this method , problem of specifi ca t ion of desired sta tes a nd errors in hidden layers is arose. Using new solut ion by taking account of the relationship between the standard backpropagation a lgorithm a nd present m ethod at the last layer, hi gh-elli cient a lgorith m that propagates the learning errors a t the 1

L co,.IIillg Rilles - Rl.S Me/hod

o?n~,,( k) = max 10Z

a =l .... , IJ)

tl=l ,. .. , L J

whore AI , ..\ 2,).:1 an' the appropriate forgl'ttillg fa ct ors.

Init ial couditious fro \\,pight ing factors aft' generated hy uorural distributiou wit It dilrt'rPlil random numbers:

Abo , iuit inl coud it ions for cova ria nce IIIat rin's Pare given ill I he following IonII: 1,1(0) = el l", ; 1''' (0) = e" I LHI

'I. S J ~ IU L AT ION E XA ~ I P LE

III t.his -cct.i ou , sin urlntion exa m ples are give» to verify the prop osed counce t.i onist al~orit.llIlI s co ru pcus at.i ug tilt' syste m uuccr t.ni ut.ics. T ile m auipulation rob ot 1I ",'d for t lu- siruulat.io n i. a cyl ind rical type industrial robot U ~ IS -1 [10] ( f ig . :3 ) wit h n.a.e.r.

III the learuing phase . robot t.ra unng is accom plished using 1Il0 VClIIPIlt. from point A wit.h int ..rnal coord ina tes 'I E {0 .G ;0.05 ;0.05 ;0.;0.;0.} to point £3 ('I E { 1.1;0.11;0.1;0. J,O. I;0.1}). T ime duration of m ovcrueut is I = Is with triangular velocity prolilo. The Pill f, '"dhark control was chosen with following va lu es for local f,'{'''hark ga ills: 1,1' E {111.:GU::i .;41G.;41.; If,1 .;5J.}, I,V E {·I7 .;:l:I.;1J.;1.;8.;1.} . \rc' know t ha t exact II wi\slIring of the link inertin and position of mass ce ntre is very di llicuh.. 1I 1'lIce- ill siumla t. ion ex pcriu u- nt.s, tilt, ruodel uncert nint.ies are defined II)' par.uuctric d ist ll rh alH'l's with approximately 20% va rint.ion fro m nor uin a l val ues for link 11l:l:-'S and III OIlIt'lIl of iucrtia). 011 Figun-·1 and:; posi t ion er ro rs (trial 10 a nd t.rial 1;,0 ) for t.h e lirst and seco nd dq;rl'l' of Ircedoru ill the case of singlc- Iayc r neural net work are gi ven . A s WI. s-c froru tlu-sc.. figures, with repet.itive trials tr acking errors are decreased and learning i~ a ccomplis hed.

III si ruulatiou experin u- nts we have applied p u re co n uort inl1i~t and deco m posed 'I:J F- :lSF" network st ruc t u res. III all simulation ex periru-uts, ( l lll vergell ct" criterion (exactly tot.al eporh sq ua re error) was J, = 10 Nm, '1'1 ... learruug of robot dynamics is accomplished through trial-and-error approach , when we ill successive epochs of truiuing pr l' ~f'J1 t sa m e 1rajcct.ory patterns. Cun ucctioni s t a rchit ectures for con trol of umuipulation robots . . .

I •

•

y • I'

III siruula t iou cxperi rueut with hackpropagation a lgoritl n u, the learning rate for a ll layers is '1 = 0.01. In t he case of I{LS connect ionist. a lgorithm , covariance m a trices have folowing in itial values:

1"(0) = 1000; 1'•-(0) = 1000; 1'"(0 ) = 1000;

To get a com pa rison of pure connectionist st ruct ure a nd ":!F-2S F" decom posed connectionist struct ure , some sim ula tions expe riments with standard back propagation algorir luu under sa me coudit.ions were performed. Fig. () shows the ronvcrgl..'lI CP results (total epoch square error d uring t inie) for both connertionist strurt lire. The resu lt shows bet ter learni ng of dynam ic robot model alld significant

reduct i on o f lpa rllillg t i rn e for decomposed c onue c t i ou i s t structure.

'I h CUIlV -rgenc.. results of hack propugatiou algoruhm WIth ItLS algorithm ( F ig. I) III rl.e case of o n-line learn in g,

We ".u conclude fro III results , that s pecia lly with new RLS connec tionist m ethod \\p can achieve fast lea rning of robot dy namics. In F ig ure ' posit ion error for firs t d .o.f, iu t he case of tr ajectory tracking with hal k propaga tion algorit hili and ItLS ruet. hod is shown The fi g ure shows the Iirst and lOt It tra in ing Iria l. It took about}» for one trial, while sampling per iod was I filS . T he lig ures s ho w that with repet it ive rrials tracking erro rs using il LS method are considerably decreased and in this way lJ i;!, hly ellicieut. robot dynamic lea rning n. i\. t. (, ,,t ic. M . K. Vukobrnro vic

0 .00.>0

,...... '0 00020 -- - lAw.. ftO. ')0 , ,

O.O(JlO 0 <::"" I e ooco n: 0 I n: -(1.0010 I n: I ... I -0.0010 --- I

-0.00.10

0 .20

F i ~ ..1. l'o:,

0000)

TRlAL. 110 . 10 00002 -- - TRlAl no. I ~O

0 0001 I , ::i I I - I ooooo I -- c:: ... - - _ I o 0: 0: - 0.0001 w

- 0.0002

-0.000)

1 1..-. - 0.0004 0.00 0.20 0.40 0.60 0.80 1.00 Co uu cc tiouist architcctun..~ for ('011110 1 of runuipulation robo ts . ..

30C 0 , • • ,• • -- • " , <' • - < ~ - C • • • e, ~ , • lode :: #1· ...... ,- • .... ·- ou· cecorn ocsrtro n - ' ,- r • Alg o' " ,- • l~ (1 e' '' O IT'~ (. ~ t lO'l -... . c • • • • ,- 1 <: ,• • ~ · < - r • ·, • .. • - , • - • • •- , r • - • .. • - • • -< • - ,• - • , • - ,, r . • , , • • • ,• , , • , f I c v • r • , • .... r 1'\'1 r.r- ~ .. L ..... ~_ -, -J . 10 1 .'. 25 0C ', IJ :::- r;. " ~ • t.--.... .J ..", ..." _ , ~ - or ".." ...

• , ##-.- • • ------. c \ -- -- OJ: : • \ e~ : 10 ~"' 0~ t.. :,t.. ";" I O~ A ~ G J R I1 H ~ • ~_ s Al.GOR h ... -~ ---# - \ -c, t: .-. -' - - •- - - - - • -., --, - -< '---- - • \ .. ~ ,#- .. ~ -_ . - -,

, - - - j , .-- :: rJv:_ ._ 0'_, ~ -- f. DC 2: 0: t ' ,J" j ::'_ :-_ ;; '...I=- ~ ;; » "! 5

Fig. 7. To tal square er ror during ll a illill g epochs for Leckpropagntiou algoritluu ..rud HLS m ethod n .~ I. Kn. t i t~ , 1\ t. I';: . Vu ko b rat ovi-'

00030 ., • BAr t

- I------..... -I I•V, oz -00010 -V>o c, - 0 0020 • ,< • • - 0 0030 ~ • • • • -0.0040 '::-'~,,~~~~~~~~~.,..------~f ' f , i • i , 000 O'f.. C 6 " •

C C2r. -• L"t.. :;· ~ / j "" 'T t-o - ..-• • • ')~ L[ . - . . / • .... . - ...... • / C• c ' • '" - •• -, • • • 0 • < G C' O. • Ct. -, •< • • • 5~ • -c 0 -• c: • Ct. • ~ -• • z C OJJC • a - ~ • , • , -II' • , , • £ '''0 • , - 0 ~ ... "" v - , I -• , •

'1 0 ~ • j • • • -v v '''• C ' , , i ' , t f f i i , ?' < ' ,,- • . - - 0: C • • c-': c • • C •• • v r- - c

F i ~ . ~ . I'n :oo it io ll f>I I 'Or ("1 t lu - I" ... t II.•, I in 1l.<, IH" r;ui7.il t ion p h a ",f" ( "'lllll·"li"lli... 1 ;Ud lilf'.· l ll l l.... fIJI" Ct'lItl"ll,f IllallilJul;'l il'lI 1,,1 11 ,1 ...... I i~ a ccomplishr-d. III o rder 10 vt'rify genl'raliza t. ioll propcrtics o f proposed al ,!!; orilllllls, seine situ ulat.ion l'xIH'rillll'llts wvn- pvrfonucd . In t.llt ' gl'lleralizalioll pllasl' robot 111 0\'(' :-0 along dilfl ,("t'llt. t rajt'ct ory ill rom pn risou with h'al' lI ing phast" exartly [roru poilit "{'/ E {O .:I ;O .I;(J .:,;O .;O .;U .}) 10 p O;III fJ ('/ E {1.~ ;O .G ;O .(j ;O .:! ;O .:! ;O .:!} ) . TIll' gl'll l'ra liz a t ion r.-s ult (posil ion error wii.h k-aruing using HLS III PI hod a nd wit hou t I. 'arllillg for 1Ill' lirst. joint ) art> prescntcd ill Figure D.

\\'e cau Sl'l' t hat with "learned" iuverse dyuaruic ro bot 1I 1Odt'l, t.rucki ug for quit e dilfereut robot t ruject.o ries is quil l' sat i:o:- fac tory.

s. l'O~(,LUS IO~

Till' r l' ~ (' a r c h work rvsulis pn's('lIl l.'d ill this papl'r indicat e till' feasihility o f IIS ing high-cllicieu: conuert.iouist st.r uct.u rcs for lca ru ing com plex input/ otll put rela t.i ons of rohoi. 's dyu.nnic«. Basl'd Oil situulation results we ran say that. applir at ion of single-layer neural net works and 1lIlIltilayt'r IH'rn'pt.rolls i:-; very proiuisiug for feedforward cont rol o f rahat m anipula tors whose fuudauiental dY II,lI J1 ic structure is generally o r particul arly unk nown. llowever, twi n ' st raightforward 11 (,111',,1 network is not suita ble for prart ical solu tious ill renl-t iun-, because o f the in creased dirucn sio nulity of input and out put spaces a ud long lcaruing ti me. hellcv, b a~c d 0 11 t.his fa cts proposed IIC\\' couuec t.io nist control a lgorit hms with decomposed "aF-2Sr" st ru ct.ure uses availahh- iuforui.u ion about robot dynamic b ill. o nly ill ge ucrn l and s pec ific 1'0 1' 111. The li Se.' of decomposition priuciple cna hlcs t.lu- sy uchro uo us t rain ing of scv- rul multilayer per reptroll' a ll th c sim pler input/ out put relations with significa nt. reduction of learning t ir uo.

AHot her i ruport a lii fl"a t. u f(" of p roposed struct II r {' ~ is a p ossi hiIity o f acrelerat iou of new learning a lgori thms t hat use Recu rs ive Least Sq lla H' method for estimat ing of weight ing factors ill network layers . These algorithm:" han' t he Ieat.ure t hat the learuiug rate is t irue depondcn t. , whi ch enables th at. wit.h suflicieut ly co nver gent solut ion t.h ey assure [ust er lcuruing I hall t.h c gouc ra lized ddt a rille ill s ra udnnl backpropagat.ion algorilllllls . Also, as res ult, adaptive cnpa hility o f the co n nect ion ist. controller to t.11(' system uncertainties was clari fi ed .

Ackuowlcdgoiuout s. This work was su p ported ill part hy g ra nts [1'0 111 the Na ti onal Scient ilic Foundation of Itep" "I;,, Serbia under the project ·'100·1 ItOn OTI 1\ ,\". The a ut.hors wish 10 t h.u u k Vl ad a Stefa novic for his as-istancc ill t he defin ing the prcliminary version o f ItLS learning a lgorit h m. Special apprec iat ion i ~ expressed to prof. Srdj au St ankov ic for rnany helpful d iscussions in the licld of neural ~ et works.

HEF EIIE ;'; l · l: ~

[1] G .N. Sal'ili~ : h d f ltigt n t m bo/ie' control, IEEE Tr,U'... . o-tious 0 11 Automatic Control A C-28 (5). ~Iay t VS3. [2) A. Meystel: l ntrltigr n l con trol in robotir» .Iou tn.•I ..( HOUOlic Systems J (1988).269-308. [3J D . "ati.\ ;\1. Vuk..IH·al .. \'i ~~ : ('onnt'diollul f1Pl" 'O/J('ht3 10 r on trot of manip'l /a tion robots , .)011 111 ;,1 of IIII .·lliv;'·llt ;Hld Hol,,,t iC' Sy..t f'III " . ( t n aplH'iu-).

[4J I>. l '...rli ti.... A. ~i df· ri ... A. Yaru.uuurn: t1 h" rfl rchu'al m od" f or " OJ,fldIlTY mOl' I"7nt'n t lind ilj dPP/I f"lItI 011 t o rnbotu: s . f'r"("f'f'l lil lJ.!," .,f I... , IEEE l,lt"Tllilli" llill Co n fe- re nce 01 1 Ne u ral Ne- t, w ••I",...... "--';ll l ()"H'~ll ,. ) 11 11 (' I 'Ul-i,,- •.~I:,., I -,'""I. ) n. (:'1 A. (;11""'.1. Ei llwrI.;\1. Km u : St'fH"07ll 0rphIC aTc-h,t,dllrt's for jajl adaptll't robot con trol, !·r••• ·,·,·. lillV; .. ,.f 111(> IEEE Iut er untionnl ('I' Jl fl~ I"l' I II ' f' o n (in)u.ti(" s all'I Auto matiou. Haleigh, April I!JS.~. l -I ~. -I ·I U . (G) l r.F. Ha .... i: ( '('lfl1ud,o l.ut DynamiC ('onlnl! o] /lobol ,\fll n i p Il Ja loT$ , I' Ia .O. dissertarion.Ulli \·'·I ..ily ,.r S"llllwlll ( ':.lif" rJli'l. LIJ.. A lI~c1.·:-" f)" c f"llllwr IUUO. 17) II. ~Iiya ll"'('" ~1. 1'~ lW. llo . T . Sf·tOY.UII ;l. H. S u.mki : Fndbac!·-ErroT· L'4TnIFlg Finral ndlVor!' [ or Irll) H"t oTY ("l'n ITol oj a Tobo t iC' maFil""lal or. ;": " lIra l :'\" Iwol h 1 PUSS). ·! 5 1-·l ti5.

[ ~ l ~1. ""Witl... Y. 1' lito . H. l-obe, H. Suzu ki : Il lf rarcturnl ' I t ural n et wo rk: m oJd [ or vvlltutary "WI" " If li t "'Ith ap plica tio n t o robotr ("$, IEEE (',," 110 1 S y :-. It·Il I" ~1 ; lK"zille ::' 7 ( IUS';'). 11 ;!J-1 8 .'l. [0) n. ""Ii"': ( 'oIHu"dl oui$t It'arPll1lg m odt'l$ [or wlrlligt'rtt ro n t rol vl marti'"1Iation 1'0601$, Pro " ""'(ill,il,'" tof I I... I~ I,\CS Inlt'lllill iullal S.\·llI IH .... iuru U II ~liltlaf ,IllOlti('al and lur clligent ~ 1 (ulds in Sy... I"1I1 S ilH u lill iUII . H I 11 ...... ·1... , S" ph'lIlt"'r I U!JlJ. [In] ~ 1. \ ·lIkll"l';ll ..\ , j . ~ . D. S IIIl..i •..,;":. I'in\u l... ki: ,\'Ml-Adllplil'C ulld At/apliN ('CHitTO/ oj M a n , p ll' lotion /lobvl$, Spri ll,il,.·r· \ ·,·,Ia,;, Hetliu, I!JS.t"r , (11] D.E. lt uun-III" II . J .L. ~ k('ld l:' lld : I'nraltc! [)i$lribldttl f'roccu i llg ( PDP): Erplora tion in th t M U TO$l rtut llf"! oj rO!Jnl t;o ll. \'01. J and ~ . ~IIT I'I C..... Cam b rid ge. 1 ~ 81.i .

[12J L. LjullK " lid '1. S.. t1 '·I-..t rum: T h ror y and J' ra rti C' , o] Hrc xrsive Idn d ijica t io 71 . Cnmh ridge, t\IIT I'l't·...... I ~ ) S ; J .