Motor Coordination Learning for Rhythmic Movements Melanie Jouaiti, Patrick Henaff To cite this version: Melanie Jouaiti, Patrick Henaff. Motor Coordination Learning for Rhythmic Movements. Develop- ment and Learning and Epigenetic Robotics (ICDL-Epirob), 2019 Joint IEEE International Confer- ences on, Aug 2019, Oslo, Norway. hal-02144957 HAL Id: hal-02144957 https://hal.archives-ouvertes.fr/hal-02144957 Submitted on 11 Jul 2019 HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est archive for the deposit and dissemination of sci- destinée au dépôt et à la diffusion de documents entific research documents, whether they are pub- scientifiques de niveau recherche, publiés ou non, lished or not. The documents may come from émanant des établissements d’enseignement et de teaching and research institutions in France or recherche français ou étrangers, des laboratoires abroad, or from public or private research centers. publics ou privés. Motor Coordination Learning for Rhythmic Movements Melanie Jouaiti1 and Patrick Henaff2 Abstract— The perspective of ubiquitous robots raises the the human motor nervous system for rhythmic movements issue of social acceptance. It is our belief that a successful robot production and coordination. integration relies on adequate social responses. Human social The aim of this work is to learn motor coordination interactions heavily rely on synchrony which leads humans to connect emotionally. It is henceforth, our opinion, that motor with a human partner performing rhythmic arm movements coordination mechanisms should be fully integrated to robot with changing frequency, amplitude and motion. Conse- controllers, allowing coordination, and thus social synchrony, quently, plastic CPGs, i.e. CPGs which incorporate plasticity when required. The aim of the work presented in this paper is mechanisms, are implemented in the joints of the Pepper to learn motor coordination with a human partner performing robot. Results show that the robot is indeed able to achieve rhythmic movements. For that purpose, plastic Central Pattern Generators (CPG) are implemented in the joints of the Pepper motor coordination with the human performing various arm robot. Hence, in this paper, we present an adaptive versatile motions. model which can be used for any rhythmic movement and In the second part of this paper, related works are pre- combination of joints. This is demonstrated with various arm sented. In the third part, the CPG model, its architecture movements. and the equations used in this work are introduced. Then, in the fourth part, the experimental setup as well as the I. INTRODUCTION experimental results are presented. Finally, we discuss our In the last few years, social robotics has been widely results. developed with the problematic of how to make robots more II. RELATED WORKS acceptable. The question has been considered from pretty much every angle, by trying to make the robots physically In [7], subjects were asked to wave their hand to the beat of attractive to humans, by working on robot gaze, robot speech, a metronome while the experimenter uttered disruptive words robot grasping or robot walk. Another aspect, which should and either waved her hand in phase or anti-phase or not at not be neglected is the social adequacy and especially the all. It was observed that subjects remembered more words synchrony phenomena which tend to emerge consciously, or and attributed a greater likability to the experimenter when unconsciously when humans interact with each other [1], she was waving in phase. In [8], a robotic arm was able to while walking [2], rocking chairs [3] or handshaking [4]. synchronize with an external signal and perform coordinated As it just so happens, rhythmic gestures inherently induce drum beating with a changing frequency. They employed the dynamic coupling phenomena playing a fundamental role in Dynamic Movement Primitives Framework presented in [9]. physical and social interpersonal interactions [5], [6]. Closer to this work, [10] introduced a model designed to While the term coordination refers to two events occurring reproduce rhythmic arm movements with the NAO robot. A with a constant phase difference (which can differ from zero), reservoir of oscillators provides one with a close frequency the term synchronization is more restrictive and imposes a and while the oscillator can be slightly entrained during the phase difference of 0 or π. So synchronization between a interaction, the oscillators do not retain the frequency, going robot and a human performing rhythmic movements would back to their original properties right afterwards. necessarily lead to motor coordination. A CPG is a biological structure found in the spinal cord In our opinion, should a robot have the ability to respond of vertebrates. It can generate a rhythmic signal which can in a socially acceptable way in rhythmic interactions, i.e. be modulated by sensory feedbacks, without receiving any to adapt to the human, robot controllers able to produce rhythmic input. The role of CPGs in locomotion has been rhythmic movements and trigger the emergence of motor co- proven and well studied and its implication in rhythmic upper ordination in the interaction are required. One chosen way to limb movements is also strongly suspected [11], [12]. CPGs achieve this consists in designing intrinsically rhythmic bio- are based on a pair of two mutually inhibitory oscillating inspired robot controllers, such as Central Pattern Genera- neurons, called half-center [13], controlling the extensor tors (CPGs) which also incorporate synchronization learning and flexor muscles. Non-linear CPGs, also called relaxation abilities similarly to the plasticity mechanisms involved in oscillators, can synchronize with an oscillatory input or with a coupled CPG, thus ensuring coordination. *This work was supported by CPER 2015-2020, plateform IT2MP- Several oscillator models can produce movement coor- SCIARAT, region Nancy Grand-Est, France dination [14], [15]. We chose the Rowat-Selverston (RS) 1 Melanie Jouaiti is with Universite´ de Lorraine, CNRS, LORIA, F-54000 oscillating neuron model [16] which can exhibit the four Nancy, France [email protected] 2Patrick Henaff´ is with Universite´ de Lorraine, CNRS, LORIA, F-54000 characteristic behaviors of a biological neuron, i.e. endoge- Nancy, France [email protected] nous bursting, plateau potential, post-inhibitory rebound and quiescence [17], [18]. y Moreover, McCrea and Rybak [19] proposed a bio- _ ifE;F g VifE;F g = yifE;F g − W −4y + ifE;F g Fi (1) inspired model of half-center CPG for mammal locomotion. 1 + e ifF;Eg The CPG is divided into the extensor and flexor parts y_ifE;F g = and has four interneuron layers: Rhythm Generator, Pattern !! τm 2 σf Formation, Sensory Neurons and Motoneurons. It can also σf − − 1 − σf tanh VifE;F g · τs Af takes sensory feedback into account. While this model is ifE;F g 1 + σs widely used for locomotion [20], [21], [22], very few works yifE;F g ifE;F g − VifE;F g + apply it to arm movements: to our knowledge, only [23] used τm τsτm it to study the reaching movement. ! Afi σf Vi fE;F g tanh fE;F g τsτm Af ifE;F g (2) with V the membrane potential and τm and τs time III. MATERIALS AND METHODS constants, Af influences the output amplitude, while σf determines whether the neuron is able to oscillate or not. σs influences the intrinsic frequency, i 2 N, designating the joint id. Fi the CPG input, a synaptic weight designed A. CPG Architecture to scale the input and the term in W models the mutual inhibition between the extensor and flexor rhythmic cells. Pattern Formation neuron PF, Sensory neuron SN and The general architecture for the CPG is represented Fig. Motoneurons MN are defined as follows [24]: 1. In the experiments presented in this article, a SoftBank Robotics Pepper robot is used. The output of the CPG is 1 thus considered as an angular position offset and the position PF (VifE;F g ) = PFifE;F g = −V (3) ifE;F g control mode to command the joints of Pepper is employed. 1 + e 2 For a better understanding of this subsection, please refer to 1 SN(vmesi ) = SNi = α pos (4) [24] where the CPG model is extensively detailed. 1 + e s imes MN(PF ; SN ) = MN = 1) Mathematical Models of the neurons: For the rhythm ifE;F g i ifE;F g generator neurons, Rowat-Selverston cells are used. Forcing 1 (5) α PF −SN the oscillator and adding mutual inhibition between the 1 + e m ifE;F g i rhythmic cells, the RS neuron model can be written as with α = −0:061342 and α = 3. pos is the angular follows: s m mesi position measured for the given joint. While MNiF and MNiE would be the command of the flexor and extensor muscles respectively in biological systems, in robotics, it is customary to subtract both signals. So the output would be: outputi(t) = MNiF − MNiE (6) B. Plasticity mechanisms Since the RS model is a generalized Van der Pol oscillator, known properties of the Van der Pol can be applied. Hebbian mechanisms inspired by [25] can be integrated to the bio- inspired CPGs, enabling it to learn an external signal. The learning rules proposed in [24] can be applied and frequency learning, inspired by [25], is defined as: σ_ s = ifE;F g q 2i Fi τmτs(1 + σs − σf )· fE;F g ifE;F g (7) yifE;F g q V 2 + y2 ifE;F g ifE;F g σf In equation 1, the expression Af tanh( V ) influences Af Fig. 1. General CPG architecture. The CPG output values are used as the amplitude of V and hence of the CPG output. It is thus angular position commands. With A(F ) the amplitude of F interesting to adapt the amplitude of the neuron oscillations which allows rapid reconfiguration during the interaction.