Task-Space Control Interface for Softbank Humanoid Robots and Its
Total Page:16
File Type:pdf, Size:1020Kb
Task-Space Control Interface for SoftBank Humanoid Robots and its Human-Robot Interaction Applications Anastasia Bolotnikova, Pierre Gergondet, Arnaud Tanguy, Sébastien Courtois, Abderrahmane Kheddar To cite this version: Anastasia Bolotnikova, Pierre Gergondet, Arnaud Tanguy, Sébastien Courtois, Abderrahmane Kheddar. Task-Space Control Interface for SoftBank Humanoid Robots and its Human-Robot Interaction Applications. IEEE/SICE 13th International Symposium on System Integration (SII 2021), Jan 2021, Online conference (originally: Iwaki, Fukushima), Japan. pp.560-565, 10.1109/IEEECONF49454.2021.9382685. hal-02919367v3 HAL Id: hal-02919367 https://hal.archives-ouvertes.fr/hal-02919367v3 Submitted on 8 Oct 2020 HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est archive for the deposit and dissemination of sci- destinée au dépôt et à la diffusion de documents entific research documents, whether they are pub- scientifiques de niveau recherche, publiés ou non, lished or not. The documents may come from émanant des établissements d’enseignement et de teaching and research institutions in France or recherche français ou étrangers, des laboratoires abroad, or from public or private research centers. publics ou privés. Task-Space Control Interface for SoftBank Humanoid Robots and its Human-Robot Interaction Applications Anastasia Bolotnikova1;2, Pierre Gergondet4, Arnaud Tanguy3,Sebastien´ Courtois1, Abderrahmane Kheddar3;2;4 Abstract— We present an open-source software interface, Real robot called mc naoqi, that allows to perform whole-body task-space Quadratic Programming based control, implemented in mc rtc Fixed frame rate low-level actuator commands Robot sensors readings from framework, on the SoftBank Robotics Europe humanoid robots. Other robot device commands low-level memory fast access We describe the control interface, associated robot description mc_naoqi_dcm packages, robot modules and sample whole-body controllers. Device commands Local robot module Sensor data We demonstrate the use of these tools in simulation for a robot actuators - - encoders LEDs colors - - electric current interacting with a human model. Finally, we showcase and speakers audio - - IMU discuss the use of the developed open-source tools for running wheels speed* - - touch sensors tablet image* - - FSR * the human-robot close contact interaction experiments with real … - - … human subjects inspired from assistance scenarios. * Pepper robot only mc_naoqi * NAO robot only control interface I. INTRODUCTION AND BACKGROUND Forward QP solver result for new actuator commands Update real robot state Quadratic Programming (QP) task-space control has be- Forward control robot module devices state for new with new sensor data come a golden standard in humanoid robot control [1], [2], devices commands [3]. One of the most powerful software control frameworks that implements QP task-space control, called mc rtc1, is mc_rtc (FSM) controller now publicly available. This framework provides developers with useful tools for writing complex controllers for either result control robot state individual or multiple interacting robots to perform wide Used by Instances of variety of experiments, e.g. in aircraft automation [4] or in QP solver real robot state physically interacting with humans [5], [6] that we techni- cally explain and extend in this work. human model state In order to control any given complex robots with this Structure representing complete robot description framework, an interface must be developed to allow com- Kinematic tree parsed from URDF file, contact surfaces, Robot convex collision avoidance shapes, visual and physical links munication between the control framework and the robot’s representations, devices and sensors state representation, module low-level controllers, sensors and devices. One example of current robot state representation Used by 2 such open-source interface is the mc openrtm software URDF file Links meshes Robot that is used to control Humanoid Robotics Project robots Contact surfaces description files Convex collision shapes description files description from Kawada Robotics [7]. However, for different types of robots a specific interface, as well as robot description and Fig. 1: mc naoqi interface enables communication between robot module, need to be developed in order to adapt to SBRE humanoid robots and mc rtc control framework. It particularities of robot brand and low-level control strategies, can be used to steer the robot behaviour in HRI experiments. devices and onboard operating system (OS). The idea of task- space control is to lie exactly between low-level control and high-level planning with somewhat adjustable frontiers. In this work, we present the open-source software inter- that are running NAOqi OS and a customly developed local mc naoqi dcm face, called mc naoqi3, that enables running the controllers module (Sec. II). We also provide robot implemented with mc rtc framework on widely used Soft- modules and description packages that serve as a represen- mc rtc Bank Robotics Europe (SBRE) humanoid robots [8], [9], tation of SBRE robots in the framework (Sec. III). Additionally, we release a basic sample Finite State Machine 1SoftBank Robotics Europe, Paris, France (FSM) mc rtc controller for Pepper robot and its interaction 2University of Montpellier–CNRS LIRMM, Interactive Digital Humans, with a human model in simulation (Sec. IV). Finally, we Montpellier, France 3CNRS-AIST Joint Robotics Laboratory, IRL 3218, Tsukuba, Japan present our current instance use of the developed tools for 4Beijing Advanced Innovation Center for Intelligent Robots and Systems, running the mc rtc controllers developed for Human-Robot Beijing Institute of Technology, Beijing, China interaction (HRI) experiments with human subjects (Sec. V). 1https://jrl-umi3218.github.io/mc_rtc 2https://github.com/jrl-umi3218/mc_openrtm Fig. 1 shows an overview of the developed tools and their 3https://github.com/jrl-umi3218/mc_naoqi interconnection, described in detail in the rest of the paper. II. CONTROL INTERFACE B. Forwarding sensor data from the robot to the controller mc naoqi Fig. 2 gives a schematic overview of the mc rtc control The interface is also responsible for getting framework and its connection to the simulation and control the most up-to-date sensor readings from a robot low-level interfaces, such as mc naoqi. Interested readers are invited memory in real-time and forwarding sensor measurements mc rtc to visit mc rtc project website for more information, instal- in a suitable form to the controller real robot state lation instructions and tutorials. representation as a feedback. This way, the task-space QP controller keeps track of the real robot state and can use it to perform closed-loop QP control computations. Besides common sensor readings, such as encoder values, force sensors or Inertial measurement unit (IMU) measure- ments, mc naoqi interface also allows to forward from the robot to the mc rtc controller such sensor readings as electric motor current and touch sensor readings (from tactile or bumper sensors). We describe how custom robot sensors are implemented on the robot module side in detail in Sec III. For HRI applications, the touch sensor readings are espe- cially beneficial to be forwarded to the controller, as they allow to detect when a human touches the robot. This signal can then be used inside the mc rtc FSM controller, for instance, to trigger an appropriate reaction of the robot to the touch or to trigger a transition to a specific FSM state of Fig. 2: mc rtc control framework architecture. the controller. We demonstrate this use-case in our sample HRI experiment, presented in detail in Sec. V. Fig. 1 illustrates schematically the role of the mc naoqi interface as a communication layer between mc rtc control C. Local low-level robot module framework and NAOqi OS running onboard SBRE humanoid A customized local robot low-level module, called robots, such as NAO, Pepper or Romeo. In the following mc naoqi dcm4, is cross-compiled for NAOqi OS and is subsections, we describe the entire system in detail. set to run onboard the robot to read sensor values and set device commands synchronized with a robot control loop A. Forwarding device commands from controller to the robot via Device Communication Manager (DCM) every 12 ms, fastest update rate currently available for SBRE robots. mc naoqi The main role of the interface is to forward When a user connects to the robot via mc naoqi inter- the fixed frame rate control commands computed by the QP face, and commands to turn on robot motors, a preprocess mc rtc solver of whole-body QP controller to the onboard function is connected to DCM loop, to start setting the low-level robot actuators control. The decision variable of actuator commands at a fixed time rate, synchronized with mc rtc the QP controller is robot joint accelerations, which the other DCM operations. After the user commands to turn is integrated once to get desired velocity (e.g. for Pepper off robot motors via mc naoqi interface, the preprocess mobile base command), and then once more to get desired actuator command update function is disconnected from the robot joint angles for joint actuator commands. DCM loop. This way, the user can safely use other control For social, user-friendly and interacting humanoid robots, applications, such as Choregraphe, with the robot right after such as Pepper or NAO, it is highly beneficial to endow using mc naoqi interface, even though mc naoqi dcm mc rtc controller with the functionality to also forward module is still active on the robot. other device commands, such as sentence to play from the In order to prevent the default high-level robot behaviours speakers, desired tablet screen image or eye led color, from to interfere with the commands forwarded to the robot via the mc rtc controller to the robot devices via mc naoqi mc naoqi from the mc rtc controller, the default robot interface. This functionality allows mc rtc framework users safety reflexes are disabled when robot motors are turned to develop controllers which can provide a richer interaction on via mc naoqi.