A Robust Balance-Control Framework for the Terrain-Blind Bipedal Walking of a Humanoid Robot on Unknown and Uneven Terrain

sensors

Article A Robust Balance-Control Framework for the Terrain-Blind Bipedal Walking of a Humanoid Robot on Unknown and Uneven Terrain

Hyun-Min Joe 1 and Jun-Ho Oh 2,*

1 Department of Artiﬁcial Intelligence Machinery, Korea Institute of Machinery & Materials, 156 Gajeongbuk-ro, Yuseong-gu, Daejeon 34103, Korea; [email protected] 2 HUBO LAB, Department of Mechanical Engineering, Korea Advanced Institute of Science and Technology, 291 Daehak-ro, Yuseong-gu, Daejeon 34141, Korea * Correspondence: [email protected]; Tel.: +82-10-5402-8049

Received: 24 August 2019; Accepted: 25 September 2019; Published: 27 September 2019

Abstract: Research on a terrain-blind walking control that can walk stably on unknown and uneven terrain is an important research field for humanoid robots to achieve human-level walking abilities, and it is still a field that needs much improvement. This paper describes the design, implementation, and experimental results of a robust balance-control framework for the stable walking of a humanoid robot on unknown and uneven terrain. For robust balance-control against disturbances caused by uneven terrain, we propose a framework that combines a capture-point controller that modifies the control reference, and a balance controller that follows its control references in a cascading structure. The capture-point controller adjusts a zero-moment point reference to stabilize the perturbed capture-point from the disturbance, and the adjusted zero-moment point reference is utilized as a control reference for the balance controller, comprised of zero-moment point, leg length, and foot orientation controllers. By adjusting the zero-moment point reference according to the disturbance, our zero-moment point controller guarantees robust zero-moment point control performance in uneven terrain, unlike previous zero-moment point controllers. In addition, for fast posture stabilization in uneven terrain, we applied a proportional-derivative admittance controller to the leg length and foot orientation controllers to rapidly adapt these parts of the robot to uneven terrain without vibration. Furthermore, to activate position or force control depending on the gait phase of a robot, we applied gain scheduling to the leg length and foot orientation controllers, which simplifies their implementation. The effectiveness of the proposed control framework was verified by stable walking performance on various uneven terrains, such as slopes, stone fields, and lawns.

Keywords: balance control; biped walking; capture point; DRC-HUBO+; humanoid robot; terrain- blind walking; zero-moment point

1. Introduction Considering the eventuality of a future accident, such as that of the Fukushima power plant nuclear accident [1], the DARPA Robotics Challenge was held to test robots performing tasks that would replace human ones in disaster environments resembling the Fukushima accident [2]. Environments where humans act, including power plants, are designed with consideration for human morphology. This fact implies that humanoid robots are suitable candidates for carrying out tasks in such environments and employing human-made objects, and most of the teams in the challenge used humanoid robots. However, many of these robots were limited by unstable walking, especially when ground slope and height variation appeared during the tasks.

Sensors 2019, 19, 4194; doi:10.3390/s19194194 www.mdpi.com/journal/sensors Sensors 2019, 19, 4194 2 of 19

Although many robots used vision and LiDAR sensors for the perception of uneven terrain, inherent errors of the sensors and the measurements the environment caused terrain sensing errors. In order to ensure stable locomotion in the presence of uncertain information regarding the ground, a robot should have the ability of balance control to overcome the ground information error. In addition, the robustness of balance control is more important than the accuracy of ground terrain sensing for environments composed of rough local terrain smaller than the foot, such as in a gravel field. Thus, for a humanoid robot to accomplish a mission in real-world environments, a robust, terrain-blind walking control is required to prevent falls on unknown and uneven terrain. The aforementioned observations show the necessity of terrain-blind walking control research, which can guarantee stable locomotion without terrain information to account for uncertain ground information and improve balance-control abilities. The most basic and important balance-control method for a biped humanoid robot with a finite sized foot is maintaining the zero-moment point [3] within the support polygon. Nagasaka [4] proposed a method of stabilizing the zero-moment point (ZMP) using the acceleration of the humanoid torso based on the cart-table model. Likewise, Choi et al. [5] proposed a method to balance a humanoid robot through ZMP error feedback. In another approach, Kim et al. [6] implemented the stable walking of the humanoid robot HUBO by designing an observer-based state feedback controller to regulate the ZMP by estimating the state of the robot from the ZMP. Martinez at al. proposed a method to more accurately obtain a ZMP control model, improving ZMP control performance [7]. These methods are effective on flat ground, but their performance is not guaranteed on uneven terrain, thus imposing the necessity of robust ZMP controllers for unknown, uneven terrain. In addition to the ZMP, it is important to maintain posture along the walking pattern during biped walking. To maintain posture on unknown uneven terrain, the foot position and orientation should be controlled, and contact force control is commonly used for adapting the foot to unknown uneven terrain. For instance, Ott et al. [8] proposed a contact force distribution and control method inspired by the grasping method. Furthermore, quadratic-programming based balance controllers have been proposed that calculate the optimal joint torque input, considering constraints [9–11]. Among them, Feng et al. [8] developed a quadratic-programing-based balance controller to simultaneously optimize the joint acceleration, joint torque, and contact force. These controllers are implemented on joint-torque-controlled humanoid robots (e.g., TORO [12], Atlas [13], and Valkyrie [14]) and stabilize the robots’ postures on unknown uneven terrain. However, these controllers cannot be used in position-controlled humanoid robots (e.g., DRC-HUBO+ [15], HRP-4 [16], and JAXON [17]). Position-controlled robots rely on impedance or admittance control for contact force control. Lim et al. [18] applied a position-based impedance control to reduce the impact forces generated between the landing foot and the unknown, uneven terrain. Similarly, Kim et al. [19] applied admittance control by modeling the leg as a spring–damper system, and Kajita et al. [20] proposed a linear inverted pendulum tracking controller and a damping controller, which is a kind of admittance control, to regulate posture. However, these position-based contact-force control methods do not show comparable performance relative to their torque-based counterparts [9–11]. Considering the limitations of the studies presented above, in this paper, we propose a robust balance control framework to improve the terrain-blind walking performance of a position-controlled humanoid robot. To build a control frame that is robust against disturbances from uneven ground and enables fast posture stabilization on challenging terrain that conventional position-controlled humanoid robots fail to navigate, we synthesized a capture-point controller [21–23] and a balance controller in a cascaded structure. We applied a capture-point controller, which proved robust to disturbances, as a control reference generator for the balance control framework. The disturbed state of the robot is reflected in the capture point, and the capture-point controller adjusts the ZMP reference to stabilize the perturbed capture point. The adjusted ZMP reference is used as a control reference in the balance controller. Sensors 2019, 19, 4194 3 of 19

The balance controller is comprised of a ZMP controller to stabilize the motion of the center of mass (CoM) and a posture controller to stabilize the posture of the robot. Our ZMP controller is designed to control the ZMP in uneven terrain by receiving a ZMP reference that considers disturbances from capture-point control. In addition to stabilizing the CoM through ZMP control, leg length and foot orientation controllers act as posture controllers that adapt the leg length and foot orientation to the shape of the terrain to maintain the posture of the robot. Then, for faster posture stabilization in challenging terrain, we applied a proportional-derivative (PD) admittance controller to each posture controller to enable fast reference tracking without oscillation. Furthermore, we applied the gain scheduling method for the convenience of implementation when the leg length and foot orientation controllers were applied to the robot. To test the performance and effectiveness of the proposed control framework, we implemented it on the DRC-HUBO+ humanoid, whose arms and head were replaced by two dumbbells with the same weight. Then, we conducted various experiments under different ground conditions. Videos from the experiments can be seen in [24]. The rest of this paper is organized as follows. The overview of robot system and proposed balance control framework is described in Section2. In Section3, we describe the ZMP controller for stabilizing the horizontal CoM motion on tilted ground using capture-point control. Section4 presents the design of the leg length and foot orientation controllers for adaptation to uneven terrain. In Section5, experimental results of balance maintenance on di fferent uneven terrain conditions are reported. In Section6, we draw conclusions based on the experimental results.

2. Control Framework and Robot System

2.1. Balance Control Framework Bipedal walking control systems are traditionally composed of a walking pattern that generates the walking motion of the robot and a balance controller that stabilizes the walking pattern from disturbance [25]. In an ideal environment and conditions, the robot can walk with only a walking pattern, but in real environments, various disturbances act on the robot, so a balance controller must be designed to stabilize the walking pattern through sensor feedback. An overview of the proposed balance control framework is shown in Figure1. The walking pattern generator uses preview control [26] to create a CoM reference. The walking pattern also generates a ZMP reference (zre f ), capture-point reference (ξre f ), desired posture reference, foot position reference, and foot orientation reference. To design a robust balance controller, we applied capture-point control, which proved robust to disturbance, to the balance control framework. In the capture-point controller, a zcp reference is generated to maintain the balance of the robot. The generated zcp is used as a reference for a balance controller composed of ZMP, leg length, and foot orientation controllers. Because the CoM behavior represents the stability of the bipedal walking robot system, the ZMP controller is applied for CoM stabilization. The ZMP controller receives the zcp input generated through the capture-point control and controls the ZMP tracking. Because CoM stabilization alone is insuﬃcient for maintaining the walking pattern posture in uneven terrain, leg length and foot orientation controllers are added to the balance controller to maintain a desired posture. The following sections describe, in detail, each component of the balance controller. Finally, the control inputs from the balance controller and the walking pattern are converted into joint angles (θre f ) by inverse kinematics and each joint ane controller of the robot tracks the generated joint angles. SensorsSensors 20192019,, 1919,, 4194x FOR PEER REVIEW 44 of 1919 Sensors 2019, 19, x FOR PEER REVIEW 4 of 19

Figure 1. Overview of the information flow in proposed balance control framework. FigureFigure 1.1. OverviewOverview ofof thethe informationinformation flowflow inin proposedproposed balancebalance controlcontrol framework.framework. 2.2. Robot System 2.2.2.2. RobotRobot SystemSystem Figure 2 shows the humanoid robot DRC-HUBO+. DRC-HUBO+, a disaster rescue robot developedFigureFigure by2 2shows HUBOshows theLAB the humanoid ( humanoida mechanical robot robot engineering DRC-HUBO DRC-HUBO+. department+. DRC-HUBODRC of-HUBO+ the Korea+,, a disasterAdvanced disaster rescue rescue Institute robot robot of developedSciencedeveloped and byby Technology HUBO LAB ) was( (aa mechanical mechanical designed engineeringto engineering participate department department in the DARPA of of the the RoboticsKorea Korea Advanced Advanced challenge Institute Institute in 2015 of. ofRainbowScience Science and Robotics and Technology Technology) currently) was produces was designed designed and tosells to participate participate it with distributorship. in in the the DARPA DARPA In the Robotics Robotics actual challengerobot experiment, in 2015. 2015. RainbowweRainbow used RoboticsaRobotics modified currentlycurrently DRC-HUBO+. producesproduces Because andand sellssells the itit withwithhumanoid distributorship.distributorship. robot, which InIn the is actuala floating robot base experiment, system, wetendswe usedused to abreak a modified modified down DRC-HUBO when DRC- itHUBO+. falls+. down, Because Because and the thethe humanoid proposedhumanoid robot, balance robot, which controllerwhich is a floating is adoes floating base not system,usebase arms system, tends and toheads,tends break to DRC downbreak-HUBO+ whendown’ itswhen fallshead down,it and falls arm down, and were the andproposed replaced the proposed balanceby dumbbells balance controller ofcontroller the does same not does weight. use not arms use Considering and arms heads, and DRC-HUBOtheheads, points DRC men+-HUBO+’stioned head and’ sabove, head arm andwhen were arm replaceddevel wereoping replaced by dumbbellsa bipedal by dumbbells walking of the same controller of weight.the same of Considering aweight. humanoid Considering the robot, points it mentionedisthe often points the men above,casetioned that when there above, developing is nowhen arm devel aor bipedal head.oping Because walking a bipedal the controller walkingimmobile of controller a arm humanoid and of head a robot, humanoid connected it is often robot, to the it casetorsois often that were the there modeled case is that no armas there a orsingle is head. no rigid arm Because body,or head. thethere Because immobile was no the armperformance immobile and head arm differ connected andence head in to connectedterms the torso of control to were the modeledaftertorso the were robot as modeled a single was modified. rigid as a body,single there rigid was body, no there performance was no diperformancefference in terms differ ofence control in terms after of the control robot wasafter modified. the robot was modified.

Figure 2. Humanoid Robot DRC-HUBO+. Figure 2. Humanoid Robot DRC-HUBO+. The DRC-HUBO+ is designedFigure with 2. Humanoid high structural Robot DRC sti-HUBO+.ffness and high joint torque capacity. A harmonicThe DRC drive-HUBO+ was used is designed to output with high high torque structural to each joint, stiffness and theand frame high of joint the robottorque was capacity. designed A inharmonic theThe form DRCdrive of an-HUBO+ was exoskeleton used is to designed output for high high with rigidity torque high with structuralto respecteach joint, tostiffness weight. and the and Tablefra mehigh1 showsof joint the robot thetorque basic was capacity. hardware designed A specifications.inharmonic the form drive of The anwas robot exoskeleton used has to output a height for high high of about torque rigidity 1.7 to m witheach and joint, respect weighs and to 80 the weight.kg, fra includingme of Table the two robot 1 shows batteries was thedesigned with basic a weighthardwarein the of form 4 specifications. kg of each. an exoskeleton The joint The angle robot forreferences highhas a rigidity height generated of with about atrespect a 1.7 frequency m to and weight. of weighs 200 Table Hz 80 in 1kg the shows, including embedded the basic two PC arebatterihardware transmittedes with specifications. a toweight each jointof The4 kg controller roboteach. The has via joint a CAN height angle (Controller of references about Area 1.7 generatedm Network) and weighs at communication, a frequency 80 kg, including of 200 and Hz each two in jointthebatteri embedded controlleres with a performs PCweight are of transmitted joint 4 kg angle each control. toThe each joint of joint theangle PD controller references gain at a via frequency generated CAN (Controller of at 1 a kHz. frequency Force Area/ torqueof Network)200 Hz (F/ T)in sensorscommunication,the embedded (6-axis strain PCand are gaugeeach transmitted joint sensors controller developed to each performs joint by KAIST controllerjoint HUBOangle via control LAB) CANand of (Controllerthe a gyro PD gain sensor Area at (aa frequency DSP-1760 Network) fiberofcommunication, 1 kHz optic. Force/ gyro, KVH),torque and each attached(F/T) joint sensors tocontroller both (6- theaxis performs ankle strain and gauge joint pelvis, anglesensors are control used developed for of sensor the byPD feedback KAIST gain at HUBO control.a frequency LAB The) Fandof/T 1 sensor akHz gyro. Force/ issensor usedtorque to(a calculateDSP (F/T)-1760 thesensors fiber ZMP optic (6 for-axis gyro ZMP strain, KVH feedback gauge), attache control; sensorsd to a bothdeveloped description the ankle by of theandKAIST ZMP pelvis, HUBO calculation are LABused) isforand given sensor a gyro in feedback [ 25sensor]. ZMP (a control DSP feedback-1760. The controlfiber F/T opticsensor is described gyro is used, KVH into), Sectioncalculate attache3.Fd the to/T both ZMP feedback the for ankle ZMP is described and feedback pelvis, in control Section are used;4 a. Thedescriptionfor sensor gyro sensor feedback of the is usedZMP control to calculation calculate. The F/T the is sensor given CoM positionisin used [25]. to forZMP calculate capture-point feedback the ZMPcontrol feedback for is ZMP described [21 ,feedback22]. More in Section control detailed ;3. a hardwareF/Tdescription feedback specifications o fis the described ZMP andcalculation in system Section configurationsis 4.given The ingyro [25 ].sensor are ZMP available isfeedback used in to [ 15control calculate]. is describedthe CoM inposition Section for 3. F/T feedback is described in Section 4. The gyro sensor is used to calculate the CoM position for

Sensors 2019, 19, 4194 5 of 19

Table 1. Speciﬁcation of DRC-HUBO+.

Item Description Height 170 [cm] Weight 80 [kg] Total 32 DOFs Arm 2 Arms 7 DOFs × Hand 2 Hands 1 DOF × Degree of freedom (DOF) Waist 1 DOF Leg 2 Legs 6 DOFs × Wheel 2 Wheels 1 DOF × Head 1 DOF Actuators 200 [W], 100 [W] BLDC/DC Motor 3-axis ﬁber-optics gyro, Sensors 3-axis inertia measurement unit (IMU) Power DC voltage: 48 [V], Capacity: 11.4 [Ah]

3. Stabilization of the CoM The CoM dynamics are reflected to the ZMP through the interaction between the foot and floor, and the motion of CoM can be stabilized indirectly through the ZMP control [6,27]. The motion of the CoM is stabilized by designing the ZMP controller to follow the desired ZMP reference. Therefore, to stabilize the horizontal CoM motion, we propose a more robust ZMP controller by combining capture-point control with a previously developed ZMP controller [6,27]. Unlike the previous ZMP controller, this enhanced ZMP controller is capable of ZMP control on flat as well as tilted ground.

3.1. Previous ZMP Control Kim et al. [27] added a damping element to the ZMP control model of the biped humanoid robot KHR-2 [6] to consider the damping factor of the six-legged robot, and implemented stable ZMP control of the six-legged robot. This ZMP controller can be applied to various types of legged robots if ZMP can be measured. In this paper, the ZMP control model of [27] is applied, considering the damping factor of DRC-HUBO+, and it is extended to a more robust ZMP controller by combining the ZMP controller of [27] and the capture-point control of [21–23]. In [26], the robot is modeled as the inverted pendulum with spring and damper (IPSD) shown in Figure3. We describe the model with respect to the frontal plane (y-axis direction) for convenience, and the same process is applied to the sagittal plane (x-axis direction). Equations (1)–(5) are derived from [27]. The equation of motion of the linearized dynamic model of the inverted pendulum is given by (1), where Kc is the damping coeﬃcient, Ks is the spring constant, g is the gravitational acceleration, m is the point mass, l is the inverted pendulum length, θ is the inclination angle, θu is the command angle, and u f b is the horizontal displacement equal to lθu.

.. ml2θ = mglθ τ (1) − . where τ = Ks(θ θu) + Kcθ. − From the equation of motion, the state-space equations with input and output being horizontal displacement u f b and ZMP z are derived as

. x = Ax + Bu f b, (2) Sensors 2019, 19, 4194 6 of 19

  " # " #  θ  0 1 0  .  where x =  , A = g Ks Kc , B = Ks ,  θ  l − ml2 − ml2 ml3

z = Cx + Du f b, (3)

h i Ks Kc Ks whereSensors 2019C =, 19, xmg FOR mgPEER, REVIEWD = mgl . 6 of 19

Figure 3. Inverted pendulum model with spring and damper.

TheFrom state the observerequation isof designedmotion, the to state estimate-space state equations x in Equation with input (2), whichand output can be being modiﬁed horizontal to an observeddisplacement state 푢 feedback푓푏 and ZMP controller z are derived as follows: as . xˆ = Axˆ + 푥Bu̇ = 퐴푥+ L+z퐵푢푓푏Cx,ˆ Du (2)(4) f b − − f b 휃 0 1 0 where 푥 = [ ], 퐴 = [푔 퐾푠 퐾푐 ] , 퐵 = [ 퐾푠 ], ̇ − − u f b = Kxˆ. (5) 휃 푙 푚푙2 푚푙2 푚푙3 − Damping coeﬃcient Kc and spring constant Ks are estimated from the oscillation of the ZMP 푧 = 퐶푥 + 퐷푢푓푏, (3) (Figure4), which is obtained by applying external disturbance to the standing robot. The external 퐾 퐾 퐾 where 퐶 = [ 푠 푐 ] , 퐷 = [ 푠 ]. disturbance was푚푔 applied푚푔 to푚푔푙 the robot by a person pushing the side of the robot by hand for a short moment.The state The observer intensity is of designed the external to estimate disturbance state isx in such Equation that the (2), ZMP which does can not be reachmodified the to edge an ofobserved the foot state sole, feedback and only controller the joint as position follows: controller is applied to the robot, without the balance controller. We estimated the values of Kc and Ks to be 5774 Nm/rad and 69.375 Nms/rad, respectively. 푥̂̇ = 퐴푥̂ + 퐵푢 + 퐿(푧 − 퐶푥̂ − 퐷푢 ) (4) The estimation of these parameters is detailed in푓푏 [27]. Figure5 shows푓푏 the block diagram of the previous ZMP controller [26], where u f f is the feedforward CoM input generated from the walking pattern, zre f 푢푓푏 = −퐾푥̂. (5) is the ZMP reference, and xd is the CoM input to the robot given by the sum of the feedforward and feedbackSensorsDamping 2019 CoM, 19, x inputscoefficient FOR PEER generated REVIEW 퐾c and from spring the constant ZMP control. 퐾s are estimated from the oscillation of the ZMP7 of 19 (Figure 4), which is obtained by applying external disturbance to the standing robot. The external disturbance was applied to the robot by a person pushing the side of the robot by hand for a short moment. The intensity of the external disturbance is such that the ZMP does not reach the edge of the foot sole, and only the joint position controller is applied to the robot, without the balance

controller. We estimated the values of 퐾c and 퐾s to be 5774 Nm/rad and 69.375 Nms/rad, respectively. The estimation of these parameters is detailed in [27]. Figure 5 shows the block diagram

of the previous ZMP controller [26], where 푢푓푓 is the feedforward CoM input generated from the walking pattern, 푧푟푒푓 is the ZMP reference, and 푥푑 is the CoM input to the robot given by the sum of the feedforward and feedback CoM inputs generated from the ZMP control. The system is lightly damped and non-minimum phase, as shown in Figure 4. The damping of the system should be increased by placing the current pole (−0.732 ± 10.4i) further away from the imaginary axis. Through pole placement, we can determine state feedback gain 퐾 and observer gain 퐿. We adjusted the controller poles to reduce the settling time and to prioritize the undershoot

minimization. The controller poles were determined as (−4 ± 1.5i) and the poles for the observer wereFigure setFigure to 4.be 4.Oscillation threeOscillation times of of faster zero-moment zero- thanmoment the point pointcontroller (ZMP) (ZMP ) andpoles, and center center based of of masson mass modern (CoM) (CoM when)control when a a disturbance theory. disturbance is is appliedWeapplied applied to to the the the robot. robot. ZMP (For (For controller improved improved visualization,to visualization, the actual therobot the CoM CoM (DRC was was increased-HUBO+) increased three andthree times).tested times) .its performance. Figure 6 shows the measured ZMP and control input when a person momentarily pushed the side of the robot. It was confirmed that the measured ZMP is quickly stabilized within 1 s, and the CoM was also stabilized without oscillation (compare with Figure 4), which is the experimental result without the ZMP controller. However, because the ZMP controller estimates the state of the robot from the ZMP, the control performance is not guaranteed when the ground is not flat. In the next section, we describe the proposed ZMP controller that overcomes these drawbacks and compare it with this ZMP controller. Figure 5. Block diagram of previous ZMP control without capture-point feedback.

Figure 6. Control input and measured ZMP when the robot is perturbed.

3.2. ZMP Controller with Capture-Point Feedback In [27], the ZMP controller is designed assuming a flat surface for robot walking. As ground inclination was not considered during modeling, the robot becomes unstable on tilted ground. To include ground inclination in the ZMP controller, we utilized capture-point control [21–23], whose state reflects the tilting, and the ZMP reference was adjusted through capture-point control. The adjusted ZMP reference is reflected to the observer of the ZMP controller such that the ZMP is stabilized on tilted ground. In [21,23], capture point 휉 is defined by the sum of CoM position 푥 and the scaled CoM velocity 푥̇:

푥̇ 휉 = 푥 + , (6) 휔 where 휔 = √푔/ℎ with ℎ being the height from the stance foot to the CoM. Although capture-point error ∆휉 is ideally zero without disturbance, a disturbance applied to the robot causes the error

Sensors 2019, 19, x FOR PEER REVIEW 7 of 19

Sensors 2019, 19, x FOR PEER REVIEW 7 of 19 SensorsFigure2019, 19 4., 4194Oscillation of zero-moment point (ZMP) and center of mass (CoM) when a disturbance is7 of 19 applied to the robot. (For improved visualization, the CoM was increased three times).

Figure 5. Block diagram of previous ZMP control without capture capture-point-point feedback.

The system is lightly damped and non-minimum phase, as shown in Figure4. The damping of the system should be increased by placing the current pole ( 0.732 10.4i) further away from − ± the imaginary axis. Through pole placement, we can determine state feedback gain K and observer gain L. We adjusted the controller poles to reduce the settling time and to prioritize the undershoot Figure 4. Oscillation of zero-moment point (ZMP) and center of mass (CoM) when a disturbance is minimization. The controller poles were determined as ( 4 1.5i) and the poles for the observer were applied to the robot. (For improved visualization, the CoM− ± was increased three times). set to be three times faster than the controller poles, based on modern control theory. We applied the ZMP controller to the actual robot (DRC-HUBO+) and tested its performance. Figure6 shows the measured ZMP and control input when a person momentarily pushed the side of the robot. It was conﬁrmed that the measured ZMP is quickly stabilized within 1 s, and the CoM was also stabilized without oscillation (compare with Figure4), which is the experimental result without the ZMP controller. However, because the ZMP controller estimates the state of the robot from the ZMP, the control performance is not guaranteed when the ground is not ﬂat. In the next section, we describe the proposed ZMPFigure controller 5. Block diagram that overcomes of previous these ZMP drawbacks control without and compare capture- itpoint with feedback. this ZMP controller. Figure 6. Control input and measured ZMP when the robot is perturbed.

푥̇ 휉 = 푥 + , (6) 휔 where 휔 = √푔/ℎFigure withFigure 6.ℎ 6. Controlbeing Control the input input height and and measuredfrom measured the ZMPstance ZMP when whenfoot the tothe robotthe robot CoM. is is perturbed. per Althoughturbed. capture-point error ∆휉 is ideally zero without disturbance, a disturbance applied to the robot causes the error 3.2.3.2. ZMP ZMP Controller Controller with with Capture-Point Capture-Point Feedback Feedback InIn [27 [27],], the the ZMP ZMP controller controller is is designed designed assuming assuming a a flat flat surface surface for for robot robot walking. walking. As As ground ground inclinationinclination was was not not consideredconsidered duringduring modeling, the the robot robot becomes becomes unstable unstable on on tilted tilted ground. ground. To Toinclude include ground ground inclination inclination in in the the ZMP ZMP controller, wewe utilizedutilized capture-pointcapture-point control control [ 21[21–23–23],], whose whose statestate reflects reflects the the tilting, tilting, and and the the ZMP ZMP reference reference was wa adjusteds adjusted through through capture-point capture-point control. control. The The adjustedadjusted ZMP ZMP reference reference is reflected is reflected to the to observer the observer of the ZMP of the controller ZMP cont suchroller that suchthe ZMP that is the stabilized ZMP is on tilted ground. stabilized on tilted ground. . InIn [21 [21,23,23],], capture capture point pointξ is휉 definedis defined by by the the sum sum of of CoM CoM position positionx and 푥 and the the scaled scaled CoM CoM velocity velocityx: 푥̇: . x ξ = x + ,푥̇ (6) 휉 = 푥 ω+ , (6) 휔 p wherewhereω =휔 = g√/푔h/withℎ withh being ℎ being the heightthe height from from the stance the stance foot to foot the to CoM. the CoM. Although Although capture-point capture error-point ∆ξerroris ideally ∆휉 is zero ideally without zero disturbance, without disturbance, a disturbance a disturbance applied to the applied robot to causes the robot the error causes between the error the

Sensors 2019, 19, 4194 8 of 19

Sensors 2019, 19, x FOR PEER REVIEW 8 of 19 reference (ξre f ) and measured (ξ) capture points. To mitigate the capture-point error under disturbance, thebetween following the reference capture-point (휉푟푒푓) feedback and measured controller (휉) iscapture proposed points. in [22 To]: mitigate the capture-point error under disturbance, the following capture-point feedback controller is proposed in [22]: zcp = kcp ξ ξ + z , (7) − re f re f 푧푐푝 = 푘푐푝|⏟(휉 {z − 휉 푟푒푓 } ) + 푧푟푒푓, (7) ∆∆ξ휉 푘 푧 where kcp푐푝is is the the capture-point capture-point feedback feedback gain gain and and zcp is 푐푝 the is adjusted the adjusted ZMP ZMP reference reference by capture-point by capture- control.point control. Note thatNote ZMP thatz cpZMPfor controlling푧푐푝 for controlling the capture the point capture is the point sum is of the the sum capture-point of the capture error scaled-point byerror the scaled gain andby the the gain ZMP and reference. the ZMP If reference.kcp > 1, a ZMP If 푘푐푝 deviation> 1, a ZMP (kcp ∆deviationξ) greater (푘 than푐푝∆휉 the) greater capture-point than the errorcapture is- addedpoint toerror the isreference, added to causing the reference, the capture-point causing errorthe capture to converge-point in error the decreasing to converge direction. in the Thedecreasing proof of direction. capture-point The proof control of capture is detailed-point in [control22]. is detailed in [22]. Figure 77 showsshows thethe blockblock diagramdiagram ofof thethe proposedproposed ZMPZMP controller,controller, wherewhere thethe disturbancedisturbance isis reﬂectedreflected in the capture capture-point-point state, state, and and 푧z푐푝cp is used instead of of 푧z푟푒푓re f inin the the observer observer to to reduce reduce the capture-pointcapture-point error. ZMP ZMP reference reference 푧z푟푒푓re f doesdoes not contain information of the terrain because it is generated from the walking pattern. In In contrast, contrast, adjusted adjusted reference reference 푧z푐푝cp containscontains informationinformation about the disturbance disturbance by by reflecting reﬂecting the the state state of ofthe the capture capture point, point, hence hence the robot the robot can be can stabilized be stabilized on tilted on tiltedground. ground.

Figure 7. BlockBlock diagram of proposed ZMP controller.

To evaluateevaluate thethe proposedproposed ZMP ZMP controller, controller, we we compared compared the the proposed proposed and and previous previous controller controller on aon tilted a tilted ground. ground. In evaluatingIn evaluating the the performance performance of theof the ZMP ZMP controller, controller, not not only only the the ZMP ZMP but but also also the attitudethe attitude of the of robotthe robot can becan used be used as a performanceas a performance evaluation, evaluation, so we so measured we measured the attitude the attitude of the of robot the androbot the and ZMP. the AsZMP. shown As inshown the Figure in the8 ,Figure a horizontal 8, a horizontal CoM motion CoM from motion the ZMPfrom referencethe ZMP (Figurereference9) was(Figure commanded 9) was commanded to a robot to placed a robot on placed a tilted on surface, a tilted and surface, both theand ZMP both andthe ZMP attitude and of attitude the robot of (pelvisthe robot attitude) (pelvis were attitude) measured were bymeasured applying by each applying controller. each The controller. ZMP is calculatedThe ZMP is by calculated a Force/Torque by a sensorForce/Torque mounted sensor on the mounted foot sole, on and the the foot attitude sole, and of thethe robotattitude is measured of the robot by theis measured IMU sensor by mountedthe IMU onsensor the pelvis.mounted In on this the paper, pelvis. we In used this the paper, preview we used control the [ 26preview] to generate control the [26 CoM] to generate motion, butthe CoM motionSensorsmotion, 2019 canbut, 19 be,CoM x generatedFOR motion PEER REVIEW usingcan be various generated walking using pattern various generation walking methodspattern generation [28–30]. methods9 of[28 19– 30]. The measured ZMP for each ZMP controller on the tilted surface is shown in Figure 9. The previous ZMP controller does not track the reference ZMP well because the disturbance caused by the tilted surface is not considered in the controller design. On the other hand, because the capture- point controller adjusts 푧푐푝 in order to stabilize the capture point disturbed by the inclination, the proposed ZMP controller provides better ZMP tracking performance than the previous ZMP controller. Figure 10 shows the adjusted 푧푐푝 and reference ZMP. Because the robot was tilted in the y-direction (frontal plane), 푧푐푝 was adjusted more in the y-direction than the ZMP reference. In other words, as 푧푐푝 was adjusted more in an inclined direction than the existing ZMP reference, the ZMP did not move in a tilted direction.

Figure 8. A t testest of the ZMP controller on an inclined surface. We commanded the CoM trajectory of the robot and measured ZMPZMP datadata usingusing thethe proposedproposed andand conventionalconventional ZMPZMP controllers.controllers.

Figure 9. Reference and measured ZMPs with/without capture-point feedback on tilted ground. The reference corresponds to the original walking pattern.

Figure 10. Generated 풛퐜퐩 by capture-point feedback.

Figure 11 shows the robot’s attitude measured in the roll direction according to each ZMP controller. When the proposed ZMP controller is applied, the attitude of the robot changes within 0.9°–2.5°, and when the previous ZMP is applied, the attitude of the robot changes within 1° and 4.5°. In the case of the previous ZMP controller, a large postural change of the robot was shown relative

Sensors 2019, 19, x FOR PEER REVIEW 9 of 19 Sensors 2019, 19, x FOR PEER REVIEW 9 of 19

Sensors 2019, 19, 4194 9 of 19

The measured ZMP for each ZMP controller on the tilted surface is shown in Figure9. The previous ZMP controller does not track the reference ZMP well because the disturbance caused by the tilted surface is not considered in the controller design. On the other hand, because the capture-point controller adjusts zcp in order to stabilize the capture point disturbed by the inclination, the proposed ZMP controller provides better ZMP tracking performance than the previous ZMP controller. Figure 10 shows the adjusted zcp and reference ZMP. Because the robot was tilted in the y-direction (frontal

plane), zcp was adjusted more in the y-direction than the ZMP reference. In other words, as zcp was adjustedFigureFigure more 8.8. AA in ttest anest of inclinedof thethe ZMPZMP direction controllercontroller than onon anan the inclinedinclined existing surface.surface. ZMP WeWe reference, commandedcommanded theZMP thethe CoMCoM did trajectorytrajectory not move ofofin a tilted direction.thethe robotrobot andand measuredmeasured ZMPZMP datadata usingusing thethe proposedproposed andand conventionalconventional ZMPZMP controllers.controllers.

FigureFigureFigure 9. 9.9.Reference ReferenceReference and andand measured measuredmeasured ZMPs ZMPZMPs withs with/withoutwith/without/without capture-point capturecapture--pointpoint feedback feedbackfeedback on onon tilted tiltedtilted ground. ground.ground. The TheThe referencereferencereference corresponds correspondscorresponds to toto the thethe original originaloriginal walking walkingwalking pattern. pattern.pattern.

FigureFigure 10. 10. Generatedz 풛퐜퐩 by capture-point feedback. Figure 10.Generated Generated cp풛퐜퐩by by capture-point capture-point feedback. feedback. Figure 11 shows the robot’s attitude measured in the roll direction according to each ZMP FigureFigure 11 11 showsshows the the robot robot’’ss attitude attitude measured measured in in the the roll roll direction direction accordingaccording to to each each ZMP ZMP controller. When the proposed ZMP controller is applied, the attitude of the robot changes within controller.controller. WhenWhen thethe proposedproposed ZMPZMP controllercontroller isis applied,applied, thethe attitudeattitude ofof thethe robotrobot changeschanges withinwithin 0.9 –2.5 , and when the previous ZMP is applied, the attitude of the robot changes within 1 and 4.5 . 0.90.9◦ °°––2.5°,2.5°,◦ andand whenwhen thethe previousprevious ZMPZMP isis applied,applied, thethe attitudeattitude ofof thethe robotrobot changeschanges withinwithin 1°◦1° andand 4.5°.4.5°.◦ In the case of the previous ZMP controller, a large postural change of the robot was shown relative InIn thethe casecase ofof thethe previousprevious ZMPZMP ccontroller,ontroller, aa largelarge posturposturalal changechange ofof thethe robotrobot waswas shownshown relativerelative to the proposed ZMP controller, because the CoM of the robot was unstable due to unstable ZMP control. On the other hand, in the case of the proposed ZMP controller, because the CoM of the robot is stabilized by stable ZMP control, variation of the posture was relatively small. When the proposed ZMP was applied, the posture change was relatively small, but there was also a slight posture change. Because the ZMP controller stabilizes the CoM in the horizontal direction, an Sensors 2019, 19, x FOR PEER REVIEW 10 of 19

to the proposed ZMP controller, because the CoM of the robot was unstable due to unstable ZMP control. On the other hand, in the case of the proposed ZMP controller, because the CoM of the robot is stabilized by stable ZMP control, variation of the posture was relatively small. Sensors 2019, 19, 4194 10 of 19 When the proposed ZMP was applied, the posture change was relatively small, but there was also a slight posture change. Because the ZMP controller stabilizes the CoM in the horizontal additionaldirection, controller an additional for posture controller stabilization for posture is needed. stabilization The leg lengthis needed. and foot The orientation leg length controller and foot inorientation the following controller Section in4 arethe designedfollowing forSection posture 4 are stabilization. designed for posture stabilization.

FigureFigure 11. 11. Measured orientation orientation of of pelvis pelvis with/wi with/withoutthout capture capture-point-point feedback feedback on ontilted tilted ground. ground. The Theorientation orientation was was measured measured from from an aninertial inertial measurement measurement unit unit mounted mounted on onthe the robot robot pelvis. pelvis.

4.4. Stabilization Stabilization of of Posture Posture InIn Section Section3 ,3 the, the ZMP ZMP controller controller design design for for the the horizontal horizontal stabilization stabilization of of the the CoM CoM is is provided. provided. TheThe proposed proposed ZMP ZMP controller controller showed showed satisfactory satisfactory ZMP ZMP control control performance performance on on inclined inclined ground ground throughthrough stabilization stabilization of of the the CoM CoM in in the the horizontal horizontal direction. direction. However, However, it it was was confirmed confirmed that that the the postureposture could could not not be be maintained maintained at at 0 ◦0°,, which which is is the the desired desired posture. posture. ZMP ZMP control control and and posture posture control control areare required required for for stabilizing stabilizing the the walking walking pattern pattern on on unknown unknown and and uneven uneven terrain. terrain. InIn the the case case of of flat flat ground, ground, it it is is possible possible toto achieveachieve stablestable walkingwalking withwith onlyonly aa ZMPZMP controller.controller. However,However, in in the the case case of of uneven uneven terrain, terrain an, an additional additional posture posture controller controller is is needed needed for for adapting adapting to to the the unknownunknown slope slope ofof thethe floorfloor andand thethe unknownunknown didifferencefference inin heightheight toto maintainmaintain thethe desireddesired posture.posture. BecauseBecause the the CoM CoM trajectory trajectory is is generated generated under under the the assumption assumption that that the the ground ground is flat,is flat, if the if the ground ground is notis not flat, flat, the the robot robot cannot cannot move move according according to the to the desired desired walking walking pattern. pattern. This This causes causes the the instability instability of walking.of walking. Thus, Thus, to stabilizeto stabilize the the posture, posture, we we additionally additionally designed designed a leg-lengtha leg-length controller controller and and a a foot foot orientationorientation controller. controller. In In this this paper, paper, the th aime aim of of the the posture posture control control is tois maintainto maintain the the upright upright posture posture of theof the robot. robot. The The leg-length leg-length control control corresponding corresponding to to the the height height di differencefference of of the the ground ground and and the the foot foot orientationorientation control control corresponding corresponding to to the the slope slope of of the the ground ground that that are are used used to to control control the the posture posture are are describeddescribed in in this this section. section.

4.1.4.1. Leg Leg Length Length Control Control for for Adaptation Adaptation to to Terrain Terrain Elevation Elevation in in the the Double Double Support Support Phase Phase IfIf the the height height of of the the ﬂoor floor varies, varies, as as shown shown in in Figure Figure 12 12a,a, the the leg leg length length of of the the robot robot should should be be adjustedadjusted to to maintain maintain desired desired posture. posture. In In [ [2020],], the the desired desired ZMP ZMP is is generated generated from from linear linear inverted inverted pendulumpendulum tracking, tracking, and and a a force force reference reference is is generated generated to to adjust adjust the the vertical vertical leg leg length. length. InIn a a similar similar approach,approach, we we generate generate the the force force reference reference that that each each foot foot of of the the robot robot should should apply apply on on the the ﬂoor floor using using reference zcp generated through a capture-point control, as follows: reference 푧푐푝 generated through a capture-point control, as follows: 푓d푑== 푘 푀푔, (8) fL퐿 k f푓Mg, d f 푓=푑 = (1(1k−f )푘Mg)푀푔, R 푅 − 푓 , (8)

zcp,Y PY,L k f = − 0 k f 1 , PY,L PY,R ≤ ≤ | − | where subscripts R and L indicate the right and left legs, respectively, M is the mass of the robot, PY,L and PY,R are the positions of the left and right feet along the y direction, respectively, and k f is the gain Sensors 2019, 19, x FOR PEER REVIEW 11 of 19 Sensors 2019, 19, x FOR PEER REVIEW 11 of 19 |푧 −푃 | 푘 = 푐푝,푌 푌,퐿 (0 ≤ 푘 ≤ 1), 푓 ||푧푃푐푝,푌−−푃푃푌,퐿|| 푓 푘푓 = 푌,퐿 푌,푅 (0 ≤ 푘푓 ≤ 1), Sensors 2019, 19, 4194 |푃푌,퐿−푃푌,푅| 11 of 19 where subscripts R and L indicate the right and left legs, respectively, 푀 is the mass of the robot, where subscripts R and L indicate the right and left legs, respectively, 푀 is the mass of the robot, 푃푌,퐿 and 푃푌,푅 are the positions of the left and right feet along the y direction, respectively, and 푘푓 is 푃푌,퐿 and 푃푌,푅 are the positions of the left and right feet along the y direction, respectively, and 푘푓 is thedistributing gain distributin the forceg the to each force foot. to each Note foot. that Notezcp determines that 푧푐푝 determines the foot that the should foot that exert should more forceexert on more the the gain distributing the force to each foot. Note that 푧 determines the foot that should exert more forceﬂoor. on Equation the floor. (8) Equation was derived (8) was from derived [20]. from [20]. 푐푝 force on the floor. Equation (8) was derived from [20].

(a) (b) (a) (b) Figure 12. Leg lengthlength andand foot foo orientationt orientation control control on on uneven uneven terrain. terrain. (a) Height(a) Height diff erencedifference and (andb) both (b) Figure 12. Leg length and foot orientation control on uneven terrain. (a) Height difference and (b) bothheight height and orientationand orientation difference. difference. both height and orientation difference. Using calculated force references f d 푑and f d and푑 measured vertical forces f and f retrieved from Using calculated force references L푓퐿 andR 푓푅 and measured vertical forcesL 푓퐿 andR 푓푅 retrieved Using calculated force references 푓푑 and 푓푑 and measured vertical forces 푓 and 푓 retrieved fromforce /force/torquetorque sensors sensors on the on feet, the the feet, leg the length퐿 leg length can푅 be can adjusted be adjusted via the via following the following PD퐿 control: PD 푅control: from force/torque sensors on the feet, the leg length can be adjusted via the following PD control: . 푑 . . d푑 푢̇ 푧 = 푘푝((∆퐹) − (∆퐹 d)) + 푘푑 ((∆퐹̇ ) − (∆퐹̇ )) − 푘푟푢푧, (9) uz = kp (∆F) ∆F푑 + kd ∆Ḟ ∆F ̇ 푑 kruz,(9) (9) 푢̇ 푧 = 푘푝((∆퐹) −−(∆퐹 )) + 푘푑 ((∆퐹)−− (∆퐹 )−) − 푘푟푢푧, where 푘푝 and 푘푑 are the proportional and derivative gains, respectively, with 푘푝, 푘푑 > 0; and 푘푟 is where k푘p푝and andk d푘푑are are the the proportional proportional and and derivative derivative gains, gains, respectively, respectively, with withk p푘,푝k,d 푘>푑 0;> and0; andkr is푘푟 the is the gain that returns the control input. 푢̇ 푧 to zero. We experimentally tuned the proportional and thegain gain that that returns returns the control the control input uinputz to zero. 푢̇ 푧 to We zero experimentally. We experimentally tuned the tuned proportional the proportional and derivative and derivative gains. Under the assumption that the floor is flat, 푘푟 is adjusted to retrieve the leg length, derivative gains. Under the assumption that the floor is flat, 푘 is adjusted to retrieve the leg length, whichgains. is Under changed the by assumption the control thatinpu thet, back floor to isthe flat, originalkr is adjustedlength.푟 Specifically, to retrieve by the scheduling leg length, the which gain whichis changed is changed by the by control the control input, inpu backt, back to the to original the original length. length. Specifically, Specifically, by schedulingby scheduling the the gain gainkr 푘푟 according to the support phase, as shown in Figure 13, the leg length in the double support phase 푘according푟 according to theto the support support phase, phase as, as shown shown in in Figure Figure 13 13,, the the leg leg length length inin thethe doubledouble supportsupport phase (DSP) is adjusted because the gain 푘푟 is set to small and the adjusted leg length in the DSP returns ((DSP)DSP) is adjusted because thethe gaingain k푘r 푟is is set set to to small small and and the the adjusted adjusted leg leg length length in in the the DSP DSP returns returns to to the original leg length in the single support phase (SSP) because the gain 푘푟 is set to large. In tothe the original original leg leg length length in the in singlethe single support support phase phase (SSP) (SSP) because because the gain thek rgainis set 푘 to푟 large.is set to In addition,large. In addition, if the elevation of the ground is constant, we can set 푘푟 in the SSP to be small enough to addition,if the elevation if the elevation of the ground of the is ground constant, is constant, we can set wek rcanin theset SSP푘푟 in to the be SSP small to enoughbe small to enough keep the to keep the previous control input. The calculated velocity. input, 푢̇ 푧, is integrated into position 푢푧 and keepprevious the previous control input. control The input calculated. The calculated velocity input,velocityuz input,, is integrated 푢̇ 푧, is integrated into position into pouz siandtion added 푢푧 and to added to the vertical position of the right and left feet as 0.5 푢푧 and −0.5 푢푧, respectively. addedthe vertical to the position vertical ofposition the right of andthe right left feet and as left 0.5 feetuz and as 0.50.5 푢푧 uandz, respectively. −0.5 푢푧, respectively. −

푘 Figure 13. GainGain scheduling scheduling according according to to the the supporting supporting phase. phase. By By increasing increasing the the gain gain krr in the single Figure 13. Gain scheduling according to the supporting phase. By increasing the gain 푘r in the single support phase, the modified modified leg length in double support phase returns to its original leg length in support phase, the modified leg length in double support phase returns to its original leg length in single support phase. single support phase. In [20], only proportional gain is used, but increasing the proportional gain to improve the In [20], only proportional gain is used, but increasing the proportional gain to improve the controller performance generates oscillation. By By adding adding derivative derivative gain, gain, the the resulting resulting damping damping effect effect controller performance generates oscillation. By adding derivative gain, the resulting damping effect mitigates oscillation oscillation,, while allowing a high proportional gain. Figure Figure 1414 showsshows the the reference and mitigates oscillation, while allowing a high proportional gain. Figure 14 shows the reference and measmeasuredured forcesforces ofof the the right right foot foot with with and and without without the the derivative derivative gain gain at fixed at fixed proportional proportional gain. gain. Note measured forces of the right foot with and without the derivative gain at fixed proportional gain. that when the PD gain is applied, oscillation of the force is significantly diminished in the DSP. That is, a high P gain is applied for the fast adaptation of the leg length to the height difference of the floor in Sensors 2019, 19, x FOR PEER REVIEW 12 of 19

SensorsNote2019 that, 19 when, 4194 the PD gain is applied, oscillation of the force is significantly diminished in the12 ofDSP. 19 That is, a high P gain is applied for the fast adaptation of the leg length to the height difference of the floor in the DSP, and the vibration due to the high P gain is attenuated by the D gain. As mentioned the DSP, and the vibration due to the high P gain is attenuated by the D gain. As mentioned earlier, in earlier, in the SSP, the damping effect does not appear, because the third term of Equation (9), which the SSP, the damping effect does not appear, because the third term of Equation (9), which retrieves the retrieves the adjusted leg length relative to the original leg length, is dominant. Because the leg length adjusted leg length relative to the original leg length, is dominant. Because the leg length is already is already adapted to the height difference in the DSP, the damping effect is not significant in the SSP. adapted to the height difference in the DSP, the damping effect is not significant in the SSP.

FigureFigure 14. 14.Leg Leg length length controller controller with with proportional proportional and and proportional-derivative proportional-derivative (PD) (PD control.) control.

4.2.4.2. Foot Foot Orientation Orientation Control Control for for Adaptation Adaptation to to Ground Ground Slope Slope BesidesBesides the the leg leg length, length, the the foot foot orientation orientation must must be be controlled controlled to to maintain maintain the the desired desired robot robot attitude,attitude, as as shown shown in in Figure Figure 12 12b.b. Humans Humans generate generate the the desired desired torque torque on on the the foot foot according according to to the the currentcurrent state state and and adapt adapt to theto the shape shape of the of ﬂoor.the floor. Similarly, Similarly, we calculate we calculate the desired the desired torque torque of the robotof the footrobot from foot the from diﬀerence the difference between between the reference the reference and adjusted and ZMPs,adjusted and ZMPs, the foot and orientation the foot orientation is adjusted is toadjuste track thed to desired track the toque. desired toque. TheThe torque torque reference reference for for the they yaxis axis of of the the right right foot foot is is the the same same as as in in Equation Equation (9), (9), and and torques torques τd 푑, τd 푑, and τd 푑 are calculated analogously. R휏,푅x,푥, L휏,퐿y,푦, and L휏,퐿x,푥 are calculated analogously. 푑 푑 푑 d휏푅,푦== 푓d푅 (푧푐푝,푥 − 푧푟푒푓,푥=) =d(푓푅 (∆)휉푥), (10) τR,y fR zcp,x zre f ,x fR ∆ξx , 푑 − 푑 푓d = 푘 푀푔, d푓 = (1 − 푘 )푀푔, f 퐿= ktMg푡 , f 푅= (1 kt)Mg푡 , L R − (10)

z|re푧푟푒푓 f ,y ,푦P−y푃,L푦 ,퐿| 푘 = − (0 ≤ 푘 ≤ 1). kt 푡= |푃 −푃 |(0 kt 푡 1). Py L푦,퐿Py R푦 ,푅 , − , ≤ ≤ TheThe desired desired torque torque is is tracked tracked by by the the following following PD PD control: control: 푑 푑 푢̇ 표푟푖 = 푘푝((휏 − 휏)) + 푘푑((휏̇ − 휏̇)) − 푘푟푢표푟푖, (11) . d . d . u = kp τ τ + k ((τ τ)) kru , (11) ori − d − − ori 푑 where 푘푝 and 푘푑 are the proportional and derivative gains, respectively, with 푘p, 푘푑 > 0, and 휏 where k and k are the proportional and derivative gains, respectively, with k , k > 0, and τd and τ and τ arep the ddesired and measured torques, respectively. Like for leg lengthp control,d the calculated are the desired and measured torques, respectively. Like for leg length control, the calculated velocity velocity. input 푢̇ 표푟푖 is integrated into orientation 푢표푟푖 and added to the orientation of the input u is integrated into orientation u and added to the orientation of the corresponding foot. correspondingori foot. ori 5. Experiments and Results 5. Experiments and Results To test the proposed balance control framework, we conducted three experiments. Each experiment To test the proposed balance control framework, we conducted three experiments. Each was performed on the DRC-HUBO+ with all of the balance controllers designed in Sections3 and4 experiment was performed on the DRC-HUBO+ with all of the balance controllers designed in Section simultaneously applied. The ﬁrst experiment consisted of the robot maintaining balance on a sloping 3 and Section 4 simultaneously applied. The first experiment consisted of the robot maintaining wooden plank. The second and third experiments were terrain-blind walking on uneven terrain and a balance on a sloping wooden plank. The second and third experiments were terrain-blind walking stony area without terrain information, respectively. Videos from the experiments can be seen in [24]. on uneven terrain and a stony area without terrain information, respectively. Videos from the experiments can be seen in [24].

SensorsSensors 20192019,, 1919,, 4194x FOR PEER REVIEW 1313 of 1919

5.1. Hardware Implementation for Real-Time Control 5.1. Hardware Implementation for Real-Time Control The control cycle of the framework is 5 ms and runs on an Intel NUC5i7RYH embedded computerThe control in the cyclebody of of the the framework robot; the isoperating 5 ms and system runs on of an the Intel computer NUC5i7RYH is Linux embedded Ubuntu computer and the inrobot the program body of’s the development robot; the environment operating system is PODO of the[15], computer which was is developed Linux Ubuntu by the andKAIST the HUBO robot program’sLab and guarantees development hard environment real-time control.is PODO A [15 ZMP], which reference was developed is generated by the at KAIST the capture HUBO-point Lab andcontroller guarantees every hard5 ms real-timeand the balance control. controller A ZMP reference receives isthe generated reference at and the generates capture-point control controller inputs. everyThe control 5 ms and input the for balance the position controller of receivesthe CoM the is calculated reference and from generates the ZMP control controller inputs. of Thethe balance control inputcontroller for theand position the control of theinputs CoM for is the calculated foot position from in the the ZMP z direction controller and the of thefoot balance orientation controller in the andx and the y directions control inputs are calculated for the foot from position the leg in length the z direction and foot andorientation the foot controller orientations. Control in the x inputsand y directionsdesigned in are the calculated task space from are the converted leg length to andjoint foot angle orientation references controllers. through inverse Control kinematics. inputs designed Each injoint the angle task spacereference are convertedgenerated tois jointtransmitted angle references to the motor through driver inverse of each kinematics. joint at 200 Each Hz through joint angle the referencecan, and the generated angle reference is transmitted is track toed the through motor driverthe PD of position each joint control at 200 in Hz each through motor thedriver. can, Because and the anglethe position reference controller is tracked has through a control the PD period position of 1 control kHz, inthe each joint motor angle driver. references Because at the 200position Hz are controllerinterpolated has to a control1 kHz. A periods the robot of 1 kHz, is moved the joint byangle the motors, references the atsensor 200 Hz data are from interpolated the gyro to and 1 kHz. F/T Assensor thes robot are transmitte is movedd by to the the motors, embedded the sensorPC every data 5 fromms. The the control gyro and framework F/T sensors wa ares programmed transmitted toin thethe embeddedC ++ language. PC every 5 ms. The control framework was programmed in the C ++ language.

5.2.5.2. Balance Control under Varying Slope

FigureFigure 1515 showsshows thethe robotrobot placedplaced onon aa woodenwooden board with a slope varying between 5° 5◦ and –5–5°.◦. WeWe evaluatedevaluated thethe performanceperformance ofof thethe proposedproposed balancebalance controllercontroller whilewhile arbitrarilyarbitrarily modifyingmodifying the slope.slope. Figure 16 shows the experimental results corresponding to Figure 1515 andand slantingslanting thethe woodenwooden plankplank fromfrom 55°◦ toto –5–5°◦ in the roll direction.direction. As shown in the top-lefttop-left graphgraph inin FigureFigure 1616,, whenwhen aa personperson tiltstilts aa woodenwooden boardboard atat 55°◦ (between(between 56 s and 59 s),s), thethe robotrobot tiltstilts inin thethe plusplus rollroll directiondirection (between(between 56 s and 59 s). At this moment, the z is adjusted through the capture-point control to maintain the 56 s and 59 s). At this moment, the 푧cp푐푝 is adjusted through the capture-point control to maintain the balance of the robot, as shown in the upper-right graph of Figure 16. The adjusted zcp is used for the balance of the robot, as shown in the upper-right graph of Figure 16. The adjusted 푧푐푝 is used for the ZMPZMP controller, andand thethe ZMPZMP controllercontroller adjustsadjusts thethe y-directiony-direction CoMCoM (CoMy)(CoMy) toto keep y-directiony-direction ZMP (ZMPy) at zero. Simultaneously with the ZMP control, zcp is input to the foot orientation controller and (ZMPy) at zero. Simultaneously with the ZMP control, 푧푐푝 is input to the foot orientation controller theand leg the length leg length controller, controller, and the and foot the orientation foot orientation and leg and length leg length are adjusted are adjusted to maintain to maintain the robot’s the attituderobot’s attitude at zero. at The zero. foot Th controllere foot controller adjusted adjusted the direction the direction of both of feet both by feet 5◦ in by the 5° rollin the direction roll direction as the ﬂooras the tilted floor (shown tilted in(shown the bottom-right in the bottom graph-right in Figure graph 16 in), Figure and the 16 leg), and length the controller leg length adjusted controller the right foot and left foot positions in the vertical direction by 0.5uz and –0.5uz, respectively (shown in adjusted the right foot and left foot positions in the vertical direction by 0.5푢푧 and –0.5푢푧, respectively the(shown bottom-left in the bottom graph in-left Figure graph 16 in). Figure When the16). slope When in the opposite slope in direction, opposite we direction, can see thatwe can the balancesee that controllerthe balance operates controller on theoperates same on principle. the same principle.

Figure 15.15. Snapshots of balance control experiment under varyingvarying inclination.inclination.

Sensors 2019, 19, 4194 14 of 19 SensorsSensors 2019 2019, ,19 19, ,x x FOR FOR PEER PEER REVIEW REVIEW 1414 ofof 1919

FigureFigure 16. 16. ExperimentalExperimental results resultsresults from from thethe robot robot onon aa varyingvarying inclinincline.inclinee. .

5.3.5.3. Terrain Terrain-BlindTerrain--BlindBlind WalkingWalking onon UnevenUneven TerrainTerrain InIn this this experiment, experiment, terrain terrain-blindterrain--blindblind walking walking teststests werewere performed performed onon uneven uneven terrain terrain,terrain,, as as shown shown in in FigureFigure 17 17... The The unevenuneven terrainterrain consisted consistconsisteded of ofof a aa ﬁrst firstfirst section sectionsection with withwith a aa local locallocal slope slopeslope and andand a aa second secondsecond section sectionsection with withwith a aglobala globalglobal slope. slope.slope. The TheThe robot robotrobot was wawas commandeds commandedcommanded to toto walk walkwalkon onon the thethe local lolocalcal slope slopeslope and andand the thethe global globalglobal slope. slope.slope. InIn thethe firstﬁrstfirst section, aluminum and plastic plates with an area smaller than the foot sole area (24 cm 16 cm) and section,section, aluminumaluminum andand plasticplastic platesplates withwith anan areaarea smallersmaller thanthan thethe footfoot solesole areaarea (24(24 cmcm × ×× 1616 cmcm)) andand aa heightheight of of between between 1 11 cm cm and and 2 22 cm cm we werewerere placed placed randomly, randomly, and and the the second second section sectionsection ha hadhadd a aa slope slopeslope of of 8°. 88°.◦. TheThe robotrobot walkedwalked withwith aa stridestride ofof 0.150.15 mm andand aa steppingstepping cyclecycle ofof 11 ss (single(single supportsupport phasephase ofof 0.80.8 ss andand doubledouble supportsupport phasephase ofof 0.20.2 s).s).

FigureFigure 17. 17. SnapshotSnapshot of ofof walking walkingwalking experiment experimentexperiment onon uneven uneven terrainterrain withwith varyingvarying locallocal and and global global slopes.slopes.

FigureFigure 18 1818 shows showsshows capture-point capturecapture--pointpoint error errorerror∆ ξ ∆∆measured휉휉 measuredmeasured along alongalong the thextheand xx andyanddirections, yy directionsdirections from,, from whenfrom whenwhen the robot thethe walkedroborobott walkedwalked on the onon uneven thethe unevenuneven terrain terrain interrain this experiment.inin thisthis experiment.experiment. The error TheThe variation errorerror variationvariation with respect withwith to respectrespect the x-direction toto thethe xx-- directionisdirection relatively is is relatively relatively large because large large thebecause because inclination the the inclination inclination of the pitch of of the the direction pitch pitch direction direction is large is whenis large large the when when robot the the walks robot robot onwalks walks the onglobalon thethe slopeglobalglobal surface slopeslope surfacesurface (from 12 (from(from s). Figure 1212 s).s). 19FigureFigure shows 1919 the showsshows foot thethe orientation footfoot orientationorientation input pitch inputinput changing pitchpitch changingchanging to 8◦ after toto 128°8° after s,after almost 1212 s,s, almostagreeingalmost agreeingagreeing with the withwith global thethe globalglobal slope. slope.slope. In the InIn experiment, thethe experiment,experiment, we set wewe a setset small aa smallsmall time-constant timetime--constantconstantT for TT forthefor the swingingthe swingingswinging foot, foot,foot, such suchsuch that itsthatthat orientation itsits orientation orientation returned returnedreturned close close toclose 0◦. to Figureto 0°.0°. FigureFigure 20 shows 2020 shows theshows nominal thethe nominalnominal height heighttrajectoryheight trajectorytrajectory (blue dot (blue(blue line dotdot in lineline the infigure)in thethe figure)figure) and modified andand modifiedmodified height heightheight trajectory trajectorytrajectory (red line (red(red in lineline the inin figure) thethe figufigu ofre)re) both ofof bothfeet.both feet.Thefeet. nominal TheThe nominalnominal foot heightfootfoot heightheight trajectory trajectorytrajectory is modified isis modifiedmodified by leg-length byby legleg--lengthlength control. control.control. As the AsAs floor thethe floor isfloor higher isis higherhigher than thanthethan expected thethe expectedexpected landing landinglanding position positionposition on the onon global thethe globalglobal slope, slope,slope, the foot thethe position ffoooott positionposition is high. isis high. Figurehigh. FigureFigure 21 shows 2121 showsshows the ZMP thethe ZMPreference,ZMP reference,reference, measured measuredmeasured ZMP and ZMPZMP CoM andand trajectory. CoMCoM trajectrajec Thetory.tory. grey TheThe rectangles greygrey rectanglesrectangles in the graph inin thethe represent graphgraph therepresentrepresent footprints. thethe footprints.Becausefootprints. the BecauseBecause front edge thethe of frontfront each edgeedge foot of landedof eacheach footfirstfoot onlandedlanded the slope firstfirst on whenon thethe the slopeslope robot whenwhen was thethe walking robotrobot waonwas thes walkingwalking global onon thethe globalglobal slope,slope, thethe changechange ofof ZMPZMP wawass relativelyrelatively large.large. However,However, thethe robotrobot cancan stablystably walkwalk onon uneunevenven terrainterrain becausebecause thethe ZMPZMP waswas maintainedmaintained wellwell withinwithin thethe boundaryboundary ofof footfoot byby thethe balancebalance controller.controller.

Sensors 2019, 19, 4194 15 of 19

slope,Sensors the 2019 change, 19, x FOR of ZMPPEER REVIEW was relatively large. However, the robot can stably walk on uneven terrain15 of 19 because the ZMP was maintained well within the boundary of foot by the balance controller. Sensors 2019,, 19,, xx FORFOR PEERPEER REVIEWREVIEW 15 of 19

Figure 18. Measured capture-point error along x and y directions . FigureFigure 18. 18. MeasuredMeasured capture capture-point--point error error along along x x and and y y directions directions...

Figure 19. Foot orientation along x (roll) and y (pitch) axes. Figure 19. Foot orientation along x (roll) and y (pitch) axes. Figure 19. Foott orientationorientation alongalong xx (roll)(roll) andand yy (pitch)(pitch) axes.axes.

Figure 20. Feet height trajectory under varying local and global slopes. Figure 20. Feet height trajectory under varying local and global slopes. Figure 20. Feet heightight trajectorytrajectory underunder varyingvarying locallocal andand globalglobal slopes.slopes.

Sensors 2019, 19, 4194 16 of 19 Sensors 2019,, 19,, xx FORFOR PEERPEER REVIEWREVIEW 16 of 19

Figure 21. ZMPZMP reference, reference, measured measured ZMP, ZMP, and CoM trajectory under varying local and global slopes. 5.4. Terrain-Blind Walking on a Stony Area and a Lawn 5.4. Terrain-Blind Walking on a Stony Area and a Lawn To test the performance of the proposed balance control in a realistic environment, we conducted a To test the performance of the proposed balance control in a realistic environment, we conducted terrain-blind walking experiment on a stony area and a lawn, as shown in Figures 22 and 23. The stones a terrain-blind walking experiment on a stony area and a lawn,, as shown in Figures 22 and 23.. TheThe were 5–10 cm width-wise, 5–10 cm in depth, and 2 cm to 4 cm in height. The robot walked with a stride stones were 5–10 cm width-wise,, 55–10 cm in depth, and 2 cm to 4 cm in height. The robot walked of 0.15 m and a stepping cycle of 1 s (single support phase of 0.8 s and double support phase of 0.2 s). with a stride of 0.15 m and a stepping cycle of 1 s (single support phase of 0.8 s and double support As stones move when the robot walks over them, a quick posture stabilization should be accomplished. phase of 0.2 s). As stones move when the robot walks over them, a quick posture stabilization should With the proposed balance control framework, the robot was able to stably walk on the stony area. be accomplished.. With the proposed balance control framework,, thethe robotrobot waswas ableable toto stablystably walkwalk Furthermore, we also tested the robot walking on lawn. The lawn was relatively compliant, unlike the on the stony area. Furthermore, we also tested the robot walking on lawn. The lawn was relatively stony areas, but this had no signiﬁcant impact on walking performance, and the robot walked stably compliant, unlike the stony areas, but this had no significant impact on walking performance, and on the lawn. Related experimental videos can be seen in [29]. the robot walked stably on the lawn. Related experimental videos can be seen in [29].

Figure 22. SnapshotsSnapshots from from the walking experiment on a stony area.

Sensors 2019, 19, 4194 17 of 19 Sensors 2019, 19, x FOR PEER REVIEW 17 of 19

Figure 23. Snapshots from the walking experiment onon aa lawn.lawn.

6. ConclusionsConclusions and Future Work To realize the stable stable walking walking of of a ahumanoid humanoid robot robot despite despite disturbances disturbances caused caused by unknown by unknown and anduneven uneven terrain, terrain, we designed we designed a balance a balance-control-control framework framework based based on the on capture the capture-point-point control. control. The Theproposed proposed framework framework consists consists of a ofcapture a capture-point-point controller controller and anda balance a balance controller controller that that tracks tracks the thegenerated generated reference. reference. The reference The reference of each of balance each balance controller controller is derived is derived from the from desired the desired ZMP, which ZMP, whichis in turn is in generated turn generated by capture by capture-point-point control. control. By using By usingthat control that control approach, approach, the horizontal the horizontal CoM CoMand desired and desired posture posture are stabilized are stabilized on un oneven uneven terrain. terrain. The Theproposed proposed control control framework framework enabled enabled the themodified modiﬁed DRC DRC-HUBO-HUBO+ to stably+ to stably walk on walk varying on varying local/global local/ globalslopes, slopes,a stony aarea, stony and area, lawn and without lawn withoutterrain information. terrain information. Moreover, Moreover, this control this framework control framework can be easily can implemented be easily implemented on other humanoid on other humanoidrobots to improve robots to walkingimprove walking performance performance,, given its given simplicity its simplicity and robustness and robustness,, guaranteed guaranteed by the by thecapture capture-point-point control. control. We consider consider that that the the control control robustness robustness can be can further be improved further improved in terms of in walking terms performance of walking ifperformance combined with if combined foot-placement with foot control.-placement Hence, control. we will Hence, include we a walking will include pattern a walking with foot pattern placement with controlfoot placement into the proposedcontrol into balance the proposed control framework balance control in a future framework development. in a future Furthermore, development. the authorsFurthermore believe, the it authors will contribute believe it to will more contribute robust walking to more whenrobust combined walking when with acombined terrain sensing with a algorithmterrain sensi inng the algorithm future. in the future.

Supplementary Materials: Experiment videos are available at https://youtu.be/4xk-UP4Qcfo. Supplementary Materials: Experiment videos are available at https://youtu.be/4xk-UP4Qcfo. Author Contributions: Conceptualization, H.-M.J.; methodology, H.-M.J.; implementation, H.-M.J.; validation, H.-M.J.;Author formal Contributions: analysis, Conceptualization H.-M.J. and J.-H.O.;, H.M.J.; writing methodology, and editing, H.-M.J.; H.M.J.; funding implementation, and supervision, H.M.J.; J.-H.O. validation, H.M.J.; formal analysis, H.M.J. and J.H.O.; writing and editing, H.M.J.; funding and supervision, J.H.O.

Funding: This research was funded by the project, “Development of Core Technologies and a Standard Platform for a Humanoid Robot (10060103)” of the Ministry of Trade Industry and Energy, Republic of Korea.

Sensors 2019, 19, 4194 18 of 19

References

1. Holt, M.; Campbell, R.J.; Nikitin, M.B. Fukushima Nuclear Disaster; Congressional Research Service: Washington, DC, USA, 2012. 2. Pratt, G.; Manzo, J. The DARPA robotics challenge [competitions]. IEEE Robot. Autom. Mag. 2013, 20, 10–12. [CrossRef] 3. Vukobratović,M.; Borovac, B. Zero-moment point—Thirty five years of its life. Int. J. Hum. Robot. 2004, 1, 157–173. [CrossRef] 4. Nagasaka, K.I. Stabilization of dynamic walk on a humanoid using torso position compliance control. In Proceedings of the 17th Annual Conference on Robotics Society of Japan, Osaka, Japan, 1 January 1999. 5. Choi, Y.; Kim, D.; Oh, Y.; You, B.J. Posture/walking control for humanoid robot based on kinematic resolution of com Jacobian with embedded motion. IEEE Trans. Robot. 2007, 23, 1285–1293. [CrossRef] 6. Kim, J.Y.; Park, I.W.; Oh, J.H. Experimental realization of dynamic walking of the biped humanoid robot KHR-2 using zero moment point feedback and inertial measurement. Adv. Robot. 2006, 20, 707–736. [CrossRef] 7. Martinez, S.; Garcia-Haro, J.; Victores, J.; Jardon, A.; Balaguer, C. Experimental robot model adjustments based on force–torque sensor information. Sensors 2018, 18, 836. [CrossRef][PubMed] 8. Ott, C.; Roa, M.A.; Hirzinger, G. Posture and balance control for biped robots based on contact force optimization. In Proceedings of the 2011 11th IEEE-RAS International Conference on Humanoid Robots, Bled, Slovenia, 26–28 October 2011; pp. 26–33. 9. Feng, S.; Xinjilefu, X.; Atkeson, C.G.; Kim, J. Optimization based controller design and implementation for the Atlas robot in the DARPA Robotics Challenge finals. In Proceedings of the 2015 IEEE-RAS 15th International Conference on Humanoid Robots (Humanoids), Seoul, Korea, 3–5 November 2015; pp. 1028–1035. 10. Kuindersma, S.; Deits, R.; Fallon, M.; Valenzuela, A.; Dai, H.; Permenter, F.; Koolen, T.; Marion, P.; Tedrake, R. Optimization-based locomotion planning, estimation, and control design for the Atlas humanoid robot. Auton. Robot. 2016, 40, 429–455. [CrossRef] 11. Koolen, T.; Bertrand, S.; Thomas, G.; De Boer, T.; Wu, T.; Smith, J.; Englsberger, J.; Pratt, J. Design of a momentum-based control framework and application to the humanoid robot Atlas. Int. J. Hum. Robot. 2016, 13, 1650007. [CrossRef] 12. Englsberger, J.; Werner, A.; Ott, C.; Henze, B.; Roa, M.A.; Garofalo, G.; Burger, R.; Beyer, A.; Eiberger, O.; Schmid, K.; et al. Overview of the torque-controlled humanoid robot TORO. In Proceedings of the 2014 IEEE-RAS International Conference on Humanoid Robots, Madrid, Spain, 18–20 November 2014; pp. 916–923. 13. Boston Dynamics. Atlas Humanoid Robot. Available online: https://www.bostondynamics.com/atlas (accessed on 20 November 2018). 14. Radford, N.A.; Strawser, P.; Hambuchen, K.; Mehling, J.S.; Verdeyen, W.K.; Donnan, A.S.; Holley, J.; Sanchez, J.; Nguyen, V.; Bridgwater, L.; et al. Valkyrie: Nasa’s first bipedal humanoid robot. J. Field Robot. 2015, 32, 397–419. [CrossRef] 15. Jung, T.; Lim, J.; Bae, H.; Lee, K.K.; Joe, H.M.; Oh, J.H. Development of the Humanoid Disaster Response Platform DRC-HUBO+. IEEE Trans. Robot. 2018, 34, 1–17. [CrossRef] 16. Kaneko, K.; Kaminaga, H.; Sakaguchi, T.; Kajita, S.; Morisawa, M.; Kumagai, I.; Kanehiro, F. Humanoid Robot HRP-5P: An electrically actuated humanoid robot with high-power and wide-range joints. IEEE Robot. Autom. Lett. 2019, 4, 1431–1438. [CrossRef] 17. Kojima, K.; Karasawa, T.; Kozuki, T.; Kuroiwa, E.; Yukizaki, S.; Iwaishi, S.; Ishikawa, T.; Koyama, R.; Noda, S.; Sugai, F.; et al. Development of life-sized high-power humanoid robot jaxon for real-world use. In Proceedings of the 2015 IEEE-RAS 15th International Conference on Humanoid Robots (Humanoids), Seoul, Korea, 3–5 November 2015. 18. Lim, H.O.; Setiawan, S.A.; Takanishi, A. Position-based impedance control of a biped humanoid robot. Adv. Robot. 2004, 18, 415–435. [CrossRef] Sensors 2019, 19, 4194 19 of 19

19. Kim, J.Y.; Park, I.W.; Oh, J.H. Walking control algorithm of biped humanoid robot on uneven and inclined ﬂoor. J. Intell. Robot. Syst. 2007, 48, 457–484. [CrossRef] 20. Kajita, S.; Morisawa, M.; Miura, K.; Nakaoka, S.; Harada, K.; Kaneko, K.; Kanehiro, F.; Yokoi, K. Biped walking stabilization based on linear inverted pendulum tracking. In Proceedings of the 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems, Taipei, Taiwan, 18–22 October 2010; pp. 4489–4496. 21. Englsberger, J.; Ott, C.; Roa, M.A.; Albu-Schäﬀer, A.; Hirzinger, G. Bipedal walking control based on capture point dynamics. In Proceedings of the 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems, San Francisco, CA, USA, 25–30 September 2011; pp. 4420–4427. 22. Englsberger, J. Combining Reduced Dynamics Models and Whole-Body Control for Agile Humanoid Locomotion. Ph.D. Thesis, Technische Universität München, Munich, Germany, 2016. 23. Hong, Y.D. Capture Point-Based Controller Using Real-Time Zero Moment Point Manipulation for Stable Bipedal Walking in Human Environment. Sensors 2019, 19, 3407. [CrossRef][PubMed] 24. Robust Balance Controller for Stable Walking on Uneven Terrain. Available online: https://youtu.be/4xk- UP4Qcfo (accessed on 23 April 2019). 25. Kajita, S.; Hirukawa, H.; Harada, K.; Yokoi, K. Introduction to Humanoid Robotics; Springer: Berlin/Heidelberg, Germany, 2014; Volume 101, p. 2014. 26. Kajita, S.; Kanehiro, F.; Kaneko, K.; Fujiwara, K.; Harada, K.; Yokoi, K.; Hirukawa, H. Biped walking pattern generation by using preview control of zero-moment point. In Proceedings of the 2003 IEEE International Conference on Robotics and Automation (Cat. No. 03CH37422), Taipei, Taiwan, 14–19 September 2003. 27. Kim, J.Y.; Jun, B.H.; Park, I.W. Six-legged walking of ‘Little Crabster’ on uneven terrain. Int. J. Precis. Eng. Manuf. 2017, 18, 509–518. [CrossRef] 28. Feng, S.; Xinjilefu, X.; Atkeson, C.G.; Kim, J. Robust dynamic walking using online foot step optimization. In Proceedings of the 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Daejeon, Korea, 9–14 October 2016; pp. 5373–5378. 29. Urbann, O.; Hofmann, M. A reactive stepping algorithm based on preview controller with observer for biped robots. In Proceedings of the 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Daejeon, Korea, 9–14 October 2016; pp. 5324–5331. 30. Joe, H.M.; Oh, J.H. Balance recovery through model predictive control based on capture point dynamics for biped walking robot. Robot. Auton. Syst. 2018, 105, 1–10. [CrossRef]

© 2019 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).