Bipedal Hopping: Reduced-Order Model Embedding Via Optimization-Based Control

Bipedal Hopping: Reduced-order Model Embedding via Optimization-based Control

Xiaobin Xiong and Aaron D. Ames

Abstract— This paper presents the design and validation of ݖ controlling hopping on the 3D bipedal robot Cassie. A spring- ,ŝƉǇĂǁ ݕ mass model is identified from the kinematics and compliance of ݔ the robot. The spring stiffness and damping are encapsulated ,ŝƉƌŽůů by the leg length, thus actuating the leg length can create and ,ŝƉƉŝƚĐŚ control hopping behaviors. Trajectory optimization via direct collocation is performed on the spring-mass model to plan <ŶĞĞƉŝƚĐŚ jumping and landing motions. The leg length trajectories are utilized as desired outputs to synthesize a control Lyapunov ^ŚŝŶƉŝƚĐŚ function based quadratic program (CLF-QP). Centroidal angular momentum, taking as an addition output in the CLF- dĂƌƐƵƐƉŝƚĐŚ QP, is also stabilized in the jumping phase to prevent whole body rotation in the underactuated flight phase. The solution to the CLF-QP is a nonlinear feedback control law that achieves dynamic jumping behaviors on bipedal robots with dŽĞƉŝƚĐŚ compliance. The framework presented in this paper is verified experimentally on the bipedal robot Cassie.

I.INTRODUCTION Fig. 1. Hopping on Cassie [1] (left) and its coordinate system (right). Reduced-order models such as the canonical Spring Loaded Inverted Pendulum (SLIP) have been widely applied for controlling walking [2] [3] [4], running [5] and hopping that achieve this behavior on the full-order model. Roughly [6] of legged robots. One important benefit of using low speaking, the springs on the physical robot are expected to order dynamical systems for control is that it renders the gait behave similarly to the spring in the spring-mass model when and motion generation problems for legged robots computa- the robot tracks the leg length trajectory. This motivates tionally tractable. However, reduced-order models are often defining the leg length trajectory as a desired output on directly implemented on the full-order model of the robot, each leg and thus formulate a control Lyapunov function e.g., through inverse kinematics [7] or inverse dynamics [8], based quadratic program (CLF-QP) [11] [12] for output without a faithful connection to the structure and morphology stabilization. The end result is a nonlinear optimization-based of the robot. controller that represents the reduced-order dynamics in the In this paper, we present an approach to identifying the full-order model of the robot. spring-mass model for bipedal robots with mechanical com- The QP formulation for hopping is inspired by the ap- pliance, and synthesizing nonlinear controllers by embedding proach for walking in [4], wherein the SLIP dynamics is the spring-mass model into the full-order dynamics. Specifi- embedded onto the center of mass (COM) dynamics of the cally, the spring in the spring-mass model comes from view- full robot via an equality constraint in the QP. The difference ing each leg as a deformable prismatic spring as motivated of our approach is that the hopping dynamics is embedded by the mechanical design of robots with compliance [9]. We by taking the leg length trajectory as a desired output, which borrow the idea of end-effector stiffness from manipulation becomes an inequality CLF constraint in the QP, rendering community [10], and formally derive the stiffness/damping a more feasible QP formulation. Additionally, the hopping of the leg spring from the compliant components in the leg as motion naturally requires a consideration on momentum functions of robot configurations. This facilitates the spring- regulation due to the conservation law on centroidal angular mass model being virtually actuated by changing robot momentum [13] in flight phase. This is done by including the configurations, i.e., by changing the leg length on the spring- angular momentum as an output to stabilize in the CLF-QP. mass model. Trajectory optimization can thus be utilized to The proposed approach is successfully implemented on create hopping behaviors on the spring-mass model. the 3D underactuated bipedal robot Cassie (see Fig. 1) in The planned leg length trajectory from the spring-mass both simulation and experiment [1], achieving the hopping model encodes the underactuation of the leg compliance of on Cassie with ground clearance of ∼ 7 inches and air-time the robot, and can therefore be used to synthesize controllers of ∼ 0.423s. The ground reaction force and toe-off timing of hopping motions on the robot match closely with these *This work is supported by NSF grant NRI-1526519. of the spring-mass model. This further indicates a faithful The authors are with the Department of Mechanical and Civil Engineering, California Institute of Technology, Pasadena, CA 91125 construction and embedding of the reduced-order model onto [email protected], [email protected] the full order model of the robot. (a) II.ROBOT MODEL (b) hip pitch 𝑞 achilles rod 𝑞 hip link The Cassie-series robot from Agility Robotics [14] is a 𝑞 full 3D bipedal robot that is designed to be agile and robust. knee link 𝑞 Like its predecessor ATRIAS [9], Cassie is designed with heel spring concentrated mass at its pelvis and lightweight legs with leaf shin spring 𝑞 shin link springs and closed kinematic chains. The mechanical design, 𝑞 toe motor thus, embodies the SLIP model [2]. From the perspective 𝑞 main chain tarsus link of model-based control, the compliant closed chain on each 𝑞 leg can, however, create additional complexities. Therefore, toe link plantar rod 𝑞 rigid model [4] or overly simplified model [9] is oftentimes motor joints 𝑞 spring joints pure passive joints 𝑞 applied. Here, we present the full body dynamics model with justifiable simplifications. Fig. 2. (a,b) Cassie’s leg and its model. As shown in Fig. 1, Cassie has five motor joints (with the axis of rotation shown in red) on each leg, three of which locate at the hip and the other two are the knee and toe pitch. spring-mass model. It is expected that the stiffness of the leg Fig. 2(a) and (b) provide a close look at the leg kinematics spring changes with different robot configurations, thus we and the abstract model, respectively. We model the shin and explicitly derive the leg stiffness Kleg as a function of joint heel springs as torsion springs at the corresponding deflection angles, the analogy of which is the end-effector stiffness for robotic(a) manipulators [10].(b) With an eye towards the motion axes. Therefore the spring torques are: leg spring planning for the spring-mass model, Kleg is approximated by τshin/heel = kshin/heelqshin/heel + dshin/heelq˙shin/heel, (1) a polynomial𝐿 function of leg length L. Lastly, we present the trajectory𝐿 optimization via direct collocation for the spring- where kshin/heel, dshin/heel are the stiffness and damping, pro- 𝐾𝐿 vided by the manufacturer [14]. Since the achilles rod is mass model. stance width very lightweight, we ignore the achilles rod and replace it by A. Leg Stiffness and Leg Length setting a holonomic constraint hrod on the distance between The leg stiffness Kleg is the resistance of the leg to the connectors (one locates on the inner side of hip joint, external𝑠 forces. The complementary concept is called leg the other locates at the end of the heel spring). The plantar −1 compliance Cleg = Kleg . When the leg is under external rod is also removed and the actuation is applied to the toe load at the foot, the leg deforms due to compliant elements pitch directly thanks to the parallel linkage design. These in the leg. Assuming that we only consider the transitional two simplifications removed unnecessary passive joints and deformations, the external force can be calculated by, associated configuration variables. As a consequence, the configuration of the leg can be described only by five motor Fext = Klegδ, (4) joints, two spring joints and a passive tarsus joint. The total 3 3×3 3 where Fext ∈ R ,Kleg ∈ R and δ ∈ R . Under the number of degrees of freedom of the floating base model is assumption that the deformation is small and only happens then n = 8 × 2 + 6 = 22. The dynamics can be derived from at the joints, the leg deformation δ can be mapped from joint the Euler-Lagrange equation with holonomic constraints as: deformations ∆q by the foot Jacobian as, T T M(q)¨q + H(q, q˙) = Bu + Js τs + Jh,vFh,v, (2) δ = J∆q, (5) Jh,v(q)¨q + J˙h,v(q)q ˙ = 0, (3) where J ∈ R3×n and ∆q ∈ Rn. Let τ denotes the moments n T where q ∈ R , M(q) is the mass matrix, H(q, q˙) is the at the joints caused by the external load, thus τ = J Fext. 10 Coriolis, centrifugal and gravitational term, B and u ∈ R If the stiffness at each joint is ki with i = 1, ..., n, then the are the actuation matrix and the motor torque vector, τs and joint stiffness matrix is defined as KJ = diag(k1, ..., kn), and Js are the spring joint torque vector and the corresponding τ = KJ ∆q. The joint stiffness matrix KJ and leg stiffness n Jacobian, and Fh,v ∈ R h,v and Jh,v are the holonomic Kleg are hence related by the joint moments, force vector and the corresponding Jacobian respectively. The K ∆q = τ = J T F = J T K δ = J T K J∆q. (6) subscript v is used to indicate different domains which have J ext leg leg different numbers of holonomic constraints. For instance, Then the leg stiffness can be calculated from the joint when the robot has no contact with the ground, nh,v = 2 as stiffness matrix by, there are two holonomic constraints on h . In case when the rod K (q) = (J(q)K−1J(q)T )−1. (7) feet contact the ground, five additional holonomic constraints leg J are introduced on each foot, hence nh,v = 12. This indicates the leg stiffness is a function of the Jacobian and thus a function of the configuration q. The leg damping III.SPRING-MASS MODEL Dleg(q) is derived in the same way by dropping the assump- In this section, we derive our spring-mass model from the tion on small deformation at joints. kinematics of the robot. The compliant components on the Now we apply the calculation of leg stiffness and damping leg are characterized as a prismatic spring on the leg in the on Cassie. Note that the main difference is the closed (a) (b) hip pitch 𝑞 achilles rod 𝑞 hip link 𝑞 knee link

𝑞 heel spring shin spring 𝑞 shin link 𝑞 toe motor 𝑞 main chain tarsus link 𝑞

toe link plantar rod motor joints 𝑞 spring joints 𝑞 pure passive joints 𝑞

kinematic chain inside the leg with pure passive joints and (a) (b) compliant joints. The pure passive joints have no contribution leg spring towards leg stiffness, so we need to derive the forward 𝐿 kinematics from active joints, i.e. the spring joints and motor 𝐿 joints. As there are two chains towards the toe, the velocity 𝐾𝐿 of the toe relative to the hip can be calculated as, stance width

vToe←Hip = J1(q1)q ˙1 = J2(q2)q ˙2, (8) where q1 = [qhp; qknee; qshin; qtarsus; qtoe] and q2 = 𝑠

[qr2 ; qr1 ; qheel; qtoe] (see Fig. 2). Eq. (8) can be rewritten as, q˙ J (q ) −J (q ) 1 = 0. (9) Fig. 3. (a) Illustration of the leg spring from the compliant joints. (b) The 1 1 2 2 q˙ | {z } 2 vertical leg stiffness v.s virtual leg length for different stance width. E

Then we can rearrange the matrix E and group q1, q2 into active joints qA = {qspring, qmotor} and pure passive joints qP . leg as a massless rod between the point mass and the spring. Eq. (8) becomes, The spring stiffness and damping are obtained from the abovementioned polynomial regressions. As the leg length q˙A EA(q) EP (q) = 0. (10) can change under motor joint actuation, we define the second q˙P order derivative of the leg length L¨ as the virtual actuation. −1 Fig. 4(a) illustrates the actuated spring-mass model on the Then the passive joint velocity q˙P = −EP EAq˙A. As there is only one passive joint, the tarsus, on the main ground, where x = Lr is the vertical position of the mass. −1 The dynamics are, kinematic chain, we can find q˙tarsus = −EP EAq˙A, where s qA = {qknee, qshin, qheel, qtoe}. Then the forward kinematics F x¨ = − g, and the leg stiffness can be expressed in terms of qA, m s¨ = L¨ − x,¨ vToe←Hip = JA(qA)q ˙A, (11) −1 T −1 s Kleg(qA) = (JA(qA)KA JA(qA) ) , (12) where s is the spring deformation, and F = K(L)s+D(L)s ˙ is the spring force. K(L) and D(L) are the stiffness and where K = diag(∞, k , k , ∞) as we assume the motor A shin heel damping of the spring, respectively. The kinematic con- joints being rigidly controlled to fixed positions. Assuming straints are s + x = L and s˙ +x ˙ = L˙ . the spring joints have small deflections under normal load, Kleg(qA) can be approximated as Kleg(qknee, qshin = 0, qheel = C. Jumping and Landing Optimization 0, q ). q has trivial contribution in terms of J and K . toe toe A leg This actuated spring-mass model naturally enables the Thus K (q ) ≈ K (q ). leg A leg knee traditional trajectory optimization framework to function for This naturally inspires a definition of virtual leg length planning dynamic tasks such as jumping to a desired height L(q ) to approximate K (q ) by K (L). The real leg knee leg knee leg and landing to a static configuration. One can parameterize a length L (q , q , q ) is defined as the distance between r knee shin heel time-based actuation profile for L¨ and then find the optimal the hip pitch joint and the toe joint, whereas the virtual leg trajectories for the task while minimizing a cost such as the length is the real leg length with zero spring deflections, consumed energy. Here, we apply direct collocation methods L(qknee) = Lr(qknee, 0, 0), (13) [15] to formulate the trajectory optimization problems; the approach is similar to [16]. An even nodal spacing is used as illustrated in Fig. 3(a). Due to Cassie’s specific leg for discretizing the trajectory in time. The defect constraint design, the compliance mainly appears in the direction of is introduced algebraically by an implicit trapezoidal inte- the leg. As we are interested in vertical hopping, the last gration scheme. To enable hopping, we optimize two tasks element in K , denoted by Kz , is taken as the stiffness leg leg on the spring-mass model as follows. of the leg. Fig. 3(b) shows how Kz changes with L at leg Jumping. First, we optimize the spring-mass system to different static stance configurations. We apply a polynomial jump to a desired height x from standing at rest on the regression1to approximate the function Kz (L). The leg des leg ground. When the spring-mass is off the ground, it is in damping is approximated in the same way. ballistic phase. Therefore, we only discretize the trajectories B. The Actuated Spring-Mass Model to the end of ground contact. The task of jumping to the desired height is defined as the equality constraint, As the total mass of Cassie is concentrated above its hip, we model the pelvis as the point mass and its compliance as x˙ 2 x + f = x , (14) the spring attached beneath the mass. As the leg length can f 2g des change with different kinematic configurations, we model the where xf is the last state of x at standing. Initial states also 1 z 2 4 in the form of Kleg(L) = β0 + β1L + β2L + β4L need to satisfy the initial condition of the system, and the (b) (c) IV. CONTROLLER SYNTHESISFOR HOPPING (a) 푥 The hopping of the spring-mass system offers important insights into the intrinsic hopping dynamics of the full-order dynamics from which it was derived. In this section, we 퐿 explain how the trajectory of leg length can be encoded via a control Lyapunov Function based quadratic program (CLF- QP). This yields a nonlinear controller that achieves hopping on the biped Cassie—thus a priori nonlinear optimization on 푠 the full robot [15] is not required. A. The Multi-domain Hybrid System of Hopping The hopping motion inherently is a hybrid dynamical phenomenon with a ground and flight phase. The behavior Fig. 4. (a) The spring-mass model. (b,c) Results of the jumping and landing will, therefore, be described by a hybrid control system of trajectory optimization. the form: HC = (Γ, D, U, S, ∆,FG). (17) leg length L is constrained to be in the range of kinematic Detailed definitions can be found in [17]. We assume that capability of Cassie. Additionally, the ground reaction force, the robot only hops in the sagittal plane and its left and which equals the spring force, must be nonnegative during right toe always have the same contact mode. It is also standing and reaches to zero at the end of ground contact. It desirable to assume that the front and back part of the toe is desirable to minimize the virtually consumed energy for 2 have the same contact modes for simplification purposes. As this task by defining the cost , illustrated in Fig. 5, three domains are defined for hopping, Z T ¨ 2 i.e. D = {DJ , DF , DL}, where {J, F, L} represent Jumping, JJumping = L(t) dt, (15) Flight Landing 0 and respectively. Consequently the directed where T is the duration of the ground contact phase. graph Γ = (V,E) is defined by the vertices V = {J, F, L} Landing. After the ballistic phase, the spring-mass system and the edges E = {J → F,F → L}. will land on the ground. One could optimize the trajectory to In Jumping, the feet are in contact with the ground. The enable continuous jumping. In our case, we want the system number of holonomic constraints is nh,J = 12. The transition to come to a desired resting configuration, which is enforced from Jumping to Flight happens when the ground reaction by equality constraints on the final states: normal forces cross zero. Thus the domain and associated guard can be defined by, xf = xdes, D := {(q, q,˙ u): h (q) = 0,F Foot(q, q,˙ u) > 0}, (18) L˙ =s ˙ = 0, L¨ =s ¨ = 0, J J z f f f f Foot s ¨ SJ→F := {(q, q,˙ u): hJ (q) = 0,Fz (q, q,˙ u) = 0}. (19) Ff − mg = 0, Lf = 0. As there is no impact at the transition from Jumping to Again, the spring force has to be nonnegative. More im- Flight, the reset map is an identity map. portantly, we enforce that the spring force must be larger In Flight, the feet are off the ground, n = 2, and than a desired constant value to increase the feasibility of h,F the transition from Flight to Landing happens when the the trajectory on the full dynamics of the robot, i.e., the feet strike the ground. Therefore, we define the domain and robot does not leave off the ground again. Also, the spring corresponding guard by, deflection s has to be smaller than a certain value so that the Foot Foot spring deflections on the robot are within the ranges of the DF := {(q, q,˙ u): Pz (q) > 0,F (q, q,˙ u) = 0}, (20) hardware limits. The cost function can be the same as that Foot Foot SF →L := {(q, q,˙ u): Pz (q) = 0, vz (q, q˙) < 0}. (21) in jumping to minimize the consumed energy. We add an extra term to minimize the spring oscillation simultaneously, We model the impact between the feet and the ground as yielding: plastic impact, the reset map of which is detailed in [17]. Landing Z T In , the feet are in contact with the ground again, ¨ 2 2 JLanding = L(t) + αs˙(t) dt, (16) nh,L = 12, and the domain DL can be defined as the same 0 as DJ . As we only focus on a single hopping behavior, the where α is a weighting coefficient. system stays in Landing after it is reached. There is no need As a direct result of the low dimensionality and mild to define its guard. nonlinearity of the system, the optimization is solved in a fast The continuous dynamics of the system for each domain and reliable fashion, typically within 2 seconds. Examples of can be obtained from (2) and (3), wherein the exact forms are the optimization results are shown in Fig. 4 (b) and (c). specified from the corresponding holonomic constraints. Let hrod, hFoot denote the holonomic constraints on the closed 2The cost is calculated via trapezoidal integration over time discretization. To simplify the notation, we use R instead of P to avoid introducing kinematic chains and the foot contacts, respectively. Then additional variables for each discretization. hJ = hL = {hrod, hFoot} and hF = {hrod}. which is exponentially stable when > 0. However, such µ does not utilize the natural dynamics of the system and oftentimes may not be realizable on the robotic system if there are stringent physical constraints (e.g. torque bounds) that must be enforced. The above construction motivates constructing rapidly exponentially stabilizing control Lyapunov functions (RES- CLF) [12] from continuous time algebraic Riccati equations (CARE)3or continuous time Lyapunov equations (CTLE)4to :ƵŵƉŝŶŐ &ůŝŐŚƚ >ĂŶĚŝŶŐ stabilize the output dynamics exponentially at a chosen rate ε. Given a solution P = P T > 0 to CTLE or CARE with T Fig. 5. Discrete domains associated with hopping. Q = Q > 0, the Lyapunov function is constructed as, T Vε(η) = η IεPIεη, (29) | {z } B. CLF-QP Pε To control the motion of hopping, we suppose that the 1 where Iε = diag(I1, ε I2,I2). The goal of exponential desired motion is defined by some desired outputs yd for each stabilizing η → 0 is encoded by the condition: domain v ∈ V . For robotic systems, the desired outputs can γ V˙ε(η) ≤ − Vε(η), (30) be outputs with relative degree 1 (RD1) and degree 2 (RD2) ε o1 [18]. Let RD1 outputs be represented by y1 ∈ R and RD2 with some γ > 0, where, outputs represented by y ∈ o2 . We assume that the desired 2 R V˙ (η) = L V (η) + L V (η)µ, (31) motion is of time-based trajectories, thus the outputs can be ε F ε G ε T T defined as follows [17] [18]: LF Vε(η) = η (F Pε + PεF )η, (32) T a d LGVε(η) = 2η PεG. (33) y1(q, q,˙ t) =y ˙1 (q, q˙) − y1 (t), (22) a d Eq. (30) and (31) indicate an inequality constraint on µ y2(q, t) = y2 (q) − y2 (t), (23) to achieve exponential stability. This naturally leads to the where the superscript a denotes the actual and d denotes formulation of quadratic program (QP) to find µ to minimize the desired. The objective of the control is to drive y1 → 0 the quadratic cost µT µ . With (25), the cost and constraint and y2 → 0. Differentiating y1 once and y2 twice yields the of the QP can be transformed back onto the original control affine control system on the output dynamics: input u by noting that: d y˙1 Lf y1(q, q˙) − y˙1 Lgy1(q, q˙) T T T T T = 2 d + u, (24) µ µ = u A Au + 2Lf Au + Lf Lf , (34) y¨2 Lf y2(q, q˙) − y¨2 LgLf y2(q, q˙) | {z } | {z } and the inequality from (30) and (31) becomes: Lf A γ L V (η) + L V (η)L + L V (η)Au ≤ − V (η). (35) where A is the decoupling matrix, L denotes the Lie deriva- F ε G ε f G ε ε ε m tive and u ∈ R is the control input. The dependency on t Now the QP can be formulated in terms of solving for u at is dropped from here to simplify the notation. In case when a current state (q, q˙) as follows, u can be found to satisfy the following equality: ∗ T T T u = argmin u A Au + 2Lf Au m Au = −Lf + µ, (25) u∈R s.t. ACLF(q, q˙)u ≤ bCLF(q, q˙), (CLF) the output dynamics becomes this linear control system:     where, 0 0 0 I1 0 CLF η˙ = 0 0 I2η +  0 0 µ, (26) A (q, q˙) :=LGVε(q, q˙)A(q, q˙), (36) 0 0 0 0 I γ 2 bCLF(q, q˙) := − V (q, q˙) − L V (q, q˙) | {z } | {z } ε ε F ε F G − L V (q, q˙)L (q, q˙). T G ε f (37) where η = [y1, y2, y˙2] , I1 and I2 are identity matrices with o o µ The result of solving the CLF-QP is a feedback optimal dimension 1 and 2 respectively, and is the auxiliary con- a a µ control law to drive the outputs [y ˙1 (q, q˙); y2 (q)] to follow trol input [18]. One can choose to exponentially stabilize d d the linear system. For example, choosing [11] the desired time based trajectories [y1 (t); y2 (t)] with exponentially convergence. This formulation also applies when −y1 there are only relative degree 2 outputs to be tracked [4]. µ = 2 (27) −2y˙2 − y2 For applications of using CLF-QP on robotic systems, torque results the linear output dynamics: bounds and additional nontrivial constraints can be included       in the QP [11]. y˙1 −I1 0 0 y1 y˙2 =  0 0 I2  y2 , (28) 3F T P + PF − P GGT P + Q = 0. 2 4 T y¨2 0 − I2 −2I2 y˙2 F P + PF + Q = 0. C. Output Definition for Hopping One can interpret the outputs as these on the 6 Dof of To apply the CLF-QP formulation on the multi-domain the pelvis (the mass in the spring-mass model). The specific hybrid control system associated with hopping, we define leg length on left and right leg constraints the height and the outputs with reference output trajectories for each domain the roll of the pelvis. The specific horizontal COM positions separately so that the hopping behavior can be enabled. It is constraint the forward and lateral positions of the pelvis. The important to define two outputs: leg length and centroidal pitch momentum output constraints the pitch of the pelvis. momentum. 2) Flight: The robot is off the ground and the 6 Dof of Leg Length. The virtual leg length trajectories L(t) from the floating base is in underactuation. Ten outputs are needed the jumping and landing of the spring-mass model (see and thus we specify all the motor positions as the outputs: Section III) are mainly used as the desired virtual leg length F des y2 (q) = qm − qm . (43) Ldes(t) on the full robot. The springs on the full robot are expected to behave similarly to the spring on the spring-mass As the centroidal momentum is controlled to be 0 or small model when the virtual leg length of the full robot follows in Jumping, the robot is not expected to have large whole- Ldes(t). In other words, the underactuation of the springs is body rotation in Flight. We simply let the motor positions at expected to behave accordingly so that the robot can jump the end of Jumping be the desired motor positions for Flight. off the ground and land on the ground. The desired toe motor positions are adjusted to keep the toes Centroidal Momentum. In Flight, the only external parallel to the ground. force is the gravitational force, thus the robot obeys the 3) Landing: The outputs are defined similarly to the out- conservation of angular momentum about its COM, i.e., puts in Jumping. The left and right leg length are selected as the centroidal angular momentum [13]. To keep the control two outputs to embed the spring-mass landing dynamics onto in Flight simple, it is desirable to have small centroidal the robot. It is not necessary to keep the pitch momentum angular momentum when the robot jumps off the ground. zero for all time during landing, so we remove the RD1 The centrodial momentum of a multi-link robotic system can output HPitch. Instead, the pelvis pitch is selected. Thus the be expressed as [13]: outputs are defined as:    des  LL(q) L (t) HG(q, q˙) = AG(q)q, ˙ (38) des  LR(q)   L (t)  6    des  where HG ∈ R is the centroidal momentum vector and L xcom(q) xcom(t) 6×n y2 (q, t) =   −   . (44) AG(q) ∈ R is the centroidal momentum matrix. As we  ycom(q)   0     des  are mainly concerned with vertical jumping, only the pitch φpitch(q) φpitch(t) angular momentum is selected: φyaw(q) 0 des HPitch(q, q˙) = APitch(q)q. ˙ (39) The desired pelvis pitch trajectory φpitch(t) and desired COM des forward trajectory xcom(t) are designed smoothly from the Note that the pitch centroidal momentum is a relative degree post-impact positions to 0. 1 output. Differentiating it once yields, D. Main Control Law H˙ = A˙ (q, q˙)q ˙ + A (q)¨q. (40) Pitch Pitch Pitch We apply the CLF-QP based feedback control to the Now we can specify the outputs for each domain. hybrid control system associated with hopping. With an eye 1) Jumping: When two feet are on the ground, 10 addi- towards the constrained optimization formulation [11], we tional holonomic constraints are added to the system. As the rewrite (2) viewing the torque and force as inputs: robot has 16 degrees of freedom (DoF) except for the tarsus T T u joints and spring joints, we need to apply 6 outputs to guide M(q)¨q + H(q, q˙) − Js (q)τs(q) = BJh,v(q) , Fh,v the motion. As noted above, we select the centroidal pitch | {z } | {z } Y (q,q˙) B¯v (q) | {z } RD u¯v momentum as the 1 output. Left and right leg length are (45) used as two RD2 outputs to embed the spring-mass jumping m+n where u¯v ∈ R h,v is the augmented control input. The dynamics onto the full robot. It is desired to keep the COM affine control system can thus be defined as: position projected onto the center of the support polygon and to avoid yaw motion of the pelvis, which requires x˙ = f(x) + gv(x)¯uv, (46) x → 0, y → 0 and φ → 0. Therefore, we define com com yaw where, the outputs for Jumping as, q˙ 0 J y (q, q˙) = H (q, q˙) − 0, (41) f(x) = −1 , gv(x) = −1 . 1 Pitch −M (q)Y (q, q˙) M (q)B¯v(q)    des  LL(q) L (t) (47) des  LR(q)  L (t) The reason of including Fh,v as the control input is to J     y (q, t) = xcom(q) −  0  . (42) easily incorporate the holonomic constraints as equality con- 2     ycom(q)  0  straints and ground reaction force constraints as inequality φyaw(q) 0 constraints in the quadratic program. Holonomic Constraint. Eq. (3) can be rewritten as a importantly, the spring joint forces on the robot show the function of u¯v, same profile as the GRF, which equals to the spring force in the spring-mass. This ratifies our hypothesis that the spring A (q, q˙)¯u = b (q, q˙), h,v v h,v (48) of the robot behaves similar to the spring in the spring-mass. −1 ¯ Ah,v(q, q˙) = Jh,v(q)M (q)Bv(q), (49) The QP is not yet implemented on the hardware but this is −1 bh,v(q, q˙) = Jh,v(q)M (q)Y (q, q˙) − J˙h,v(q, q˙)q. ˙ (50) a subject of future work. To validate our method experimentally, we extract the motor joint positions from the simulation Ground Reaction Force. In Jumping and Landing, the feet and apply position tracking by a PD+feedforward controller contact the ground. The ground reaction forces have to satisfy on the hardware at 2kHz. The feedforward term is the motor the physics constraints such as nonnegative normal forces torque from the simulation. Under this setting, the QP is Foot and non-slipping, formulated by RF ≤ 0, where R is a viewed to perform joint trajectory generation. Fig. 7 (a) (b) constant matrix. The constraint on u¯v can be written as and (c) show the experiment results of a hopping motion with GRF GRF ground clearance of ∼ 7 inches. Due to the model inaccuracy A u¯v ≤ b , (51) v v of the physical robot, the spring deflection difference is GRF Foot GRF GRF where AJ/Lu¯J/L = RF , AF = 0, bv = 0. ineligible and the robot jumps forward consequently. Future Torque Constraints. The motor torque must be within the work will try to address this issue by utilizing feedback. feasible limits of the robot hardware: ulb ≤ u ≤ uub. Experiment videos can be found at [1]. Optimization-based controller: The resulting QP controller for each domain v ∈ V is given as follows:

∗ T T T 2 (a) Outputs (Jumping) (b) Outputs (Landing) u¯v = argmin u¯v Av Avu¯v + 2(Lf,v) Avu¯v + pδ , m+nh,v u¯v ∈R ,δ∈R CLF CLF s.t. Av (q, q˙)¯uv ≤ bv (q, q˙) + δ, (CLF) GRF GRF Av u¯v ≤ bv , (GRF)

ulb ≤ u ≤ uub, (Torque)

Ah,v(q, q˙)¯uv = bh,v(q, q˙).(Holonomic) To increase the feasibility of the QP, we follow the method in [12] to relax the CLF constraints by introducing δ and penalizing the relaxation by adding pδ2 in the cost, with some large positive constant p. desired QP simulation V. SIMULATION AND EXPERIMENTAL RESULTS (c) (d) To validate our proposed control methodology, we first implement this framework in simulation, and then apply the simulation results on the robot in experiment. The simulation starts from an initial static configuration q0 of the robot. Given a desired apex height, we first optimize the jumping on the spring-mass model from the initial condition corresponding to q0. Then the leg length trajectory is used as the desired output in the CLF-QP. The QP is formulated and (e) (f) solved every at 0.5ms using qpOASES [19]. The dynamics is numerically integrated using MATLAB’s ode113 function. At the transition from Flight to Landing, the post-impact state decide the initial condition for landing optimization on the spring-mass. The leg length trajectory of the landing planning is then used in the CLF-QP for controlling the landing of the robot. The desired and actual output trajectories are shown in Fig. 6 (a) and (b). The simulation indicates fast convergence of output dynamics. Fig. 6 (c) shows the comparison on the system energies. Since the robot is not a point mass exactly, the kinetic energy and potential energy differ slightly from these of the spring-mass. Fig. 6 (d) and (f) illustrate Fig. 6. Simulation results. (a,b) Outputs of jumping and landing. (c) Spring energy SE , kinetic energy KE and gravitational potential energy PE during the spring torques at joints and the ground reaction force hopping. (d) The spring torques of Cassie during hopping. (e) Centroidal (GRF) respectively. Note that the GRF of the robot model momentum of Cassie during hopping. (f) Comparison on the ground reaction align closely with the GRF of the spring-mass model. More normal force of the hopping of Cassie vs that of the spring-mass model. (a) Motor joint position (b) Shin and Tarsus Position 0.1 2.5

0 2

-0.1 1.5

-0.2 1 012 012

Simulation Experiment (c) Jumping Flight Landing

𝑡 0.609𝑠 𝑡 1.032𝑠 Every ~180ms

Fig. 7. Experiment results. (a) Comparison of motor joint positions on the left leg in simulation vs these in experiment. (b) Comparison of shin and tarsus joints on the left leg in simulation vs these in experiment. (e) Snapshot of the hopping of Cassie.

VI.CONCLUSIONAND FUTURE WORK [7] S. Kajita, F. Kanehiro, K. Kaneko, K. Fujiwara, K. Harada, K. Yokoi, and H. Hirukawa, “Biped walking pattern generation by using preview This paper presented the planning and controller synthesis control of zero-moment point,” in Robotics and Automation, IEEE International Conference on, vol. 2, 2003, pp. 1620–1626. to achieve hopping of the bipedal robot Cassie via reduced- [8] J. Pratt, T. Koolen, T. De Boer, J. Rebula, S. Cotton, J. Carff, order model embedding. The spring-mass model is faithfully M. Johnson, and P. Neuhaus, “Capturability-based analysis and control established from the mechanical (kinematic and compliant) of legged locomotion, part 2: Application to m2v2, a lower-body humanoid,” The International Journal of Robotics Research, vol. 31, properties of the robot. The planned trajectories of leg length no. 10, pp. 1117–1133, 2012. and additional outputs are rigorously defined in the context [9] C. Hubicki, J. Grimes, M. Jones, D. Renjewski, A. Sprowitz,¨ A. Abate, of a CLF-QP formulation to facilitate the feedback control and J. Hurst, “Atrias: Design and validation of a tether-free 3d-capable spring-mass bipedal robot,” The International Journal of Robotics of hopping. The realized hopping on Cassie validates the Research, vol. 35, no. 12, pp. 1497–1521, 2016. proposed approach. [10] B. Siciliano and O. Khatib, Springer handbook of robotics. Springer The future work will be devoted to extending the approach Science & Business Media, 2008. [11] A. D. Ames and M. Powell, “Towards the unification of locomotion to create different dynamic behaviors on bipedal robots, and manipulation through control lyapunov functions and quadratic likely including reduced-order modeling in two or three programs,” in Control of Cyber-Physical Systems. Springer, 2013, dimensions for walking and running behaviors. From a pp. 219–240. [12] A. D. Ames, K. Galloway, K. Sreenath, and J. W. Grizzle, “Rapidly theoretical viewpoint, we want to establish stability condi- exponentially stabilizing control lyapunov functions and hybrid zero tions on how the reduced-order model quantitatively predicts dynamics,” IEEE Transactions on Automatic Control, vol. 59, no. 4, dynamics behaviors of the full-order system. The hope is that pp. 876–891, 2014. [13] D. E. Orin and A. Goswami, “Centroidal momentum matrix of a this will inform rigorous controller synthesis for complex humanoid robot: Structure and properties,” in Intelligent Robots and robots via simple models. Systems, 2008. IROS 2008. IEEE/RSJ International Conference on. IEEE, 2008, pp. 653–659. [14] http://www.agilityrobotics.com. REFERENCES [15] A. Hereid, E. A. Cousineau, C. M. Hubicki, and A. D. Ames, “3d dynamic walking with underactuated humanoid robots: A direct col- [1] https://youtu.be/5s19IWqN4-c. location framework for optimizing hybrid zero dynamics,” in Robotics [2] J. Rummel, Y. Blum, H. M. Maus, C. Rode, and A. Seyfarth, “Stable and Automation (ICRA), 2016 IEEE International Conference on. and robust walking with compliant legs,” in Robotics and automation IEEE, 2016, pp. 1447–1454. (ICRA), 2010 IEEE international conference on, pp. 5250–5255. [16] C. M. Hubicki, J. J. Aguilar, D. I. Goldman, and A. D. Ames, [3] X. Xiong, A. D. Ames, and D. I. Goldman, “A stability region criterion “Tractable terrain-aware motion planning on granular media: an impul- for flat-footed bipedal walking on deformable granular terrain,” in sive jumping study,” in Intelligent Robots and Systems (IROS), 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems, IEEE/RSJ International Conference on. IEEE, 2016, pp. 3887–3892. IROS 2017, Vancouver, BC, Canada, pp. 4552–4559. [17] A. D. Ames, “Human-inspired control of bipedal walking robots,” [4] A. Hereid, M. J. Powell, and A. D. Ames, “Embedding of slip IEEE Transactions on Automatic Control, vol. 59, no. 5, pp. 1115– dynamics on underactuated bipedal robots through multi-objective 1130, 2014. quadratic program based control,” in Decision and Control (CDC), [18] H. K. Khalil, “Noninear systems,” Prentice-Hall, New Jersey, vol. 2, 2014 IEEE 53rd Annual Conference on. IEEE, pp. 2950–2957. no. 5, pp. 5–1, 1996. [5] M. H. Raibert, Legged robots that balance. MIT press, 1986. [19] H. Ferreau, C. Kirches, A. Potschka, H. Bock, and M. Diehl, [6] I. Poulakakis and J. Grizzle, “The spring loaded inverted pendulum “qpOASES: A parametric active-set algorithm for quadratic program- as the hybrid zero dynamics of an asymmetric hopper,” IEEE Trans- ming,” Mathematical Programming Computation, vol. 6, no. 4, pp. actions on Automatic Control, vol. 54, no. 8, pp. 1779–1793, 2009. 327–363, 2014.