Real-Time Online Model Predictive Control!

Real-time online Model Predictive Control!

Colin Jones, Melanie Zeilinger, Stefan Richter!

Automatic Control Laboratory, EPFL! Real-time synthesis : Complexity as a speciﬁcation! MPC problem! Computational" Embedded" Real" method! Processor! time!!

N 1 − J (x0) = min VN (xN )+ l(xi,ui) ui i=0 s.t. xi+1 = f(xi,ui) (x ,u ) i i ∈ X × U x N ∈ XN

• Hardware platform bounds computation time and storage! • Current real-time explicit methods are limited to small problem dimensions! ! Online MPC can be applied to all problem dimensions ! ! This talk: Real-time online MPC for high-speed large-scale systems! ! Fast online optimization! ! Satisfaction of real-time constraint! !

!"#$%!#&'('$)#*$+( +!,$*!#$*- Fast online optimization! Many methods available:! ! CVX! CVXMOD! CVXGEN! ! Matlab Software for Convex optimization Code Generation for Disciplined Convex software in Python! Convex Optimization ! Programming! http://cvxmod.net/! http://cvxgen.net/! qpOases! http://cvxr.com/cvx/! Online Active Set Strategy! ! ! ! http://www.kuleuven.be/ ! optec/software/qpOASES! ! OOQP! Object-oriented software ! for quadratic programming! QPSchur! http://pages.cs.wisc.edu/ A dual, active-set, Schur-complement ! ~swright/ooqp/! method for large-scale and structured convex quadratic programming! …many more! ! [Bartlett et al., ‘06]! ! ! Online optimization can be applied to control high-speed systems!

No guarantees on system theoretic properties when applied to MPC in a real-time setting. !

!"#$%!#&'('$)#*$+( +!,$*!#$*- Example: " Effect of limited computation time!

Unstable example! Closed loop trajectory: 1.21 1 x + = + Optimal control law ! 01 0.5 2 x 5, 5 x 1 | 1| ≤ − ≤ 2 ≤ u 1,N =5,Q= I,R =1 1.5 | | ≤

-4 1 x 10

2 x 0.5 3

0 2

-0.5 1 -4 -3 -2 -1 0 0 5 10 15 20 25

Computationtime[s] x 1 Time Step Closed loop trajectory:! Optimization stopped after 4 iterations! = max computation time of 21ms !

Limited computation time ! No stability properties!

!"#$%!#&'('$)#*$+( +!,$*!#$*- Real-time online MPC: Goals! Real-time online MPC:! Guarantee that! • within the real-time constraint ! • a feasible solution! NOTE: Optimality not required! • satisfying stability and performance criteria! • for any admissible initial state ! is found. ! ! We present two methods for linear systems:! Setting! ! Linear state and input ‘Simple’ input constraints! constraints! (e.g. box constraints)! Time scale! Milliseconds! Microseconds! Idea! Provide guarantees for any time Compute a priori bounds on the constraint ! required online computation time ! Approach! Robust MPC with stability Fast gradient method ! constraints! ! [M.N. Zeilinger et al., CDC 2009]! [S. Richter et al., CDC 2009]! !"#$%!#&'('$)#*$+( +!,$*!#$*- Real-time online MPC: Goals! Real-time online MPC:! Guarantee that! • within the real-time constraint ! • a feasible solution! NOTE: Optimality not required! • satisfying stability and performance criteria! • for any admissible initial state ! is found. ! ! We present two methods for linear systems:! Setting! ! Linear state and input ‘Simple’ input constraints! constraints! (e.g. box constraints)! Time scale! Milliseconds! Microseconds! Idea! Provide guarantees for any time Compute a priori bounds on the constraint ! required online computation time ! Approach! Robust MPC with stability Fast gradient method ! constraints! ! [M.N. Zeilinger et al., CDC 2009]! [S. Richter et al., CDC 2009]! !"#$%!#&'('$)#*$+( +!,$*!#$*- Example: " Stability under proposed real-time method!

Unstable example! Closed loop trajectory: 1.21 1 x + = + Optimal control law ! 01 0.5 Proposed real-time 2 MPC method" x 5, 5 x 1 | 1| ≤ − ≤ 2 ≤ stopped after 4 1.5 u 1,N =5,Q= I,R =1 online iterations! | | ≤

-4 1 x 10

2 x 0.5 3 time[s]

0 2

-0.5 1 -4 -3 -2 -1 0 0 5 10 15 20 25

Computation x 1 Time Step Closed loop trajectory:! Optimization stopped after 4 iterations! = max computation time of 21ms !

Real-time robust MPC : Nearly optimal and satisﬁes time constraints!

!"#$%!#&'('$)#*$+( +!,$*!#$*- Optimal MPC scheme!

Optimal ! solution!

Warm-start! Online optimization!

Common warm-start: Shifted sequence! u (x)=[u0, ... , uN-1]

ushift (x)=[u1, ... , uN-1, KxN]

!"#$%!#&'('$)#*$+( +!,$*!#$*- Optimal MPC scheme!

Optimal ! solution!

Warm-start! Online optimization!

Optimal MPC:! • Recursively feasible! • Stabilizing! • Unknown computation time…!

!"#$%!#&'('$)#*$+( +!,$*!#$*- Real-time MPC scheme – General idea!

Suboptimal ! solution!

! Properties?!

Warm-start! Online optimization! + Early termination! General approach for real-time MPC:! • Use of warm-start method ! • Exploitation of structure inherent in MPC problems ! • Early termination of the online optimization ! ![Wang & Boyd 2008; Ferreau et al., 2008; Schoﬁeld, 2008; Cannon et al., 2007; .. Many more]! !"#$%!#&'('$)#*$+( +!,$*!#$*- Real-time MPC scheme - Current methods!

Suboptimal ! solution!

! Properties?!

Warm-start! Online optimization! + Early termination!

Suboptimal solution during online optimization steps! - can be infeasible ! - can destabilize the system ! - can cause steady-state offset!

!"#$%!#&'('$)#*$+( +!,$*!#$*- Problem deﬁnition! MPC problem:! N 1 − T T T J∗(x) = min VN (x,u) xi Qxi + ui Rui + xN PxN x,u i=0 Z[ xi+1 = Axi + Bui

Cxi + Dui b ≤ Assumption: is a polytope, xN f Xf !! ∈ X but approach can be equivalently ! x0 = x applied to ellipsoidal constraints. ! ! ! Parametric Quadratic Program ! ! ! !!

!"#$%!#&'('$)#*$+( +!,$*!#$*- Two QP formulations!

T T T T T T T T Vectorized notation: x =[! x0 ,x1!!,...,xN ] ,! u =[u0 ,u1 ,...,uN 1] −

Formulation 1:! • The predicted states can be expressed as ! x = x + u A B • The MPC problem can be written using only the optimization variable u :! T min u Hd u u Matrices are dense! Z[ G u f + E x d ≤ d d

Formulation 2:! • Optimize over sequence of states and inputs z! =[xT , u! T ] T :! • Introduce equality constraints relating the states and inputs: ! min z T Hz z Z[ Gz f Matrices are sparse ! ≤ Fz = Ex

!"#$%!#&'('$)#*$+( +!,$*!#$*- Problem deﬁnition! MPC problem:! N 1 − T T T J∗(x) = min VN (x,u) xi Qxi + ui Rui + xN PxN x,u i=0 Z[ xi+1 = Axi + Bui

Cxi + Dui b ≤ Assumption: is a polytope, xN f Xf !! ∈ X but approach can be equivalently ! x0 = x applied to ellipsoidal constraints. ! ! ! Parametric Quadratic Program: ! ! ! ! ! T J∗(x) = min z Hz z Z[ Gz d ≤ Fz = Ex

!"#$%!#&'('$)#*$+( +!,$*!#$*- Current real-time MPC methods " - Loss of feasibility! In practice: System will be subject to disturbances! ! Consider uncertain system: x+=Ax+Bu+w where! is a bounded disturbance. ! w !W ! Problem: Disturbances cause loss of ! ! ! feasibility of the warm-start ! ! ! solution! ! ! ! Recovery of feasibility not ! ! ! guaranteed in real-time !! !

!"#$%!#&'('$)#*$+( +!,$*!#$*- Current real-time MPC methods " - Loss of stability! Requirement for stability: Lyapunov function! ! Use of MPC cost as Lyapunov function!

! MPC cost has to decrease at every time step:!VN (x,u(x))

!"#$%!#&'('$)#*$+( +!,$*!#$*- Background: Primal barrier interior-point method! Optimization problem: ! min z!!!T Hz ! z Note: here QP, but general Z[ Gz d nonlinear program possible! ≤ Fz = Ex

!"#$%!#&'('$)#*$+( +!,$*!#$*- Background: Primal barrier interior-point method! Optimization problem:! min z T Hz z ! Z[ Gz d ! ≤ Fz = Ex ! m T Barrier method: !! min z Hz µ log( Gi z + di ) barrier term! z − − i=1 with barrier ! Z[ Fz = Ex parameter ! ! ">0 ! 10 – Equality constrained problem! – Approximation improves as ! 5 ↘! " # 0 " ! 0 ! Gi z = di − 5 − 3 − 2 − 1 0 1 G z d i − i [Boyd & Vandenberghe, 2004]!

!"#$%!#&'('$)#*$+( +!,$*!#$*- Background: Primal barrier interior-point method! Optimization problem:! min z T Hz z ! Z[ Gz d ! ≤ Fz = Ex ! m T Barrier method: !! min z Hz µ log( Gi z + di ) barrier term! z − − i=1 with barrier ! Z[ Fz = Ex parameter ! ! ">0 ! – Solve augmented problem for decreasing values of ! " ! (central path)! z*(") – Convergence to the optimal solution of the! original optimization problem for ! " # 0 ! z*(10) ! z*

[Boyd & Vandenberghe, 2004]! !"#$%!#&'('$)#*$+( +!,$*!#$*- Current real-time MPC methods " - Loss of stability! Requirement for stability: Lyapunov function! ! Use of MPC cost as Lyapunov function!

! MPC cost has to decrease at every time step:!VN (x,u)

Real-time online MPC:! Guarantee that! • within the real-time constraint !! • a feasible solution! !!!! ! • satisfying stability criteria ! !!! • for any admissible initial state !! is found. ! !

!"#$%!#&'('$)#*$+( +!,$*!#$*- Proposed real-time MPC method!

Real-time online MPC:! Guarantee that! • within the real-time constraint ! !! Early termination! • a feasible solution! !!!!! Robust MPC ! • satisfying stability criteria ! !!! • for any admissible initial state !! is found. ! ! • Robust MPC method provides feasibility of the warm-start solution by considering all possible disturbance sequences! ! • Use of primal feasible optimization method provides feasibility of the suboptimal solution obtained during online optimization! !

!"#$%!#&'('$)#*$+( +!,$*!#$*- Proposed real-time MPC method!

Real-time online MPC:! Guarantee that! • within the real-time constraint ! !! Early termination! • a feasible solution! !!!!! Robust MPC ! • satisfying stability criteria ! !!! Lyapunov constraint! • for any admissible initial state !! is found. ! ! Introduce ‘Lyapunov constraint’:! Enforces decrease in suboptimal MPC cost at each iteration! ! T T VN (x,u)

! !"#$%!#&'('$)#*$+( +!,$*!#$*- ! Real-time robust MPC - Fast implementation" Interior point optimization! • Standard Newton step computation:! ! Speed of optimization! ! Time to solve$ linear system! ! • Tracking formulation and Lyapunov constraint ! ! Modiﬁed Newton step matrix structure!

!"#$%!#&'('$)#*$+( +!,$*!#$*- Real-time robust MPC - Fast implementation" Interior point optimization! • Standard Newton step computation:! ! Speed of optimization! ! ! Time to solve$ linear system! ! • Tracking formulation and Lyapunov constraint ! ! Modiﬁed Newton step matrix structure! • Matrices can be transformed into arrow structure! ! Solved as efﬁciently as standard MPC problems! • Custom solver in C++ was developed ! ! Extending [Rao et al., 1998, Hansson, 2000 and Wang et al., 2008]!

!"#$%!#&'('$)#*$+( +!,$*!#$*- Numerical examples!

Oscillating masses example! • Problem: 12 states, 3 inputs! • Fast MPC with guarantees: horizon N=10! !! Computation of 5 Newton steps in 2 msec! Comparison: CPLEX 26 msec, SEDUMI 252 msec! !Closed loop performance loss in % for varying iteration numbers! ! ! Optimal ! ! ~44 iterations! ! ! 2.5kHz sampling rate with stability guarantee! ! Random example! • Problem: 30 states, 8 inputs, horizon N=10! ! QCQP with 410 optimization variables and 1002 constraints! ! Computation of 5 Newton steps in 10 msec! ! !"#$%!#&'('$)#*$+( ! +!,$*!#$*- ! ! ! !

! ! ! Real-time online MPC: Goals! Real-time online MPC:! Guarantee that! • within the real-time constraint ! • a feasible solution! • satisfying stability and performance criteria! • for any admissible initial state ! is found. ! ! We present two methods for linear systems:! Target! ! Linear state and input ‘Simple’ input constraints! constraints! (e.g. box constraints)! Time scale! Milliseconds! Microseconds! Idea! Provide guarantees for any time Compute a priori bounds on the constraint ! required online computation time ! Approach! Robust MPC with stability Fast gradient method ! constraints! ! [M.N. Zeilinger et al., CDC 2009]! [S. Richter et al., CDC 2009]! !"#$%!#&'('$)#*$+( +!,$*!#$*- Structured Optimization: Gradient Method for input ;#61?,• Fast gradient method! ' – Very simple! – Easy to parallelize! !"#$%#&'()$*#%'+,#%-,.$/'0$1/2'3$%'+/41,'5$/6,%-./#2'7859' – Fast for large number of states ! (using dense problem formulation)! ! ! Work per iteration! ! • 1 matrix-vector product ! • 2 vector sums! ! • 1 projection ! !

Key result: Can compute a priori bound on required number of iterations imin!

[Y. Nesterov, 1983]! [S. Richter et al., CDC 2009]!

7-@<'./.,.-?'%#6.21-?' !"#$%!#&'('$)#*$+( +!,$*!#$*- Æ'0.?#A#?'8%$B%-&'

::' Fast Gradient Method : Time bound to -optimality! % • Solution with approximation error in i steps:! % min ! ln δ iTPU ≥ ln 1 ⌧ √ 1 ! κ " ! " • condition number! & • measure of initial residual! ' Cold start:! Warm start:!

2 δ LR /2 δ 2 max JN (u^Z; x) JN∗ (x) ≤ x N − ≤ ∈X • u • u : Warm start sequence! • Rws : =radius 0 of feasible set! • Worstws distance measured in • Easy to compute! terms of initial cost! • Hard to compute!

[S. Richter et al., CDC 2009]!

!"#$%!#&'('$)#*$+( +!,$*!#$*- Fast Gradient Method : Time bound to -optimality! % • Solution with approximation error in i steps:! % min ! ln δ NOTE: Extension to state and iTPU ≥ ln 1 ⌧ √ 1 input constraints possible using ! κ " Lagrangian relaxation ! ! " [S. Richter et al., CDC 2011]! • condition number! & • measure of initial residual! ' Cold start:! Warm start:!

[S. Richter et al., CDC 2009]!

!"#$%!#&'('$)#*$+( +!,$*!#$*- Example: Control of an AC-DC Power Converter !

2-level 3-phase inverter:! ! controlled ! inputs! outputs!

AC grid! DC source!

CL ﬁlter! Switched inverter (1…50 kHz)!

Control objectives:! • Track currents if g,1 , if g,2 , if g,3 ! • Actively dampen CL ﬁlter dynamics! ! Model:! Marginally stable system in d-q coordinates: ! !6 states / 2 inputs / 2 disturbances / 2 controlled outputs! ! [S. Richter et al., ACC 2010]! !"#$%!#&'('$)#*$+( +!,$*!#$*- Example: Control of an AC-DC Power Converter !

MPC Tracking Problem: ! !! Rotating/Scaling Feasible Set:! N 1 − !!!2 2 2 J∗ (q) = min δx + + δx + δu ! N N P i Q i R i=0 ! Z[ δxi = xi xss − …! δu = u u i i − ss xi+1 = Axi + Bui + Bw w

ui U(v,φ iωgTs ) ∈ − Implementation environment: ! • 16-bit native ﬁxed-point DSP BF-533 from Analog Devices (#10$) ! • C code (integer arithmetic) + standard C-compiler! ! Main results:! Bound: ! !!125 ! Solution Time: !< 50" s "s Memory:! !!< 1kB! Relative accuracy: < 1e-3! [S. Richter et al., ACC 2010]!

!"#$%!#&'('$)#*$+( +!,$*!#$*- Example: Ball on Plate System! • Movable plate (0.66m x 0.66m)! • Can be revolved around two axis [+17o ; -17o] by two DC motors! • Angle is measured by potentiometers! • Linearized dynamics: 4 states, 2 inputs! • Position of the ball is measured by a camera! !