Hamiltonian Decomposition for Online Implementation of Model Predictive Control

A thesis submitted to the University of Manchester for the degree of Doctor of Philosophy in the Faculty of Science and Engineering

2018

Eduardo N. Poupard

Supervised by Prof. William P. Heath

School of Electrical and Electronic Engineering Contents

1 Introduction 17 1.1 Background and motivation ...... 17 1.2 Statement of originality and thesis outline ...... 18 1.2.1 Part I ...... 19 1.2.2 Part II ...... 19 1.2.3 Part III ...... 21 1.2.4 Additional work ...... 22 1.2.5 Mathematical tools ...... 22

I Unconstrained Optimal Control Problem 23

2 Free-ﬁnal State Optimal Control Problem 24 2.1 Continuous-time analysis ...... 24 2.1.1 Problem formulation ...... 24 2.1.2 Two-point boundary-value problem ...... 25 2.2 Discrete-time analysis ...... 26 2.2.1 Problem formulation ...... 27 2.2.2 Two-point boundary-value problem ...... 28

3 Linear Quadratic Regulator 29 3.1 Finite-horizon formulation ...... 29 3.1.1 Linear system of equations ...... 31 3.2 Solution via Riccati equation ...... 32 3.2.1 Derivation via linear system of equations ...... 33 3.3 Solution via eigenvalue decomposition ...... 34 3.3.1 Receding horizon control implementation ...... 37 3.3.2 The case of A being singular ...... 38

4 LQ Tracking Problem 41 4.1 Finite-horizon formulation ...... 41 4.1.1 Linear system of equations ...... 42 4.2 Solution via Riccati equation ...... 43 4.2.1 Derivation via linear system of equations ...... 44 4.3 Solution via eigenvalue decomposition ...... 46 4.3.1 Receding horizon control implementation ...... 49

2 II Constrained Optimal Control Problem 52

5 Model Predictive Control Framework 53 5.1 Fundamentals ...... 54 5.1.1 Sparse formulation ...... 55 5.1.2 Necessary conditions ...... 57 5.2 Interior point method ...... 57 5.2.1 Primal-dual interior point algorithm ...... 58 5.2.2 Mehrotra’s predictor-corrector algorithm ...... 59

6 Newton system factorization 62 6.1 General system decomposition ...... 62 6.1.1 Hamiltonian decomposition ...... 64 6.2 Riccati recursion method ...... 66 6.2.1 Algorithm ...... 68 6.2.2 Numerical example ...... 69 6.3 Eigenvalue factorization method ...... 70 6.3.1 Algorithm ...... 70 6.4 Splitting method implementation ...... 72 6.4.1 Algorithms ...... 74 6.4.2 Numerical experiment ...... 75

7 Fast Hamiltonian/GMRES solver 79 7.1 Incomplete LU factorization ...... 80 7.1.1 Tailor-made preconditioners ...... 80 7.2 Companion-like matrix ...... 83 7.2.1 Algorithm ...... 86 7.2.2 Recursion sequence structure on the companion-like matrix . . . . . 87

7.2.3 Nonsingular matrix ZN guarantee ...... 88 7.2.4 Flop count ...... 89 7.2.5 Direct method comparison ...... 90 7.3 GMRES method application ...... 91 7.3.1 Convergence ...... 92 7.3.2 Algorithm ...... 94 7.3.3 Flop count ...... 94 7.4 Numerical experiments ...... 96 7.4.1 Experiment 1: Feasibility ...... 96 7.4.2 Experiment 2: Convergence veriﬁcation under rounding errors . . . . 97 7.4.3 Experiment 3: Convergence subject to diﬀerent horizons ...... 99 7.4.4 Experiment 4: Comparison ...... 100 7.4.5 Experiment 5: Numerical issue ...... 102 7.5 The case of A being singular ...... 105 7.5.1 Formulation ...... 105 7.5.2 Numerical example ...... 106 7.5.3 Eigenvalue trajectories ...... 108

3 7.5.4 Convergence experiment ...... 109

8 Conclusions of part I and II 112 8.1 Comments on novel methods (part I) ...... 112 8.2 Further work on novel methods (part II) ...... 113 8.2.1 On the splitting methods ...... 113 8.2.2 On the Hamiltonian recursion solver ...... 113 8.2.3 On the Hamiltonian/GMRES solver ...... 114

III Embedded Model Predictive Control 115

9 Festo AG & Co.KG Collaboration 116 9.1 The Company ...... 116 9.2 Why Model Predictive Control (MPC)? ...... 117 9.2.1 MPC oﬀered by major control-systems vendors ...... 118 9.3 Programmable Logic Controller platform (PLC) ...... 120 9.3.1 Embedded MPC on PLCs ...... 120 9.3.2 Comparison between diﬀerent PLC platforms ...... 121 9.3.3 Some commercial embedded APC software for PLCs ...... 122

10 Optimization Solver Engine 123 10.1 Tracking and Disturbances Framework on MPC ...... 123 10.1.1 Coordinate transformation and steady-state target problem . . . . . 124 10.1.2 Dynamic regulation problem ...... 125 10.1.3 Constant disturbances and zero oﬀset ...... 125 10.2 Case Study: Flotation-Cell System ...... 127 10.2.1 Single-tank model ...... 128 10.3 Numerical experiment: Flotation-cell system ...... 130 10.3.1 Control design objective and settings ...... 131 10.3.2 Simulation ...... 133 10.4 Optimization Code Generation ...... 136

11 Hardware-in-the-loop Test (HIL) 138 11.1 PLCs Test Rig ...... 138 11.1.1 HIL test diagram ...... 139 11.2 Industrial experiment 1: Flotation-cell system ...... 140 11.2.1 Feasibility of the Hamiltonian recursion method ...... 140 11.2.2 Hamiltonian Recursion vs. Riccati Recursion ...... 143 11.3 Industrial experiment 2: Continuous stirred tank reactor (CSTR) ...... 146 11.3.1 Control design objective and settings ...... 147 11.3.2 Simulation ...... 149 11.3.3 Real-time controller test ...... 151

12 Conclusions of part III 153 12.1 Further work ...... 153

4 A Relevant code for parts II and III 155

A.1 HQ preconditioner function ...... 155

A.2 D˜hqr companion-like matrix recursion function ...... 157

A.3 D˜hqr companion-like vector recursion function ...... 159

A.4 HR preconditioner function ...... 160

A.5 Application of GMRES method for companion-like matrix D˜hqr ...... 161 A.6 S-Function for Flotation-cell system ...... 163 A.7 S-Function for Continuous stirred tank reactor (CSTR) ...... 165

B Double-tank model 167

54, 359 words

5 List of Figures

1.1 Layout of the novel contribution presented in part II, and its place within the Model Predictive Control (MPC) framework...... 21

3.1 Computation of gain matrix P1 for the receding horizon control on the LQR. Comparison between the Riccati equation (red) and the eigendecomposition approach from Algorithm 3.2 (blue)...... 38

4.1 Computation of gain matrix P1 and gain vector π1 for the receding horizon control on the LQ tracking problem. Comparison between the Riccati equation (red) and the eigendecomposition approach from Algorithm 4.2 (blue)...... 51

6.1 Simulation with MPC controller using the Riccati recursion method. . . . . 69 6.2 Comparison of the computational time during the simulation of the online part of the MPC controller using the Riccati recursion and eigenvalue method. 76 6.3 Convergence of a naive implementation of the splitting method on system (6.23). The number of iterations on the splitting method was set at i = 30. Each line represents a vector whose length is the number of k−iterations of the Mehrotra’s algorithm (Algorithm 5.1). The elements of the vector correspond to the residual norm after i−iterations...... 78

7.1 Relative error corresponding to the optimal solution vector, p = [x, u, λ, µ, t], during a closed-loop MPC simulation on a second order plant. The x axis corresponds to the sample time t. The y axis represents the interior-point

iterations, k. The relative error (z axis) is computed by ρ = kpcom −

prick∞/kprick∞ at each (t, k)...... 90 7.2 Comparison of the computational time during the simulation of the MPC controller on system (7.1) between the Hamiltonian recursion (blue) and the Riccati recursion (red) methods...... 91

7.3 MPC simulation for the oscillating-masses example with nx = 40 states using the Hamiltonian/GMRES solver within an interior-point algorithm.

First plot shows the evolution of the states xk subject to constraints, with

initial conditions xk = 3.5 for the displacements, and xk = 0 for the velocities. Second plot shows the proﬁle of the control inputs subject to constraints. 97

6 7.4 Convergence of the Hamiltonian/GMRES solver for the oscillating-masses

example with nx = 40 states. The x axis corresponds to the sample time t throughout the MPC simulation. The y axis corresponds to the iterations of the interior-point algorithm at every sample time/(t − sample). The z axis shows the proﬁle of the number of iterations of the GMRES method that solves system (7.1)...... 98 7.5 Relative residual of the Hamiltonian/GMRES solver for the oscillating- masses example. The x axis corresponds to the sample time t throughout the MPC simulation. The y axis corresponds to the iterations of the interior-point algorithm at every sample time/(t − sample). The z axis shows the proﬁle of the relative residual of the GMRES method that solves system (7.1)...... 98 7.6 Convergence of the Hamiltonian/GMRES solver for the oscillating-masses

example with number of masses m = 80 (nx = 160 states). The x axis corresponds to the number of samples time/(t − sample) throughout the MPC simulation. The y axis corresponds to the iterations of the interior- point algorithm. The z axis refers to the number of iterations of the GMRES method that solves system (7.1)...... 99 7.7 Four diﬀerent MPC simulations for the oscillating-masses example with

nx = 40 states using the Hamiltonian/GMRES solver within an interior- point algorithm. With four different values of the horizon, N = 5, 10, 20, 40. The x axis corresponds to the GMRES iterations and the y axis represents the number of recurrences that happen throughout all the IPM iterations, without distinction of the sample time in which a given GMRES run occurred.100 7.8 Comparison between the Hamiltonian/GMRES solver (Algorithm 7.3) and the GMRES preconditioned method proposed in [1]. All the parameters for the MPC simulation of the oscillating-masses example resembled exactly those in [1]. The x axis corresponds to the IPM iterations and the y axis represents the number of GMRES iterations...... 101 7.9 Closed-loop simulation of the second order system (6.17). Double precision, u = 1 × 10−16, was used (which is standard for MATLAB®). The MPC controller is tested by two optimization solvers, the Hamiltonian/GMRES and the Riccati recursion...... 103 −1 −1 7.10 Infinity norm k·k∞ profile of the vector M T . For clarity, every plot also highlights the maximum and minimum norm. Double precision, u = 1 × 10−16, was used. The x axis corresponds to the number of samples time/(t − sample) throughout the MPC simulation. The y axis represents the iterations of the interior-point algorithm...... 104 7.11 Closed-loop simulation of the second order system (6.17) by an MPC controller with the Hamiltonian/GMRES solver. High precision arithmetic was set up at u = 1 × 10−45, using the Multiprecision Computing Toolbox [2]. . . 105 7.12 Condition number κ of matrix D˜ = H−1D H−1 subject to variations hqrα Qα hqr R of the α parameter. Experiment set with horizon N = 2...... 107

7 7.13 Eigenvalue trajectories of matrix D˜ = H−1D H−1. Each trajectory hqrα Qα hqr R (y axis) is the absolute value |µ| of an eigenvalue µ = a + bi. The x axis represents the variation of the α parameter...... 108 7.14 Imaginary plane for the eigenvalues µ = a+bi of matrix D˜ = H−1D H−1. hqrα Qα hqr R Experiment set with horizon N =2...... 109 7.15 GMRES experiment with diﬀerent values of the horizon N. The x axis represents the α parameter and the y axis is the number of GMRES iterations.110 7.16 GMRES experiment with diﬀerent values of the horizon N. The x axis represents the α parameter and the y axis is the condition number κ of matrix D˜ = H−1D H−1...... 111 hqrα Qα hqr R

9.1 A Bionic Kangaroo designed by Festo Bionic Learning Network. The ﬂight phase control uses an optimal state feedback controller [3]...... 117 9.2 MPC hierarchy found in industrial applications (diagram from [4])...... 118 9.3 Some of the major control-systems vendors oﬀering MPC software in their corresponding product portfolio. Within the industrial jargon, MPC software is known as APC which stands for Advanced Process Control. All these brands are registered trademarks...... 119 9.4 CPX platform. Festo proprietary PLC...... 120 9.5 Research overview of the implementation of MPC algorithms on PLCs. Ref- erences: 1. G. Valencia-Palomo & J.A. Rossiter [5], 2. M. Kvasnica et al. [6], 3. G. Frison et al. [7], 4. D.K.M. Kufoalor et al. [8], 5. B. Huyck et al. [9] and 6. M. Pereira et al. [10]...... 121

10.1 Closed-loop system diagram with MPC controller for tracking reference and zero offset...... 127 10.2 Flotation-cell system for three tanks connected in series...... 128 10.3 Closed-loop digital MPC controller for a Flotation-cell system with three tanks. The MPC controller is based on the control diagram in Figure 10.1 using a Matlab® function. For the plant, an S-function is coded for three nonlinear first-order differential equations (10.18)...... 134 10.4 Closed-loop MPC simulation for a Flotation-cell system with three tanks using the optimization solver presented in Algorithm 7.1...... 135 10.5 Code generation process diagram for an MPC algorithm, from MATLAB® code to Structured-text language (IEC 61131-3 environment)...... 136

11.1 Test rig for an MPC controller experiment. It has two PLCs and a touch screen (HMI). Physical connection between equipment is done through Eth- ernet TCP/IP protocol...... 138 11.2 Hardware-in-the-loop (HIL) diagram. The MPC controller is implemented on the PLC and the model is running in MATLAB®. All the equipment is connected through OPC protocol (Open Platform Communications). . . . 139 11.3 Image captured from CODESYS environment. It shows the code size in the PLC after the code generation process (Figure 10.5)...... 140

8 11.4 Image captured from Festo HMI touch screen device showing real-time performance of the main tailor-made screen for the Flotation-cell system. . . . 141 11.5 Real-time closed-loop MPC controller for a Flotation-cell system with three tanks using the optimization solver presented in Algorithm 7.1...... 141 11.6 Image captured from CODESYS already logged into the Festo PLC (online). It shows the MPC function block...... 142 11.7 Image captured from CODESYS already logged into the Festo PLC (online). It shows the cycle time for the solution of the optimization problem (5.1). . 142 11.8 Image captured from Festo HMI touch screen device showing real-time performance of the main tailor-made screen for the Flotation-cell system with two PLCs. Comparative test between Hamiltonian recursion and Riccati recursion methods...... 143 11.9 Measured outputs y corresponding to the level sensors. Two real-time closed-loop MPC controllers running in parallel for two Flotation-cell systems, with three tanks each. Two optimization solvers used: (i) the Hamil- tonian recursion method, in blue (Algorithm 7.1) and (ii) the Riccati recursion approach, in red (Algorithm 6.1)...... 144 11.10 Manipulated variables u corresponding to the process valves. Two real- time closed-loop MPC controllers running in parallel, for two Flotation-cell systems with three tanks each. Two optimization solvers used: (i) the Hamiltonian recursion method, in blue (Algorithm 7.1) and (ii) the Riccati recursion approach, in red (Algorithm 6.1)...... 145 11.11 Diagram of a Continuous stirred tank reactor (CSTR), from [11]...... 146 11.12 Closed-loop digital MPC controller for a CSTR reactor. The MPC controller is based on the control diagram in Figure 10.1 using a Matlab® function. For the plant, an S-function is coded for the nonlinear third-order diﬀerential equations (11.1)...... 149 11.13 Closed-loop MPC simulation for a Continuous stirred tank reactor (CSTR) using the optimization solver presented in Algorithm 7.1. The measured outputs y correspond to the molar concentration c, the reactor temperature T and the level tank h. The inputs u are the outlet ﬂow F and the

temperature Tc of the jacketed cooling system...... 150 11.14 Image captured from CODESYS showing the size of the code after generating the runtime ﬁles for the MPC controller...... 151 11.15 Image captured from the Festo HMI touch screen device showing real-time performance of the main tailor-made screen for the Continuous stirred tank reactor (CSTR)...... 151 11.16 Real-time closed-loop MPC controller experiment for a Continuous stirred tank reactor (CSTR) using the optimization solver presented in Algorithm 7.1. The measured outputs y correspond to the molar concentration c, the reactor temperature T and the level tank h. The inputs u are the outlet

ﬂow F and the temperature Tc of the jacketed cooling system...... 152

B.1 Flotation-cell system for three tanks connected in series (a single-tank array in each side and a double-tank array in the middle)...... 167

9 List of Algorithms

3.1 Solution of LQR with Vaughan’s method ...... 37 3.2 Non-recursive for unconstrained free-ﬁnal state MPC ...... 37 4.1 Eigenvalue decomposition for LQ tracking ...... 49 4.2 Eigenvalue decomposition for unconstrained MPC tracking problem . . . . 50 5.1 Mehrotra’s Predictor-Corrector ...... 61 6.1 Riccati recursion method for the step direction computation ...... 68 6.2 Eigenvalue factorization method for the step direction computation . . . . . 71

6.3 Eigenvalue factorization for Mc oﬄine part (time-variant systems) . . . . . 74

6.4 Eigenvalue factorization for Mc oﬄine part (time-invariant systems) . . . . 75

6.5 Eigenvalue factorization for Mc online part (time-invariant systems) . . . . 75

7.1 Direct factorization of system (7.1) via companion-like matrix D˜hqr ..... 86 7.2 Remaining step direction computation ∆˜ p ...... 87 7.3 Fast Hamiltonian/GMRES solver for the solution of system (7.1) using the

companion-like matrix D˜hqr ...... 95

10 List of Tables

7.1 Computational cost dependent on N of the Hamiltonian recursion Algo- rithm 7.1 ...... 89 7.2 Computational cost dependent on N of the Riccati recursion Algorithm 6.1 89 7.3 Computational cost of the fast Hamiltonian/GMRES Algorithm 7.3, dependent on horizon N ...... 94

9.1 Comparative PLC properties between diﬀerent manufacturers ...... 121 9.2 APC software embedded on PLCs currently oﬀered by some manufacturers 122

10.1 Physical parameters for the Flotation-cell system, refer to [12]...... 132

11.1 Physical parameters for the Continuous stirred tank reactor (CSTR), refer to [13]...... 147

11 Abstract

Reliable and optimal decision-making tools are essential in industry to achieve high performance. One of these tools is Model Predictive Control (MPC), which is an advanced control technique that generates an action that affects the controlled variables of a given process with respect to a performance criteria, while satisfying the process’ physical and operational restrictions. At the core of the MPC algorithm lies an optimization problem that is solved by a numerical method at every sample time. New demand for more self-contained and autonomous modular processes has seen MPC embedded in small- scale platforms, such as Programmable Logic Controllers (PLCs). This has prompted a need for custom-made numerical methods that help to efficiently run the computationally demanding optimization algorithms. In this thesis, we design effective optimization solvers for PLCs by proposing several approaches that factorize the Newton system of the interior-point method (IPM). These approaches are based on the two-point boundary- value (TPBV) problem structure, rarely explored in MPC, called the Hamiltonian system. One of the main proposals is that, once the augmented system is in the Hamiltonian form, it can be reduced to an incomplete LU factorization in which two possible options are available to compute the solution of the system: (i) A direct method called the Hamil- tonian recursion method, and (ii) an iterative method called the Hamiltonian/GMRES method. Regarding the former, a forward substitution of a sequence of matrices is carried out, whereas with the latter, a Krylov method is used. We prove that the convergence of the iterative method is bounded and its rate is quantified. Numerical experiments demonstrate that both methods are feasible and efficient compared to the state-of-the-art methods. Declaration

No portion of the work referred to in the thesis has been submitted in support of an application for another degree or qualiﬁcation of this or any other university or other institute of learning

13 Copyright

i. The author of this thesis (including any appendices and/or schedules to this thesis) owns certain copyright or related rights in it (the Copyright) and s/he has given The University of Manchester certain rights to use such Copyright, including for administrative purposes.

ii. Copies of this thesis, either in full or in extracts and whether in hard or electronic copy, may be made only in accordance with the Copyright, Designs and Patents Act 1988 (as amended) and regulations issued under it or, where appropriate, in accordance with licensing agreements which the University has from time to time. This page must form part of any such copies made.

iii. The ownership of certain Copyright, patents, designs, trademarks and other intellectual property (the Intellectual Property) and any reproductions of copyright works in the thesis, for example graphs and tables (Reproductions), which may be described in this thesis, may not be owned by the author and may be owned by third par- ties. Such Intellectual Property and Reproductions cannot and must not be made available for use without the prior written permission of the owner(s) of the relevant Intellectual Property and/or Reproductions.

iv. Further information on the conditions under which disclosure, publication and com- mercialisation of this thesis, the Copyright and any Intellectual Property and/or Re- productions described in it may take place is available in the University IP Policy (see http://documents.manchester.ac.uk/DocuInfo.aspx?DocID=24420), in any relevant Thesis restriction declarations deposited in the University Library, The University Li- brary’s regulations (see http://www.library.manchester.ac.uk/about/regulations/) and in The University’s policy on Presentation of Theses.

14 Acknowledgments

-The number of people who have been kind enough to help me, directly or indirectly, throughout this amazing journey is simply untraceable, and therefore, I apologize in advance to those who I unintentionally left out from the following acknowledgements.

-My main gratitude to my supervisor, Prof. William P. Heath, who first of all believed in my PhD proposal and thereafter, thoroughly guided me through the difficult (but yet fascinating) intricacies of the PhD. I can now say, it was all worthy and specially fun. Also special thanks to all the colleagues from the control group who have given me advice and guidance. Likewise, I am most grateful to Stefan Güttel, from the School of Mathematics, whose collaboration turned out to be essential for the content of chapters 6 and 7.

-To David Bell, whose friendship has had the perfect balance, from discussing ideas about optimal control to enduring those oﬀ-road runs up on the hills in the Peak District.

-To all my family; my mom, my grandmother, my two sisters and my brother, who have constantly given me love and support. I promise that I will make up for all the time that

I have been away.

-To Isabelle, whose love and unconditional support made this PhD project a success, which would not have been possible otherwise.

-And last but not least, the research presented throughout this dissertation has received funding from two entities, Festo AG & Co.KG and the Mexican Council of Science and

Technology (CONACYT). Festo granted me a contract for a feasibility study and the development of a built-in MPC function block for their PLCs. Specially, I would like to thank

Chris Stich and Florian Zieker, from the PA Future Concepts, for their constant support and their trust from the beginning of the project. I am also truly grateful to CONACYT for providing me during 5 years with the most valuable asset all over the world: time.

Eduardo Poupard

Manchester UK, 2018

15 To my mom, grandmother and siblings who have always been there Chapter 1

Introduction

1.1 Background and motivation

Model Predictive Control (MPC) is an optimization technique that uses a dynamic model of a process to generate a prediction of the control input actions subject to inequality constraints. It is this last feature, along with its multivariable structure, that sets MPC apart from other more classical control techniques, i.e. the natural incorporation of the system’s physical restrictions and the possibility to integrate several I/Os, all within the same controller [4]. The history of MPC dates back to the 1970s, especially in the petrochemical industry where MPC was applied to large-scale systems, for instance, distillation columns. Some of the ﬁrst research published about MPC was by (i) [14], where MPC was called Model Algorithmic Control (MAC) and (ii) [15], where MPC was deﬁned as

Dynamic Matrix Control (DMC). During the last two decades, some of the major control software vendors have picked up on the MPC advantages and it can now be found in a wide variety of industries, especially in the process industry [16]. Nowadays, due to advances in research and computational technologies, it is a matter of time before MPC is widespread and has a direct eﬀect on several public topics, such as cancer treatment [17].

From a mathematical viewpoint, an MPC controller computes the solution of an optimization problem at every sample time. This means that some necessary and sufficient conditions have to be satisfied. In the case of the necessary conditions, these boil down to a nonlinear system of equations that has to be solved by a numerical method. Since there has always been a tradeoff between the size of the system and the time needed to obtain the solution of the optimization problem, much of the research over the last years has been

17 about developing state-of-the-art techniques that render these numerical methods more eﬃcient. This is exactly where this thesis’ motivation lies.

This thesis proposes a couple of methods that eﬃciently compute the two-point boundary- value (TPBV) problem from optimal control. We chose to address this problem because it is the natural structure (in the form of a linear system of equations) that appears within the Sequential Quadratic Programming (SQP) algorithm, which is an optimization method widely used for nonlinear MPC (refer to [18] and [19]). An application using SQP based on the Riccati recursion can be found in [20]. The Riccati approach is used throughout the thesis to compare the performance of our own methods. Riccati-based solvers can be found in [21], [7], [22] and [23]. Furthermore, high performance optimization solvers that can be deployed into embedded platforms are FORCES [24], qpDUNES [23] and OSQP [25].

Analysis of their corresponding applicability and performance can be found in [26], [23] and [25] respectively. Additional optimization solvers for general purposes are ACADO

Toolkit [27] and OOQP [28].

Likewise, the need for more autonomy on self-contained processes has seen the MPC applied to small-scale systems [29], where embedded platforms are commanding the control task. Platforms such as ﬁeld-programmable gate arrays (FPGAs) have received plenty of attention from the MPC research community. Work on this can be found in [30], [31], [32] and [33]. In this thesis, we analyze the online implementation of MPC on another kind of platform quite widespread in industry, called Programmable Logic Controllers (PLCs).

The collaboration between a company (more details about this below) and our research has looked at the development of tailor-made optimization solvers for such devices.

Finally, the main focus of this thesis is on the numerical feasibility and eﬃciency of the proposed MPC algorithms, and therefore topics regarding robustness, stability and optimality of MPC are beyond the scope of this current work. Excellent references for these topics are [34], [13] and [35].

1.2 Statement of originality and thesis outline

This thesis contains 12 chapters and two appendices, split in three parts,

1. Unconstrained Optimal Control Problem.

2. Constrained Optimal Control Problem.

18 3. Embedded Model Predictive Control.

The chapters corresponding to each part and their speciﬁc details are described in the following sections.

1.2.1 Part I

This part contains four chapters related to optimal control problems that have only equality constraints. In chapter 2, we briefly outline the fundamentals of the free-final state optimal control formulation in which its necessary conditions give rise to the two-point boundary-value (TPBV) problem. This is the only mathematical framework required throughout the thesis, since we deal with only quadratic objective functions with linear constraints (equality and inequality), and therefore, the sufficient conditions are satisfied.

In chapter 3, a brief analysis of the ﬁnite-horizon Linear Quadratic Regulator (LQR) and its fundamentals are given for discrete-time dynamic systems, with emphasis on the linear system of equations required to solve the problem. The factorization of this system could be carried out by two methods mainly, (i) by the well-known Riccati equation [36], or

(ii) by an eigenvalue decomposition (less known) [37]. We present here the ﬁrst contribution of the thesis related to the implementation of the eigenvalue decomposition on the receding horizon idea for unconstrained Model Predictive Control (MPC), along with its implications for online applications.

In chapter 4, the fundamentals of the Linear Quadratic tracking problem are given along with an extension of the eigenvalue decomposition to factorize this problem. This is the second contribution of the thesis. As a third contribution, we also apply this extension to the receding horizon idea and its implications on online controllers are discussed. The novel results in this part are published in [38].

1.2.2 Part II

This part contains four chapters, mainly related to eﬃcient numerical methods applied to optimization algorithms for MPC. In chapter 5, a brief outline of the Model Predic- tive Control (MPC) framework is given, especially about the extension of the necessary conditions presented in part I, which are known as the Karush-Kuhn-Tucker (KKT) conditions [39], and include the inequality constraints into the formulation. As stated before, the suﬃcient conditions hold due to a quadratic performance index and linear constraints

19 (convex). When the necessary conditions become nonlinear a numerical method is needed, as opposed to the LQR, whose necessary conditions are linear and there exists a closed- form solution. The primal-dual interior point method [40] is chosen because the size of the problem remains constant throughout the iterations (contrary to active-set methods) and it has a polynomial-time convergence. For practical use, there exist some varieties of interior-point algorithms, such as the Mehrotra’s Predictor-Corrector algorithm [41], which is used here as an outer algorithm for the methods presented in the following chapters.

The interior-point method uses the second derivative of the gradient, which boils down to a linear system of equations (Newton system). In chapter 6, we decompose the Newton system, which resembles the structure of the two-point boundary-value (TPBV) problem, into the Hamiltonian form. We then present three algorithms to factorize this system, (i) the well-known Riccati recursion method [21], (ii) the eigenvalue decomposition and (iii) a modiﬁcation of the eigenvalue decomposition method, which drops some of the computational cost oﬄine and uses the splitting method to compute the remaining part. The contributions presented up to this part are published in [38] and [42] (in abstract form).

Chapter 7 presents the main contributions of this thesis. In this chapter, the Hamiltonian form system is preconditioned by two matrices (left and right), giving rise to an incomplete

LU factorization. An interesting result is that the matrix produced by this factorization can be given explicitly and has the form of a companion-like matrix. We propose two methods to factorize this matrix: (i) A direct method called Hamiltonian recursion solver, which carries out a forward substitution, computing a sequence of matrices that are then used to factorize the system. Some numerical experiments are shown to demonstrate feasibility and equivalence against the Riccati recursion method. (ii) An iterative method called Hamiltonian/GMRES solver, which uses a Krylov method to ﬁnd the solution of the system. Fast convergence is a requirement for iterative methods in order to be eﬃcient against direct methods. We prove that if the GMRES method is used to solve the system of the companion-like matrix, convergence is guaranteed to be bounded and depends on the size of the system (number of states). Several experiments are carried out so as to prove convergence and feasibility. We also address a special case in which this convergence is no longer guaranteed. Figure 1.1 depicts a summary of the novel methods presented in chapter 7, along with the splitting approach from chapter 6, and their place within the

Model Predictive Control framework. The methods, results, theorems and experiments

20 presented throughout this chapter (and the splitting method) will be part of a journal paper which is currently in preparation.

Nonlinear Model Predictive Control (NMPC)

Numerical Optimization Methods: Newton Type Methods

Sequential Quadratic Programming (SQP)

Interior-Point Method (IPM)

Linear-quadratic subproblems factorization Two-point boundary-value Problem (TPBV)

Preconditioned Preconditioned Hamiltonian\Eigenvalue Hamiltonian Hamiltonian Riccati-Recursion Decomposition and Decomposition with Decomposition with Method Splitting methods GMRES method Symplectic recursion

Iterative methods Direct methods

Novel methods

Band-structure exploiting methods

Figure 1.1: Layout of the novel contribution presented in part II, and its place within the Model Predictive Control (MPC) framework.

Chapter 8 gives a brief analysis of the main results from parts I and II, highlighting the opportunities for future research.

1.2.3 Part III

The ﬁnal part of the thesis contains four chapters, mainly addressing the implementation of MPC algorithms on small-scale devices known as Programmable Logic Controllers

(PLCs). In chapter 9, we give the details of the collaboration between our research project and a company called Festo AG & Co.KG.

Part I and part II, except for chapter 4, deal exclusively with the regulator problem, i.e. the controller drives the states from some initial conditions to the equilibrium point (zero).

In chapter 10, we provide the extension in which all the methods presented previously can be applied to the tracking MPC problem. Likewise, an industrial case study is presented,

21 called the Flotation-cell system, where the Hamiltonian recursion method is tested in several simulations. After this, the details and steps for the code generation are given. The objective of this part is to translate the code of the Hamiltonian recursion method, written in Matlab®, into the PLC language.

After the PLCs code is generated, a test rig is set up with two PLCs and a HMI touch panel (in chapter 11). The necessary code for the PLCs and the touch screen is developed.

To evaluate performance, a Hardware-in-the-loop (HIL) test is carried out between the real-time MPC controller (based on the Hamiltonian recursion method) embedded in the

PLCs; and a Flotation-cell system emulated in Matlab®. In addition, a comparative test is done between the Hamiltonian recursion method and the Riccati recursion approach.

For the last part of this chapter, an additional industrial case study is tested, the Con- tinuous stirred tank reactor (CSTR), in order to evaluate performance and to determine whether the Hamiltonian recursion method is versatile and numerically feasible in embedded platforms. Chapter 12 gives a brief analysis of the main results from part III, highlighting the opportunities for future research. We comment on possible alternatives to make the generated code more eﬃcient.

1.2.4 Additional work

A byproduct of this research is published in [43]. An MPC algorithm designed for this thesis identiﬁes the optimal state trajectory and commands the DC motor controllers installed on a multi-actuator system. This implementation resulted in energy savings for the electromechanical system.

1.2.5 Mathematical tools

The foundations of this thesis are:

Optimal control theory, with references found in [44] and [36].

Numerical methods for optimization problems, with references found in [45] and [46].

For the interior point method, one can refer to [40].

Applied linear algebra, specially references related to the factorization of systems of

linear equations, such as [47] and [48].

22 Part I

Unconstrained Optimal Control Problem

23 Chapter 2

Free-ﬁnal State Optimal Control Problem

2.1 Continuous-time analysis

This part deals with unconstrained optimal control problems. In order to avoid ambiguity with the title, we refer to problems that are subject solely to equality constraints (such as dynamical systems). In part II, we will work with an extension to these problems by adding inequality constraints into the formulation.

2.1.1 Problem formulation

The calculus of variations gives rise to a suitable framework to solve optimal control problems. It is an extensive topic that is supported by rigorous mathematical proofs. Here, we will present a brief summary oriented to the part that is important for our description ahead. For more information related to calculus of variations within the optimal control theory refer to [49] and [44]. We will consider a continuous system expressed as a vector form which contains nonlinear diﬀerential equations

x˙(t) = f(x(t), u(t), t), (2.1) where the vector function f : R(n+m+1) 7→ Rn has continuity condition in order to have a

n m unique solution, where x(t) ⊂ R and u(t) ⊂ R . The initial conditions, x(t0) are given.

24 Moreover, consider the following scalar performance index

Z tf J = ϕ(x(tf ), tf ) + L(x(t), u(t), t)dt, (2.2) to where ϕ(x(tf ), tf ) is the terminal cost and L(x(t), u(t), t) is a scalar function. The problem

∗ is to ﬁnd an optimal control law u (t) which minimizes (2.2) over the interval to ≤ t ≤ tf subject to system (2.1). Using the Lagrange multiplier λ(t) to adjoin the system, the performance index becomes

Z tf > Jˆ = ϕ(x(tf ), tf ) + L(x(t), u(t), t) + λ (t)[f(x(t), u(t), t) − x˙] dt. (2.3) to

Deﬁning another scalar function which encapsulates L(x(t), u(t), t) and λ(t), whose name is the Hamiltonian Function (based on the Hamiltonian’s principle in mechanics [44, 36]).

H(x(t), u(t), λ(t), t) = L(x(t), u(t), t) + λ>f(x(t), u(t), t). (2.4)

Including the Hamiltonian function (2.4) in the performance index (2.3)

Z tf > Jˆ = ϕ(x(tf ), tf ) + H(x(t), u(t), λ(t), t) − λ (t)x ˙ dt. (2.5) to

We need to ﬁnd from (2.5) an expression that represents its ﬁrst variation. Thus, carrying out integration by parts of the last term of Jˆ (see [44])

Z tf > > ˙ > Jˆ = ϕ(x(tf ), tf ) − λ (tf )x(tf ) + λ (to)x(to) + H(x(t), u(t), λ(t), t) + λ x(t) dt. (2.6) to

As previously mentioned, the control input u(t) has to minimize (or maximize) the performance index. Then, let (2.6) be evaluated by the ﬁrst variation for ﬁxed times, to and tf

" # " # ∂ϕ Z tf ∂H ∂H δJˆ = − λ> δx + λ>δx + + λ˙ > δx + δu dt. (2.7) t=to ∂x | {z } to ∂x ∂u | {z } t=tf 2 | {z } |{z} 1 3 4

2.1.2 Two-point boundary-value problem

The ﬁrst variation δJˆ has to be zero in order to be an extremum. To avoid determining

δx and δu explicitly, we make the coeﬃcients of (2.7) vanish:

1. At the ﬁnal time tf the coeﬃcient of the variation δx(tf )(x(tf ) is not known) must

25 vanish. Namely, the Lagrange multiplier λ(tf ) must be equal to

> ∂ϕ λ (tf ) = . (2.8) ∂x(tf )

2. As the initial state is given x(to), the variation δx(to) is zero and therefore this term vanishes.

3. It would be complicated to analyze directly the variation of δx and δu along the

trajectory within the integrand, so it is easier to make their coeﬃcients vanish, i.e.

for the states x

∂H λ˙ > = − . (2.9) ∂x

4. And for the control inputs u

∂H = 0. (2.10) ∂u

Equations (2.1), (2.9) and (2.10) are known as the necessary conditions of the optimal control problem (2.2). For more information, one can refer to the Pontryagin’s minimum principle [50]. The solution of this system renders Jˆ an extremum (minimum or maximum).

To ensure Jˆ to be a minimum, it is well known that its second order variation must be

δJ 2 ≥ 0 [36]. This condition holds if a quadratic performance index and linear constraints are considered. This will be the case throughout this thesis. The boundary conditions of the system are the initial condition of the state x(to) and the ﬁnal condition of the Lagrange multiplier deﬁned in (2.8). Such a problem is known as the two-point boundary- value problem (TPBVP) and there exist several methods to solve it. In the next chapter we will present two of these methods, the Riccati equation and the eigenvalue decomposition.

2.2 Discrete-time analysis

Throughout this dissertation we are mostly interested in computer-based controllers that are applicable to real-time industrial applications. Hence, it is necessary to have a suitable optimal control approach that renders the previous section (the continuous formulation) usable for these applications. Such an approach is the discrete-time formulation, which

26 is an approximation of the continuous one. We will present an analogy of the previous section suited for discrete-time systems.

2.2.1 Problem formulation

The following description is standard, from [44, p.42-45] and [36, p.19-24]. The analogy of system (2.1) to a nonlinear discrete-time dynamical system is

k xk+1 = f (xk, uk) , (2.11)

nx with initial condition x0. The state variables are deﬁned as xk ∈ R and the control

nu inputs uk ∈ R . The discrete performance index is

N−1 X k J = φ (N, xN ) + L (xk, uk) , (2.12) k=0 where function φ is a terminal cost that evaluates the ﬁnal state xN at the ﬁnal sample time k = N. The function Lk is a Lagrangian function. The form of these two functions depends on the application itself and one that suits a wide variety of systems in engineering is the quadratic function, which will be utilized throughout this dissertation. Since this optimal control problem is subject to the equality constraints (2.11), its derivative df, with respect to x and u, has to be adjoined to the derivative of the performance index (2.12), dJ. To do so, J can become

N−1 X h k | k i J = φ (N, xN ) + L (xk, uk) + λk+1 f (xk, uk) − xk+1 , (2.13) k=0 where λk+1 is called the Lagrange multiplier of the equality constraints. An additional

k scalar function named the Hamiltonian is used to encapsulate L and λk+1 and thus, facilitating handling in subsequent derivations. This Hamiltonian function is

k k | k H (xk, uk) = L (xk, uk) + λk+1f (xk, uk) . (2.14)

The performance index after some manipulations is now

N−1 | 0 X h k | i J = φ (N, xN ) − λN xN + H (x0, u0) + H (xk, uk) − λkxk . (2.15) k=1

27 A necessary condition to ﬁnd a minimum of (2.15) is that its increment dJ should be zero

0 0 ∂φ | ∂H ∂H dJ = − λN dxN + dx0 + du0 ∂xN ∂x0 ∂u0 | {z } | {z } | {z } 1 2 3a     N−1 k k N k−1 X  ∂H | ∂H  X  ∂H |  +  − λk dxk + duk +  − xk dλk . (2.16)  ∂xk ∂uk   ∂λk  k=1 | {z } | {z } k=1 | {z } 4 3b 5

2.2.2 Two-point boundary-value problem

As shown in section 2.1.2, in order for dJ to be zero, all its coeﬃcients must vanish. These are the necessary conditions for optimality. The interpretation of (2.16) is as follows,

1. The boundary condition of the Lagrange multiplier is

∂φ λN = . (2.17) ∂xN

2. As the initial state is given x0, then dx0 = 0 and this term vanishes.

3. To avoid determining directly duk from k = 0,...,N − 1, we set its coeﬃcients to zero, namely

∂Hk = 0. (2.18) ∂uk

Equation (2.18) is known as the stationary condition.

4. The same as before with dxk from k = 0,...,N − 1, its coeﬃcient must vanish

∂Hk λk = . (2.19) ∂xk

5. And also for dλk from k = 1,...,N, its coeﬃcient must vanish

∂Hk xk+1 = . (2.20) ∂λk+1

Relations (2.19) and (2.20) are known as the costate and state equations respectively. This system, along with the initial x0 and ﬁnal condition (2.17) is the two-point boundary-value problem (TPBVP) for the discrete-time optimal control problem of system (2.11).

28 Chapter 3

Linear Quadratic Regulator

In 1960, Kalman [51] showed that the linear quadratic regulator (LQR) can be solved by applying the Riccati equation backwards in time. One technique used to solve this equation was proposed by MacFarlane [52], where an eigenvalue decomposition is applied.

This method is limited since it can be only used for infinite-horizon problems. Vaughan generalized this approach (for continuous [53] and discrete-time systems [37]), where the solution can also be obtained for finite-horizon problems. In Vaughan’s work the prediction horizon N is included explicitly within the algorithm. In this part we first present the basic formulation of the free-final state finite-horizon LQR formulation, followed by the well-known solution method, the Riccati equation. We finalize with the implementation of the eigenvalue decomposition as an alternative solution for the problem.

3.1 Finite-horizon formulation

The following description is standard [36, p.32-47]. The mathematical formulation of the free-ﬁnal state problem for the ﬁnite-horizon LQR is described below

N−1 1 | 1 X | | min x PN xN + x Qxk + u Ruk X ,U 2 N 2 k k k=0

s.t. xk+1 = Axk + Buk, x0 = x(0) (3.1)

nx where the aﬃne equality constraints represent the system dynamics, xk ∈ R is the

nu state variable with sequence X = (x0, x1, . . . , xN ), uk ∈ R is the input variable with sequence U = (u0, u1, . . . , uN−1) and N is the prediction horizon. We will assume across

29 √ this dissertation that the pair (A, B) is stable and the pair Q, A is detectable. For simplicity, we will consider a time-invariant system. Weighting matrices (Q, P ) ≥ 0 and

R > 0 are symmetric. The Hamiltonian function for (3.1) is

1 Hk = x|Qx + u|Ru + λ| (Ax + Bu ) , (3.2) 2 k k k k k+1 k k

nx where λk ∈ R is the Lagrange multiplier. By applying the necessary conditions for optimality deﬁned in (2.20) and (2.19) , the coupled state and costate equations can be described as follows

      x A −BR−1B| x  k+1    k    =     , (3.3) λk QA| λk+1 where the coeﬃcient matrix in this system is referred as the discrete Hamiltonian matrix.

Assuming matrix A is non-singular, system (3.3) can be expressed as a backward recursion, i.e.

      −1 −1 −1 xk A A BR B| xk+1   =     , (3.4)    −1 −1 −1    λk QA A| + QA BR B| λk+1

2nx×2nx where the coeﬃcient matrix of (3.4) is deﬁned as Hm ∈ R . Now, using the stationary condition (2.18)

∂Hk | 0 = = Ruk + B λk+1, (3.5) ∂uk we can obtain the optimal control input

−1 | uk = −R B λk+1. (3.6)

We also need to take into account the initial condition x0 and the ﬁnal condition (2.17) for this two-point boundary-value problem, that for this case becomes

∂φ λN = = PN xN . (3.7) ∂xN

To solve the problem it is necessary to use the sweep method presented in Bryson & Ho [44, p. 47] that computes a sequence of nx × nx matrices, Pk, which makes the ﬁnal condition

30 (3.7) hold for all the times k ≤ N, i.e.

λk = Pkxk. (3.8)

Bearing in mind (3.8), we can now set the control input (3.6) in terms of the states in order to have a state feedback controller. To achieve this, we substitute the dynamical system of (3.1) and solve for uk in (3.6), i.e.

| −1 | uk = − (B Pk+1B + R) B Pk+1Axk. (3.9)

To obtain the sequence of matrices Pk, two diﬀerent methods are presented in sections 3.2 and 3.3 respectively.

3.1.1 Linear system of equations

One of the structures seldom explored in the optimal control literature of the ﬁnite-horizon problem is its general form as a linear system of equations. Strictly speaking, the solution to this problem boils down to the solution of a system in the form of Dz = b. All methods used to solve the optimal control problem are in fact carrying out a factorization of such a system and in doing so, ﬁnding the system’s solution.

To show this general form we will ﬁrst change slightly the system (3.4) by decomposing matrix Hm into

        A 0 x IBR−1B| x    k    k+1     =     . (3.11) −QI λk 0 A| λk+1

The system (3.11) does not explicitly use the inverse of A, which does not exist in systems with delays. It follows that this system forms, along with the initial condition x0 and the ﬁnal condition (3.7), the following system of equations of the two-point boundary-value

31 problem

      −I xk x0             −A 0 IG   λk   0               Q −I 0 A|  x   0     k+1                −A 0 IG  λk+1  0         ..       Q −I 0 A| .  xk+2  0      = −  , (3.12)  . . . . .       ......        λk+2  0         ......   .   .   . . . . .   .   .         .   .   .   .. −A 0 IG   .   .               |      Q −I 0 A   xN   0        PN −I λN 0 | {z } | {z } | {z } Dlqr zlqr blqr

−1 2(N+1)nx×2(N+1)nx where G = BR B|. The dimensions are Dlqr ∈ R and zlqr, blqr ∈

R2(N+1)nx . We will show that (3.12) is the general linear system of equations that applies for either the LQR, the LQ tracking problem or the Model Predictive Control (MPC), and that its factorization entails the solution of such methods.

Remark 3.1.1. From a practical viewpoint, the solution of system (3.12) for the case of the LQR (or LQ tracking problem) lacks value and usually there is no need to obtain it explicitly, since the main objective of this control strategy is to use the state feedback control law already derived in (3.9). On the contrary, for MPC, system (3.12) becomes relevant as it does need to be computed explicitly at every sample time, as we will show in chapter 6.

3.2 Solution via Riccati equation

To solve the ﬁnite-horizon problem, we can ﬁnd a sequence of matrices Pk from k = 0 ...N. It is known that such a sequence represents in fact the solution to the Riccati equation.

To show this, the control input (3.9) is substituted into the state equation of (3.1)

| −1 | xk+1 = A − B (B Pk+1B + R) B Pk+1A xk. (3.13)

32 Since λk = Pkxk, substituting (3.13) into the costate equation of system (3.3) gives

| | −1 | Pkxk = Qxk + A Pk+1 A − B (B Pk+1B + R) B Pk+1A xk.

From this relation, it is easy to see the backward recursion structure of matrix Pk, i.e.

| | | −1 | Pk = Q + A Pk+1A − A Pk+1B (R + B Pk+1B) B Pk+1A. (3.14)

Therefore the sequence of matrices Pk is found by carrying out a backward recursion of

(3.14) with the initial parameter PN , which is the weighting matrix of the terminal condition in (3.1). The matrix quadratic sequence Pk is the well-known discrete time-varying

Riccati equation. It is easy to observe that Pk just depends on given parameters, neither the state nor the costate (there is no need to compute them in real-time applications).

Therefore, it can be computed oﬄine and stored in the computer memory ready to be used in the online controller.

3.2.1 Derivation via linear system of equations

Another way to derive the Riccati equation is to factorize the system Dlqrzlqr = blqr (3.12).

For this case, we substitute the ﬁnal condition in the system, λN = PN xN , into the subsystem of xN−1 and λN−1, namely

        A 0 x IG x    N−1    N      =     , −QI λN−1 0 A| λN or

−1 xN = (GPN + I) AxN−1, and

| −1 λN−1 = QN−1xN−1 + A PN (GPN + I) AxN−1.

−1 So, if we deﬁne PN−1 = QN−1 + A|PN (GPN + I) A

λN−1 = PN−1xN−1.

33 This relation shows that it is possible to factorize the system (3.12) by computing a sequence of matrices

−1 | −1 | Pk = Q + A Pk+1 I + BR B Pk+1 A, (3.15)

from k = N − 1 ... 0. Hence, once P0 is computed, λ0 = P0x0 can be found triggering a forward substitution to solve the system. Ultimately, if the Sherman-Morrison-Woodbury formula [47] is implemented on (3.15), then evidently

| | | −1 | Pk = Q + A Pk+1A − A Pk+1B (R + B Pk+1B) B Pk+1A, which is nothing but the Riccati equation derived in (3.14).

3.3 Solution via eigenvalue decomposition

This method was proposed by Vaughan [37]. Firstly, we deﬁne a matrix

  0 I   J =   , −I 0 then, it can be shown that

| HmJ Hm = J . (3.16)

A matrix Hm (3.10) satisfying (3.16) is called symplectic. This means that if µ ∈ µ(Hm), where µ(Hm) is the set of distinct eigenvalues of Hm, then 1/µ ∈ µ(Hm) [48, p.420-421], i.e.

        u u υ 1 υ     |     Hm   = µ   ⇒ Hm   =   . υ υ −u µ −u

Assuming that Hm is diagonalizable (semisimple eigenvalues), it can be expressed in terms of the following transformation

−1 Hm = VDV , (3.17)

34 where

      M V11 V12 V¯11 V¯12 D =   , V =   , V−1 =   ,  −1     M V21 V22 V¯21 V¯22 where M ∈ Cnx is diagonal and has the eigenvalues outside the unit circle (the unstable ones). Thus, M −1 ∈ Cnx has the stable eigenvalues (inside the unit circle) and columns of V ∈ C2nx×2nx are the eigenvectors. For simplicity, it is assumed that the eigenvalues of

Hm are distinct. We carry out the following change of variable

    x w  i  i   = V   , (3.18) λi yi in system (3.4) by using transformation (3.17)

    w w  k  k+1   = D   . (3.19) yk yk+1

The solution of (3.19) is easy to obtain since D is diagonal. In terms of the ﬁnal conditions the solution is

      N−k wk M wN   =     . (3.20)    −(N−k)   yk M yN

This solution is ill-conditioned when N − k → ∞, therefore rewriting (3.20)

      −(N−k) wN M wk   =     . (3.21)    −(N−k)   yk M yN

Recalling ﬁnal condition (3.7), its relation with (3.19) at the ﬁnal time N is

V21wN + V22yN = P (V11wN + V12yN ) , (3.22)

solving for yN in terms of wN

yN = T wN , (3.23)

35 where

−1 T = − (V22 − PV12) (V21 − PV11) . (3.24)

Now, using (3.21)

yk = Tkwk, (3.25) where

−(N−k) −(N−k) Tk = M TM . (3.26)

Next step is to use xk from (3.19) and (3.25)

xk = V11wk + V12yk = (V11 + V12Tk) wk, (3.27) or

−1 wk = (V11 + V12Tk) xk. (3.28)

Substituting (3.28) into λk from (3.19)

−1 λk = (V21 + V22Tk)(V11 + V12Tk) xk, (3.29) or

λk = Pkxk, (3.30) where

−1 Pk = (V21 + V22Tk)(V11 + V12Tk) . (3.31)

Equation (3.31) is an equivalent expression of the discrete time-varying Riccati equation shown in (3.14) to solve the ﬁnite-horizon LQR problem (3.1). Algorithm 3.1 shows the procedure to solve a time-invariant closed-loop linear system,

36 Algorithm 3.1 Solution of LQR with Vaughan’s method −1 1: Hm = VDV . EV Decomposition −1 2: T = − (V22 − PV12) (V21 − PV11)

*Solution of the system given x0:

3: for k = 0 to N − 1 do

−(N−(k+1)) −(N−(k+1)) 4: Tk+1 = M TM −1 5: Pk+1 = (V21 + V22Tk+1)(V11 + V12Tk+1) −1 6: uk = − (B|Pk+1B + R) B|Pk+1Axk

7: xk+1 = Axk + Buk

8: end for

3.3.1 Receding horizon control implementation

To pave the way for Model Predictive Control (MPC), the receding horizon control (RHC) idea will be implemented on the previous eigendecomposition approach. Roughly speaking, the RHC idea ([54] and [55]) is the theoretical framework for MPC (heretofore still unconstrained) which only requires u0 of the sequence U computed for the ﬁnite-horizon LQR problem. Therefore, for time-invariant systems, the eigenvalue decomposition is computed only for u0 and with a computational load independent of the horizon. Algorithm 3.2 contemplates this, and is proposed by Poupard & Heath [38].

Algorithm 3.2 Non-recursive for unconstrained free-ﬁnal state MPC −1 1: Hm = VDV . EV Decomposition −1 2: T = − (V22 − PV12) (V21 − PV11)

−(N−1) −(N−1) 3: T1 = M TM −1 4: P1 = (V21 + V22T1)(V11 + V12T1) *Optimal control input for RHC:

−1 5: u0 = − (B|P1B + R) B|P1Ax0 . x0 given

Conversely, for computing u0 using the Riccati equation (3.14) a backward recursion

3 is required with a rough of computational cost of O Nnx . Whereas with Algorithm 3.2, one only has to set k = 0 and compute once (3.26), (3.31) and then u0 (3.9) at the expense of the eigenvalue decomposition (3.17). The decomposition of Hm only depends on the number of states nx, as shown in the coeﬃcient matrix of (3.4). The diﬀerence between the algorithm’s computational cost of both approaches is shown in Figure 3.1. For this

37 ×10 -3 7

3 Time (sec)

0 0 50 100 150 200 Prediction Horizon (N)

Figure 3.1: Computation of gain matrix P1 for the receding horizon control on the LQR. Comparison between the Riccati equation (red) and the eigendecomposition approach from Algorithm 3.2 (blue). simulation the specs of the computer used are ubuntu 16.04 LTS 64-bit operating system and Intel®Core i7-4770CPU -3.40GHz processor. This ﬁgure shows clearly that with the eigenvalue decomposition approach (in blue) it is possible to drop the horizon N out of the computational cost as opposed to that of the Riccati equation (in red), which is linearly dependent on N.

3.3.2 The case of A being singular

In real-time applications there are some systems where matrix A is singular, for instance, systems with time delays. However, solving the LQR with the eigenvalue decomposition method depends explicitly upon matrix A being non-singular. The workaround for this problem is to use the generalized eigenvalue decomposition and deflating spaces [56]. The procedure on how to compute this for the finite and infinite horizon frameworks were presented in [57] and [58] respectively. In this part, we briefly present the details of these

38 works. We ﬁrst use the Hamiltonian system (3.11), i.e.

    x x  k  k+1 C   = D   , (3.32) λk λk+1 where

    A 0 IBR−1B|     C =   ,D =   . −QI 0 A|

When matrix C is singular (which is the same as to state that A is singular) it means that some of the unstable eigenvalues of matrix M (3.17) lie at inﬁnity. To deal with this, in

[57], the generalized eigenvalue problem

Cν = µDν, (3.33) and its reciprocal eigenvalue problem

γCω = Dω, (3.34) are considered. The zero eigenvalues of (3.34) are the inﬁnite eigenvalues of (3.33). It is shown that if only the stable eigenvalues of both problems are used, an independent set of eigenvectors can be built and applied instead of the eigenvalue decomposition (3.17) that explicitly needs the inverse of A. Hence, the equivalent eigenvalue decompositions are

CV = DV J, and

CWJ = DW,

where the columns of V ∈ C2nx×nx are the eigenvectors of (3.33) and the columns of

W ∈ C2nx×nx are the eigenvectors of (3.34). Matrix J ∈ Cnx×nx is in Jordan canonical form that contains all the stable eigenvalues, i.e. ν = νi, |νi| < 1 and ν = 0. With V and W a set of independent eigenvectors can be deﬁned as Z = (V,W ), which constitutes a

39 basis for C2nx . We are now able to deﬁne an equivalent change of variable of (3.18)

    x w  i  i   = Z   , (3.35) λi yi

  V W  1 1 where Z =   . Now it is possible to redeﬁne (3.32) V2 W2

    w w  k  k+1 CZ   = DZ   , yk yk+1 or

    w w  k  k+1 [DV J, CW ]   − [DV,CWJ]   = 0, yk yk+1 and rearranging

        J 0 w I 0 w    k    k+1 [DV,CW ]     −     = 0. (3.36) 0 I yk 0 J yk+1

Term [DV,CW ] is invertible since the pair (V,W ) forms a 2nx-dimensional space independent of the pair (D,C). Therefore, the second term of the left hand side of (3.36) is zero and its solution (from k = 0,...,N), which is equivalent to (3.21), can be expressed as

      (N−k) wN J wk   =     . (3.37)    (N−k)   yk J yN

With system (3.37) it is now possible to derive the eigenvalue decomposition method when matrix A is singular just by following the procedure from relation (3.22) to (3.31). Thus

−1 Pk = (V2 + W2Tk)(V1 + W1Tk) , (3.38)

−1 (N−k) (N−k) where T = − (W2 − PW1) (V2 − PV1) and Tk = J TJ is the matrix sequence that solves the ﬁnite-horizon LQR problem when matrix A is singular, as opposed to

(3.31).

40 Chapter 4

LQ Tracking Problem

An important control strategy is to make a given system follow a desired trajectory. We

ﬁrst present a solution to the LQ tracking problem using the Riccati equation, and then we develop a novel extension to Vaughan’s method presented in section 3.3.

4.1 Finite-horizon formulation

The following results are standard [36, p.190-194]. The mathematical description of the

LQ tracking problem is as follows

1 r | r min (xN − xN ) PN (xN − xN ) X ,U 2 N−1 1 X + (x − xr )| Q (x − xr ) + u|Ru 2 k k k k k k k=0

s.t. xk+1 = Axk + Buk, x0 = x(0). (4.1) where xr is the reference signal from k = 0,...,N. The Hamiltonian function of (4.1) is

1 Hk = ((x − xr )| Q (x − xr ) + u|Ru ) + λ| (Ax + Bu ) . (4.2) 2 k k k k i i k+1 k k

Then, by using necessary conditions deﬁned in (2.20) and (2.19), we can obtain the Hamil- tonian system corresponding to the LQ tracking problem

        x A −BR−1B| x 0  k+1    k    r   =     +   xk. (4.3) λk QA| λk+1 −Q

41 It is convenient that system (4.3) is expressed as a backward recursion

        x A−1 A−1BR−1B| x 0 k k+1 r   =     +   xk. (4.4)    −1 −1 −1      λk QA A| + QA BR B| λk+1 −Q

The boundary condition (3.7) for this problem is

r λN = PN (xN − xN ) . (4.5)

As in section 3.1, to solve this problem it is necessary to use the sweep method presented in Bryson & Ho [44, p. 47]. For the LQ tracking problem, in addition to the sequence of matrices n × nx, Pk, it is also needed a sequence of vectors nx, πk. This combination makes the ﬁnal condition (4.5) hold for all the times k ≤ N, i.e.

λk = Pkxk − πk. (4.6)

We apply (4.6) to the control input (3.6) so as to obtain a state feedback controller. To achieve this, we substitute the dynamical system of (3.1) and solve for uk in (3.6), i.e.

| −1 | uk = − (B Pk+1B + R) B (Pk+1Axk − πk+1) . (4.7)

To obtain the sequence of Pk and πk, two diﬀerent methods are presented in the next two sections respectively.

4.1.1 Linear system of equations

As in section 3.1.1, the LQ tracking problem can also be expressed in the form of a general linear system of equations. If we follow the same steps, the system (4.4) can be represented as

          A 0 x IBR−1B| x 0    k    k+1   r     =     +   xk. (4.8) −QI λk 0 A| λk+1 −Q

42 Therefore, by including the initial x0 and ﬁnal condition (4.5) and (4.8) we have

      −I xk x0             −A 0 IG   λk   0               Q −I 0 A|  x   x¯r     k+1  k               −A 0 IG  λk+1  0         ..     r   Q −I 0 A| .  xk+2  x¯      = −  k+1 , (4.9)  . . . . .       ......        λk+2  0         ......   .   .   . . . . .   .   .         .   .   .   .. −A 0 IG   .   .               |    r   Q −I 0 A   xN  x¯N−1       r PN −I λN x¯N | {z } | {z } | {z } Dlqt zlqt blqt

r r r r wherex ¯i = Qxi for k = 0 ...N − 1 andx ¯N = −PN xN . The left hand side of (4.9) is the same as the LQR system (3.12), i.e. Dlqt = Dlqr and zlqt = zlqr. The only part that is diﬀerent is the right hand side blqt, which contains the reference trajectory of the states.

4.2 Solution via Riccati equation

The solution of the LQ tracking problem via the Riccati equation follows the same procedure as in section 3.2. The control input (4.7) is substituted into the state equation of

(4.1)

| −1 | | −1 | xk+1 = Axk − (B Pk+1B + R) B Pk+1Axk + (B Pk+1B + R) B πk+1. (4.10)

Since λk = Pkxk − πk (4.6), substituting (4.10) into the costate equation of system (4.3) gives

| | −1 | Pkxk − πk = Qxk + A Pk Axk − (B Pk+1B + R) B Pk+1Axk

| | −1 | | r + A Pk+1 (R + B Pk+1B) B πk+1 − A πk+1 − Qxk, (4.11)

43 or rearranging

| | | −1 | −Pk + Q + A Pk+1A − A Pk+1B (R + B Pk+1B) B Pk+1A xk | | −1 | | r + πk + A Pk+1 (R + B Pk+1B) B πk+1 − A πk+1 − Qxk = 0. (4.12)

In order for (4.12) to hold for all the states xk, it is clear that the two sequences within parentheses must vanish, namely

| | | −1 | Pk = Q + A Pk+1A − A Pk+1B (R + B Pk+1B) B Pk+1A, (4.13) and

r | | | −1 | πk = Qxk + A − A Pk+1B (R + B Pk+1B) B πk+1. (4.14)

As with the LQR analysis, the matrix sequence Pk is the discrete time-varying Riccati equation. The only diﬀerence is the vector sequence πk, which accounts for the reference

r trajectory xk of the LQ tracking problem. Both sequences can be computed independently of the states or costates with a backward recursion, starting with Pk+1 = PN and πk+1 =

r PN xN .

4.2.1 Derivation via linear system of equations

As it was shown in subsection 3.2.1, another way to derive the Riccati equation is to factorize the system Dlqtzlqt = blqt (4.9). The procedure is similar, but there is an additional sequence of vectors to account for the right hand side. For this case, we substitute the

r ﬁnal condition in the system, λN = PN (xN − xN ), into the sub-system of xN−1 and λN−1, namely

          A 0 x IG x 0    N−1    N        =     +   , | r −QI λN−1 0 A λN x¯N−1 or

−1 −1 | −1 | r xN = I + BR B PN AxN−1 + BR B PN xN ,

44 and therefore

−1 | −1 | λN−1 = A PN I + BR B PN A + Q xN−1

−1 r | r | −1 | −1 | r − QxN−1 + A PN xN − A PN I + BR B PN BR B PN xN , or

λN−1 = PN−1xN−1 − πN−1.

This relation (which is in the form of the ﬁnal condition (4.6)) shows that is possible to factorize the system (4.9) by carrying out a backward recursion of matrices

−1 | −1 | Pk = A Pk+1 I + BR B Pk+1 A + Q , (4.15) and the vectors

−1 r | | −1 | −1 | πk = Qxk + A πk+1 − A Pk+1 I + BR B Pk+1 BR B πk+1 , (4.16)

down to P0 and π0 in order to be able to compute λ0 = P0x0 − π0, thus triggering a forward substitution that will solve the problem. Finally, as in subsection 3.2.1, if the

Sherman-Morrison-Woodbury formula [47] is applied in (4.15), the Riccati equation (3.14) can be derived. For the vector sequence (4.15), a few steps are needed to

r | πk = Qxk + A πk+1

| | −1 | −1 | | −1 | − A Pk+1B (R + B Pk+1B) (R + B Pk+1B) R B − B Pk+1BR B πk+1, then

r | | | −1 | πk = Qxk + A πk+1 − A Pk+1B (R + B Pk+1B) B πk+1, which is in fact (4.14).

45 4.3 Solution via eigenvalue decomposition

The method presented in this section is proposed in Poupard & Heath [38]. First, we will solve the ﬁnite-horizon LQ tracking problem (4.1) following the same procedure presented in section 3.3, and then the receding horizon idea will follow. In this regard, the eigenvalue decomposition of Hm (3.17) still applies for this problem; and with the change of variable (3.18), system (4.4) becomes

    w w  k  k+1 r   = D   + Gxk, (4.17) yk yk+1 where

    Gw 0   −1   G =   = V   . (4.18) Gy −Q

Therefore, the solution of (4.17) is

      N−k wk M wN   =     + Sk:N−1, (4.19)    −(N−k)   yk M yN where

N−1 X N−j−1 r Sk:N−1 = D GxN−j+(k−1), (4.20) j=k is the forcing function of the solution. As shown in (3.21), this solution (4.19) is ill- conditioned when N − k → ∞. Hence

        −(N−k) w wN M wk −S   =     +  k:N−1 , (4.21)    −(N−k)    y  yk M yN Sk:N−1

w y Rnx where Sk:N−1, Sk:N−1 ∈ are

N−1 w X k−j−1 w r Sk:N−1 = M G xN−j+(k−1), j=k N−1 y X −(N−j−1) y r Sk:N−1 = M G xN−j+(k−1). j=k

46 The ﬁnal condition (4.5) and its relation with (4.17) at the ﬁnal time N is

r V21wN + V22yN = P (V11wN + V12yN ) − P xN . (4.22)

Solving for yN in terms of wN

−1 −1 r yN = − Tw Ty wN − Tw P xN , (4.23) where

Tw = (V22 − PV12) ,

Ty = (V21 − PV11) . (4.24)

Using (4.21) and after a few steps we obtain

yk = Tkwk + υk, (4.25) where

−(N−k) Tk = T¯kM , (4.26)

¯ w −(N−k) −1 r y υk = −TkSk:N−1 − M Tw P xN + Sk:N−1, (4.27)

¯ −(N−k) −1 where Tk = −M Tw Ty . The next step is to use xk from (4.17) and (4.25)

xk = (V11 + V12Tk) wk + V12υk, (4.28) or

x −1 x wk = (Tk ) (xk − υk ) , (4.29) where

x Tk = (V11 + V12Tk) , (4.30)

x υk = V12υk. (4.31)

47 To ﬁnd λk we use wk from (4.29)

λ x −1 x λ λk = Tk (Tk ) (xk − υk ) + υk , (4.32) where

λ Tk = (V21 + V22Tk) , (4.33)

λ υk = V22υk. (4.34)

We are now able to ﬁnd the optimal control input by substituting (4.32) into (3.6), i.e.

h i −1 | λ x −1 x λ uk = −R B Tk+1 Tk+1 xk+1 − υk+1 + υk+1 . (4.35)

However, input (4.35) is in terms of xk+1, which could be somehow inconvenient. To leave it in terms of xk, we substitute the dynamical system of (3.1), and after a few steps uk becomes

| −1 | uk = − (B Tk+1B + R) B [Tk+1Axk + πk+1], (4.36) where

λ x −1 Tk+1 = Tk+1 Tk+1 , (4.37)

x λ πk = −Tkυk + υk . (4.38)

Matrix sequence (4.37) is the same as that of the LQR problem deduced in (3.31) and it is equivalent to the Riccati equation. The vector sequence (4.38) is caused by the forcing function (4.20), which contains the reference values of the states. Equation (4.36) is the optimal control input for the LQ tracking problem obtained by the computation of the eigenvalue decomposition. Algorithm 4.1 recapitulates this procedure applied to a time-invariant closed-loop linear system,

48 Algorithm 4.1 Eigenvalue decomposition for LQ tracking −1 1: Hm = VDV ; . EV Decomposition

r 2: *Set: Tk+1 = P and πk+1 = P xN

3: for k = N − 1 to 0 do . Off-line Computation

4: Twk = (V22 − Tk+1V12), Tyk = (V21 − Tk+1V11);

5: T = −M −1 T −1T M −1; k wk yk x λ 6: Tk = (V11 + V12Tk), Tk = (V21 + V22Tk); λ x −1 7: Tk = Tk (Tk ) . Gain Sequence w w r y y r 8: Sk = G xk, Sk = G xk; y 9: υ = −T Sw − M −1 T −1π + S ; k k k wk k+1 k x λ 10: υk = V12υk, υk = V22υk; x λ 11: πk = −Tkυk + υk ; . Residual

12: end for

*Solution of the system given x0:

13: for k = 0 to N − 1 do . On-line Computation

−1 14: uk = − (B|Tk+1B + R) B|[Tk+1Axk + πk+1]

15: xk+1 = Axk + Buk;

16: end for

Algorithm 4.1 presents a novel approach to solve LQ tracking problems, as the one encountered in [36]. The main diﬀerence lies in how Tk and πk are computed. In algorithm

4.1, Tk is computed along with the control input uk−1 with no dependency on the horizon

N due to the eigenvalue decomposition. With the known method, Tk is computed with the

Riccati equation backwards in time. The term πk is computed similarly in both approaches with a forward recursion.

4.3.1 Receding horizon control implementation

We implement the receding horizon idea on the unconstrained MPC tracking problem, where solely u0 is needed, as we did in section 3.3.1 for the LQR.

49 Algorithm 4.2 Eigenvalue decomposition for unconstrained MPC tracking problem −1 1: Hm = VDV . EV Decomposition

2: Tw = (V22 − PV12), Ty = (V21 − PV11); ¯ −(N−1) −1 3: T1 = −M Tw Ty ; −(N−1) 4: T1 = T¯1M ;

x λ 5: T1 = (V11 + V12T1), T1 = (V21 + V22T1); λ x −1 6: T1 = T1 (T1 ) ; . Gain Sequence w PN−1 −j w r 7: S1:N−1 = j=1 M G xN−j; y PN−1 −(N−j−1) y r 8: S1:N−1 = j=1 M G xN−j; ¯ w −(N−1) −1 r y 9: υ1 = −T1S1:N−1 − M Tw P xN + S1:N−1; x λ 10: υ1 = V12υ1, υ1 = V22υ1; x λ 11: π1 = −T1υ1 + υ1 ; . Residual *Optimal control input for RHC:

−1 12: u0 = − (B|T1B + R) B| [T1Ax0 + π1];

Algorithm 4.2 is used for the unconstrained MPC tracking problem (setting k = 0).

Similar to algorithm 4.1, the computational cost of the eigenvalue decomposition Hm depends only on the states nx. The only terms that depend on the horizon N are T1 (on the diagonal matrix M, steps 3 and 4) and the residual υ1 (as a recursive procedure). The diﬀerence in both approaches, between the algorithm’s computational cost of the matrix sequence Tk and the vector sequence πk, is shown in Figure 3.1. For this simulation the specs of the computer used are ubuntu 16.04 LTS 64-bit operating system and Intel®Core i7-4770CPU -3.40GHz processor. Figure 4.1a shows clearly that with the eigenvalue decomposition approach (in blue), it is possible to drop the horizon

N out of the computational cost as opposed to that of the Riccati equation (in red), which is linearly dependent on N. With respect to the π1 computation (Figure 4.1b), the computational cost of the eigendecomposition algorithm is almost halved compared to that of the Riccati algorithm. This is due to the diagonal matrix M, which contains the eigenvalues of the system, and this structure allows to have vector-vector operations.

50 0.012

0.01

0.008

0.006 Time (sec) 0.004

0.002

0 0 50 100 150 200 Prediction Horizon (N)

(a) Computational cost of computing P1 for eigenvalue decomposition (blue) and Riccati (red).

×10 -3 1.4

1.2

0.8

0.6 Time (sec)

0.4

0.2

0 0 50 100 150 200 Prediction Horizon (N)

(b) Computational cost of computing π1 for eigenvalue decomposition (blue) and Riccati (red).

Figure 4.1: Computation of gain matrix P1 and gain vector π1 for the receding horizon control on the LQ tracking problem. Comparison between the Riccati equation (red) and the eigendecomposition approach from Algorithm 4.2 (blue).

51 Part II

Constrained Optimal Control Problem

52 Chapter 5

Model Predictive Control Framework

Model Predictive Control (MPC) is an advanced control technique [4] used in a wide variety of industrial applications such as chemical reactors, distillation columns, converters and inverters for electrical motors, pick-and-place systems, among many others.

Loosely speaking, the MPC algorithm solves an optimization problem at every sample time in which a quadratic objective function is used, subject to equality and inequality constraints. The equality constraints represent the mathematical model of the given application, while the inequality constraints represent the limitations of the states (e.g. the safe limits of temperature or pressure in a vessel) and inputs (e.g. mechanical limit of a process valve).

In addition to the natural mutlivariable structure of MPC, what sets apart MPC in industry from other more classical control techniques, such as PID controllers, is the possibility of dealing with inequality constraints at the states or inputs.

This chapter is organized in two sections. The ﬁrst section presents the connection between

MPC and the linear quadratic regulator (LQR) and the foundations of MPC. The second section presents the basic principles of one of the numerical methods, called interior-point method (IPM), used to solve the optimization problem of MPC.

53 5.1 Fundamentals

The fudamental difference between MPC and LQR has to do with the necessary conditions for optimality presented in subsection 2.1.2. On the one hand, with the LQR these conditions produce a linear system of equations. On the other hand, with MPC this system is nonlinear. Computationally speaking, this means that with the LQR it is possible to obtain a closed form solution that is computed entirely offline; whilst with MPC, since the equations are nonlinear, a numerical method has to be embedded into the online algorithm controller. In fact, in the absence of inequality constraints the entire MPC formulation boils down to Algorithm 4.2, presented in subsection 4.3.1, which requires one offline computation for the matrix sequence T1 and one online computation for the vector sequence

π1 that accounts for the reference residual. We first define the constrained free-final state linear MPC problem as follows

N−1 1 | 1 X | | min x P xN + (xi Qxi + ui Rui) X ,U 2 N 2 i=0

s.t. xi+1 = Axi + Bui, x0 = x(0)

xi ∈ X , ui ∈ U, i = 0,...,N − 1, (5.1) where the sets X ⊆ Rnx and U ⊆ Rnu are polyhedra. This could be represented as inequality constraints defined by minimum and maximum values in the states (xlow, xhigh) and inputs (ulow, uhigh). All remaining parts were already defined in (3.1). Therefore we can now generalize (5.1) into a convex quadratic problem that is defined as follows [45,

Sect. 4.4]

1 min f(θ) = θ|Hθ + g|θ θ 2 s.t. Fθ = b

Cθ ≤ d, (5.2)

Rn Rn×n Rm×n where θ ∈ is deﬁned as a decision vector, the Hessian matrix H ∈ + , F ∈ and C ∈ Rp×n. The overall aim is to ﬁnd a decision vector θ which minimizes the quadratic performance index, f(θ), subject to both constraints.

54 5.1.1 Sparse formulation

There basically exist two formulations for structuring problem (5.1) into the general form

(5.2):

Dense MPC. In this formulation, the decision vector θ contains solely the inputs,

which entails having a dense Hessian matrix H, i.e. most of its elements are non-zero.

Sparse MPC. In this formulation, the decision vector θ contains the states and the

inputs, which entails having a sparse Hessian matrix H, i.e. most of its elements are

zero, whilst its unique non-zero elements are conﬁned to the diagonal band of the

matrix.

The latter will be chosen in our approach, since the sparse formulation gives us some ad-

3 vantages. For instance, the computational cost of the optimization algorithm is O Nnx , 3 as opposed to O (Nnx) with the dense method. Having said this, the deﬁnition of the problem is

*Decision Vector:

| | | | | | | θ = [x0, x1, ..., xN ; u0, u1, ..., uN−1] . (5.3)

Dimension: θ ∈ R[(N+1)nx+Nnu].

*Performance Index:

    Q ··· 0 0 0 ··· 0 0      ......  .  ......  .          0 ··· Q 0 0 ··· 0  0             H =  0 ··· 0 P 0 ··· 0  , g = 0 .          0 ··· 0 0 R ··· 0  0      ......  .  ......  .  . . . . .  .     0 ··· 0 0 0 ...R 0

55 Dimensions: H ∈ R[(N+1)nx+Nnu]×[(N+1)nx+Nnu], g ∈ R[(N+1)nx+Nnu].

*Equality constraints:

    −I 0 0 ··· 0 0 0 ··· 0 x0          A −I 0 ··· 0 B 0 ··· 0   0          F =  0 A −I ··· 0 0 B ··· 0  , b =  0  .      ......   .   ......   .   ......   .      0 0 ··· A −I 0 0 ··· B 0

Dimensions: F ∈ R(N+1)nx×[(N+1)nx+Nnu], b ∈ R(N+1)nx .

*Inequality constraints:

For simplicity, considering time-invariant constraints

  υx    .     .    Ω ··· 0 0 Φ ··· 0     υ   ......   x  ......        C =   , d = υx ,     0 ··· Ω 0 0 ··· Φ     υu   0 ··· 0 Ω 0 ··· 0  .   .   .    υu where

        −I −I −x −u  nx   nu   low  low Ω =   , Φ =   , υx =   , υu =   . (5.4) Inx Inu xhigh uhigh

Dimensions:

Rnx×nx R2nx×nx Rnu×nu R2nu×nu R2nx R2nu Inx ∈ ,Ω ∈ , Inu ∈ ,Φ ∈ , υx ∈ , υu ∈ ,

C ∈ R2(N+1)(nx+nu)×[(N+1)nx+Nnu], d ∈ R2(N+1)(nx+nu).

56 5.1.2 Necessary conditions

The necessary conditions for optimality of (5.2), known as Karush-Kuhn-Tucker (KKT) conditions [39, Sect. 5.6] are

Hθ + F |λ + C|µ + g = 0,

Fθ − b = 0,

Cθ − d + t = 0,

MT e = 0,

(µ, t) ≥ 0, (5.5) where λ and µ are the so-called Lagrange multipliers of the equality and inequality constraints respectively and t is a slack variable. Then M = diag(µ1, µ2, ..., µp), T = diag(t1, t2, ..., tp) and e = (1, 1, ..., 1)|.

5.2 Interior point method

The nonlinear term MT e of system (5.5) requires the use of a numerical method within the MPC algorithm. Some methods for solving this problem are:

1. Fast gradient method [59], which makes use of the search direction’s ﬁrst-order infor-

mation. Recent contributions using fast gradient on embedded systems have been im-

plemented on programmable logic controllers (PLCs) [8], and on ﬁeld-programmable

gate arrays (FPGAs) [31].

2. Active set method [60], which considers only the set of inequality constraints that

are active. An optimization solver using this method is proposed in [61] and it is

also implemented on a fast online application in [62].

3. Alternating Direction Method of Multipliers (ADMM) [63], which is an adaptation

of the Lagrangian method that aims to compute local subproblems and to update

the dual variables so that the general solution of the problem can be found. It is

specially suitable to large-scale problems found in machine learning and statistics.

For MPC, ADMM has been applied to large-scale symmetric problems [64].

4. Interior point method (IPM) [40], which is based on Newton’s step method. It solves

57 a ﬁxed-size linear system of equations at every sample, as opposed to the active set

method whose structure size can change. It also ensures convergence in polynomial

time. The replacement of inequality constraints with a barrier function for MPC

algorithms has been studied by [65], and can be used for eﬃcient online solvers [66].

5.2.1 Primal-dual interior point algorithm

Any iterative numerical method aims to find a sequence of iterates θ0, θ1, θ2, ..., θk that are expected to converge to a global or local minima (global in the case of quadratic function) until some predefined tolerance is reached (). Every iteration is defined by

θk+1 = θk + αk∆pk,

where ∆pk is called the search direction, and αk is the step length. To solve the problem using the IPM method we will slightly modify the system (5.5), i.e.

Hθ + F |λ + C|µ + g = 0,

Fθ − b = 0,

Cθ − d + t = 0,

MT e = σηe,

(µ, t) ≥ 0, (5.6) where σ ∈ [0, 1] is the centering parameter and the duality gap is deﬁned by

p 1 X η = µ t = µ|t/p. p i i i=1

The term σηe plays a stabilizing role to allow the algorithm to converge steadily towards the solution of (5.2) [40, p.36-40]. In system (5.5), iterations do pure steps which could cause some pairwise products being µiti < 0. However, system (5.6) keeps all pairwise products µiti strictly positive and decreases them to zero at the same rate (η → 0). From a geometric point of view, the trajectory follows a central trail throughout µiti space, from some initial conditions towards convergence. This is known as the central path. It allows the algorithm to take longer steps without violating boundaries of the pairwise products

µiti. Hence, one can be less conservative in choosing αk.

58 We now apply Newton’s method to the following nonlinear function

  Hθ + F |λ + C|µ + g      Fθ − b    F (θ, λ, µ, t) =   , (5.7)    Cθ − d + t    MT e − σηe where the search direction ∆p = [∆θ, ∆λ, ∆µ, ∆t]| is found by solving the following linear system of equations

J(θ, λ, µ, t)∆p = −F (θ, λ, µ, t), where J(θ, λ, µ, t) is the Jacobian matrix of F (θ, λ, µ, t). Expanding the above expression

      HF | C| 0 ∆θ rθ             F 0 0 0  ∆λ r       λ     = −   , (5.8)        C 0 0 I  ∆µ rµ       0 0 TM ∆t rt where

| | rθ = Hθ + F λ + C µ + g,

rλ = Fθ − b,

rµ = Cθ − d + t,

rt = MT e − σηe. (5.9)

There are several algorithms proposed to solve optimization problems applying the interior point method. Here, we will use the so-called Mehrotra’s predictor-corrector algorithm

[41], which has been successfully proven in practice and is regarded as a very eﬃcient algorithm.

5.2.2 Mehrotra’s predictor-corrector algorithm

This algorithm has been a front runner in practical implementations due to the fact that it improves the convergence of the iterative method by introducing a corrector part and some heuristics that are backed up by years of computational experience [40]. Algorithm

59 5.1 shows the main steps, and it can be summarized as follows

1. The initial conditions of θ and λ are given and can be infeasible provided that

(µ, t) ≥ 0, and the main duality gap η is computed.

2. In the predictor part, the ﬁrst search direction ∆ppr is obtained by solving the following

      | | HF C 0 ∆θpr rθpr             F 0 0 0  ∆λpr r       λpr      = −   , (5.10)  C 0 0 I  ∆µ  r     pr  µpr        0 0 TM ∆tpr rtpr

where

| | rθpr = Hθ + F λ + C µ + g,

rλpr = Fθ − b,

rµpr = Cθ − d + t,

rtpr = MT e, (5.11)

and then the partial step length αpr that satisﬁes (µ, t) ≥ 0 is evaluated.

3. The duality gap corresponding to the predictor part ηpr is computed. Next, we

evaluate the centering parameter σ, based heuristically on the two duality gaps, ηpr and η. This allows one to obtain σ adaptively, which means that if the predictor

part gives a large step (good progress) only a small value of the centering parameter

is needed (σ 1). If the predictor part makes poor progress ((µ, t) close to limits),

a larger value of centering parameter is needed (σ close to 1) to drive iterates to the

central path.

4. The next step is to compute a combined centering-corrector step ∆pcc that will try to compensate the deviation of linearity caused in the predictor part. The system is

      | | HF C 0 ∆θcc rθcc             F 0 0 0  ∆λcc r       λcc      = −   , (5.12)  C 0 0 I  ∆µ  r     cc  µcc        0 0 TM ∆tcc rtcc

60 where

rθcc = 0,

rλcc = 0,

rµcc = 0,

rtcc = ∆Mpr∆Tpre − σηe, (5.13)

and ∆M = diag(∆µpr1 , ∆µpr2 , ..., ∆µprp ), ∆T = diag(∆tpr1 , ∆tpr2 , ..., ∆tprp ). The

step length αcc (satisfying (µ, t) ≥ 0) is then used to compute the next iteration of

the decision vector θk+1.

Algorithm 5.1 Mehrotra’s Predictor-Corrector *Input: θ = [x, u, λ, µ, t]|

1: while η > do

| 2: *Duality Gap: η = µk−1tk−1/p | 3: *Step Direction (Predictor part): ∆ppr = [∆xpr, ∆upr, ∆λpr, ∆µpr, ∆tpr] −1 −∆µpr −∆tpr 4: *Step Length: αpr = max max µ , max t , 1 | 5: *Duality Gap: ηpr = (µ + αpr∆µpr) (t + αpr∆tpr) /p 3 ηpr 6: *Centering Parameter (heuristic): σ = η | 7: *Step Direction (Corrector part): ∆pcc = [∆xcc, ∆ucc, ∆λcc, ∆µcc, ∆tcc]

8: *Total Step Direction: ∆p = (∆ppr + ∆pcc) −1 −∆µ −∆t 9: *Step Length: α = max max µ , max t , 1 | 10: *New iterates: θk+1 = [xk+1, uk+1, λk+1, µk+1, tk+1] ; θk+1 = θk+1 + α∆p

11: end while

*Output: θ∗ = [x∗, u∗, λ∗, µ∗, t∗]|

The most expensive part of Algorithm 5.1 is the factorization of the linear system (5.8).

Although it appears twice in the algorithm (predictor and corrector parts), the factorization has to be done only once since the left hand side remains the same. The state of the art in the ﬁeld aims at computing the system as eﬃciently and reliably as possible. The next chapter will be headed in that direction by presenting several novel methods that address this topic.

61 Chapter 6

Newton system factorization

The factorization of system (5.8) has been extensively studied for quadratic optimization problems and specifically for Model Predictive Control (MPC). The reduction of the system by Gauss elimination allows many techniques to efficiently solve the problem. Such is the case of the normal-equations system, whose structure as a block tridiagonal matrix allows for the implementation of the Cholesky factorization [66]. The relation between the MPC structure and the linear-quadratic problem was first highlighted in a number of works such as [67], [68] and [69]. By taking advantage of the inherent optimal control structure, this research eventually led to the development of efficient methods to solve the problem. For instance, the system can be factorized by means of the Riccati recursion approach [21]. Besides presenting the fundamentals of the Riccati method, we will present in sub-section 6.3 a novel method that also takes advantage of this structure in order to factorize the system.

6.1 General system decomposition

We ﬁrst eliminate ∆µ and ∆t from system (5.8)

      HF¯ | ∆θ r¯      θ      = −   , (6.1) F 0 ∆λ rλ where the reformulated Hessian is

−1 H¯ = H + C| M−1T C, (6.2)

62 the residuals

−1 | −1 −1 r¯θ = rθ + C M T rµ − M rt , (6.3) and part of the search direction vector

−1 −1 −1 ∆µ = M T C∆θ + rµ − M rt (6.4)

−1 ∆t = − M (T ∆µ + rt) . (6.5)

The system (6.1) is in the form of the augmented system, which is symmetric but indeﬁnite

[70]. If the Model Predictive Control (MPC) problem (5.1) is implemented, the general form of the system looks as follows

      Qk −IA| ∆xk r¯x      k   . . .   .   .   ......   .   .               |      QN−1 −IA  ∆xN−1 r¯xN−1               PN −I  ∆xN   r¯x       N         R B|   ∆u   r¯   k   k   uk         .. ..   .   .   . .   .  = −  .  .              R B| ∆u  r¯   N−1   N−1  uN−1        −I   ∆λ   r     k   λk   .       ..       A B  ∆λk+1  rλk+1         . .   .   .   .. −I ..   .   .              A −IB ∆λN rλN (6.6)

The weighting matrices are

−1 | −1 Qi =Q + Ω Mxi Txi Ω,

−1 | −1 PN =P + Ω MxN TxN Ω, −1 | −1 Ri =R + Φ Mui Tui Φ. (6.7)

It was demonstrated in [21, sec. 3.2] that matrices Qi, PN and Ri hold the same properties as their counterparts Q, P and R, i.e., Qi and PN are symmetric positive semideﬁnite,

63 whereas Ri is symmetric positive deﬁnite. Additionally, the residuals are

−1 | −1 x −1 x r¯xi = rxi + Ω Mxi Txi rµi − Mxi rti , −1 | −1 u −1 u r¯ui = rui + Φ Mui Tui rµi − Mui rti ,

rλi = Axi − xi+1 + Bui, where

| | rxi = Qxi − λi + A λi+1 + Ω µxi ,

| | rui = Rui + B λi+1 + Φ µui ,

x rµi = Ωxi − υx + txi ,

u rµi = Φui − υu + tui ,

x rti = Mxi Txi e − σηe,

u rti = Mui Tui e − σηe.

This system has in essence the optimal control structure and in particular, as we will show in the next section, the structure of the LQR (3.12) and the LQ tracking problem (4.9).

6.1.1 Hamiltonian decomposition

Loosely speaking, chapter 2 shows that the two-point boundary-value problem (TPBVP) boils down to three relations: the initial condition of the states, the ﬁnal condition of co-states and the Hamiltonian system. For MPC, this structure is highlighted in Poupard

& Heath [38], and it will be laid out in this section. In order to do so, the control inputs

64 of system (6.1) are eliminated

      Q −IA| ∆x r¯x  k   k   k   . . .   .   .   ......   .   .               |       QN−1 −IA  ∆xN−1  r¯xN−1               PN −I   ∆xN   r¯x       N      = −   , (6.8) −I   ∆λ   r     k   λk   .       ..       A −Gk  ∆λk+1  −r¯λk+1         . .   .   .   .. −I ..   .   .              A −I −GN−1 ∆λN −r¯λN

−1 where Gi = BRi B|, the control inputs are

−1 | ∆ui = −Ri (B ∆λi+1 +r ¯ui ) , and the residuals

−1 r¯λi+1 = −rλi+1 + BRi r¯ui .

It is easy to see from (6.8) that the subsystem from iterations k and k + 1 is deﬁned as

          −A 0 ∆x IG ∆x r¯    k  k  k+1  λk+1      +     = −   , (6.9) | Qk −I ∆λk 0 A ∆λk+1 r¯xk which is nothing but the Hamiltonian system of the two-point boundary-value problem

(TPBVP) with the boundary conditions, ∆x0 = rλ0 and

∆λN = PN ∆xN +r ¯xN . (6.10)

65 Therefore, system (6.8) can be rearranged as

      −I ∆xk rλk             −A 0 IGk   ∆λk  r¯λ       k+1         Q −I 0 A|  ∆x   r¯   k   k+1  xk               −A 0 IGk+1  ∆λk+1 r¯λk+2         ..       Qk+1 −I 0 A| .  ∆xk+2 r¯x      = −  k+1  .  . . . . .       ......  ∆λ  r¯     k+2  λk+3         ......   .   .   . . . . .   .   .         .   .   .   .. −A 0 IG   .   .   N−1            Q −I 0 A|   ∆x  r¯   N−1   N   xN−1        PN −I ∆λN r¯xN | {z } | {z } | {z } Dmpc zmpc bmpc (6.11)

System (6.11), Dmpczmpc = bmpc, is essentially the same as the LQR system (3.12) and

2(N+1)nx×2(N+1)nx the LQ tracking problem (4.9). The dimensions are Dmpc ∈ R and

2(N+1)nx zmpc, bmpc ∈ R . The sole diﬀerence among these systems is the right hand side vector, bmpc, which in this case represents the residuals of the Newton system of the interior-point algorithm. In the last part of this chapter we will present two eﬃcient methods to factorize this system. On the one hand, a well-known method, the Riccati recursion; and on the other hand, a novel method called the eigenvalue factorization approach.

6.2 Riccati recursion method

This approach was proposed in [21], which basically factorizes the system (6.11) by using the Riccati equation plus a vector sequence computation. One of the advantages of this

3 approach is that the complexity of the algorithm is reduced to O(Nnx), as opposed to the 3 fully dense approach that is O (Nnx) . Additionally, there are other eﬃcient methods that solve directly system (6.1), e.g. using the LDL| factorization [66], which its computational cost is also linearly dependent on the horizon. The derivation of the Riccati method in

MPC comes as no surprise since the matrix of system (6.11) is the same as the LQR and the LQ tracking system, with diﬀerent right-hand vector.

66 Having said this, we have already implicitly derived this method, speciﬁcally in section

4.2.1. Hence, in this part the derivation of the method will be mainly omitted, only brief details will be given. For instance, we will emphasize the computation of the sequence of vectors, as it changes slightly with respect to that of the LQ tracking problem.

The ﬁnal condition (6.10), ∆λN = PN ∆xN +r ¯xN , of system (6.11) is substituted into the other ﬁnal condition, i.e.

−1 ∆xN = (GN−1PN + I) (A∆xN−1 +r ¯λN − GN−1r¯xN ) .

Then the ﬁnal conditions are substituted into

| QN−1∆xN−1 − ∆λN−1 + A (PN ∆xN +r ¯xN ) = −r¯xN−1 , and after some manipulations this becomes

| −1 QN−1 + A PN (GN−1PN + I) A ∆xN−1 − ∆λN−1 = | {z } PN−1

| | −1 −r¯xN−1 − A r¯xN − A PN (GN−1PN + I) (¯rλN − GN−1r¯xN ) . (6.12) | {z } πN−1

By looking at (6.12) and (6.10), it is easy to observe the backward recursion structure of

Pk and πk, namely

−1 | −1 | Pk = Qk + A Pk+1 BRk B Pk+1 + I A, (6.13)

−1 | | −1 | −1 | πk =r ¯xk + A πk+1 + A Pk+1 BRk B Pk+1 + I r¯λk+1 − BRk B πk+1 , (6.14)

where πN =r ¯xN . By implementing the Sherman-Morrison-Woodbury formula on term −1 (GkPk+1 + I) , (6.13) and (6.14) become

| | | −1 | Pk = Qk + A Pk+1A − A Pk+1B (B Pk+1B + Rk) B Pk+1A (6.15)

| | πk =r ¯xk + A πk+1 + A Pk+1rλk+1 −

| | −1 | | A Pk+1B (B Pk+1B + Rk) r¯uk + B πk+1 + B Pk+1rλk+1 . (6.16)

Equation (6.15) is the well-known Riccati equation. A backward recursion is then computed from (6.15) and (6.16), or relations (6.13) and (6.14), down to P0 and π0 in order

67 to be able to compute ∆λ0 = P0∆x0 + π0. This triggers a forward substitution that will solve system (6.11).

6.2.1 Algorithm

We will present here the algorithm that computes the Riccati-recursion method [21], used for solving the step direction in the Mehrotra’s Predictor-Corrector Algorithm 5.1.

Algorithm 6.1 Riccati recursion method for the step direction computation

1: function riccati-mpc(Qi, Ri, PN , A, B, bmpc)

2: *Residuals: r¯x,r ¯u,r ¯λ, rµ, rt; . Predictor (5.11) or Corrector (5.13)

3: *Initial conditions: Pk+1 = PN and πk+1 =r ¯xN

4: for k = N − 1 to 0 do . Factorization of (6.11)

u | −1 5: Sk+1 = (B Pk+1B + Rk) ;

x | 6: Sk+1 = A Pk+1B; x u 7: Sk+1 = Sk+1Sk+1;

8: Pk = Qk + A|Pk+1A − Sk+1B|Pk+1A; . Riccati equation

u | | 9: ρk+1 =r ¯uk + B πk+1 + B Pk+1rλk+1 ;

| | u 10: πk =r ¯xk + A πk+1 + A Pk+1rλk+1 − Sk+1ρk+1; . Vector sequence

11: end for

12: *Initial condition given: ∆x0 = rλk ;

13: ∆λ0 = P0∆x0 + π0; −1 14: −1 x −1 x ∆µx0 = Mx0 Tx0 Ω∆x0 + rµ0 − Mx0 rt0 ;

15: −1 x ∆tx0 = −Mx0 Tx0 ∆µx0 + rt0 ;

16: for k = 0 to N − 1 do . Forward substitution

u | u 17: ∆ uk = −Sk+1B Pk+1A∆xk − ρk+1

18: ∆xk+1 = A∆xk + B∆uk + rλk+1 ;

19: ∆λk+1 = Pk+1∆xk+1 + πk+1; −1 20: ∆µ = M−1 T Ω∆x + rx − M−1 rx ; xk+1 xk+1 xk+1 k+1 µk+1 xk+1 tk+1 −1 21: ∆µ = M−1T Φ∆u + ru − M−1ru ; uk uk uk k µk uk tk 22: ∆t = −M−1 T ∆µ + rx ; xk+1 xk+1 xk+1 xk+1 tk+1

23: ∆t = −M−1 T ∆µ + ru ; uk uk uk uk tk

24: end for

25: return ∆p = [∆x, ∆u, ∆λ, ∆µ, ∆t]|;

26: end function

68 6.2.2 Numerical example

In order to observe the performance and also to carry out comparison tests with the novel method (presented in the next chapter), the code for Algorithms 5.1 and 6.1 was developed in Matlab®. For Algorithm 5.1 (Mehrotra’s algorithm), the supervisory ﬂags and the stopping criteria were taken from [28]. We take a second order system example that can be found in [71]

        x 0.7326 −0.0861 x 0.0609  1k+1     1k      =     +   uk, (6.17) x2k+1 0.1722 0.9909 x2k 0.0064

with constraints −2 ≤ uk ≤ 2, Q = I, R = 0.01 and P is the solution of the inﬁnite- horizon problem. The horizon is N = 40. The objective of this example is to drive the states from some initial conditions (x0 = [1, 1]) towards the equilibrium point, which in this case is zero. The simulation gives the next results for states and input.

1.5 x 1 x 2 1

0.5

states (x) 0

-0.5

-1 0 1 2 3 4 time (s)

(a) States.

-0.5

-1 input (u)

-1.5

-2

0 1 2 3 4 time (s)

(b) Input (blue) subject to constraint (red).

Figure 6.1: Simulation with MPC controller using the Riccati recursion method.

69 6.3 Eigenvalue factorization method

Heretofore, we have demonstrated that the solution of three diﬀerent varieties (LQR (3.12),

LQ tracking (4.9) and MPC (6.11)) of the two-point boundary-value problem can be reduced to the same linear system of equations varying only on the right hand side vector. Therefore the implementation of the eigenvalue factorization presented in section 3.3 and 4.3 is trivial for MPC. We recollect the Hamiltonian system (6.9)

        ∆x A−1 A−1G ∆x r  k  k   k+1  wk    =     +   , (6.18) −1 | −1 ∆λk QkA A + QkA Gk ∆λk+1 ryk | {z } Hmk

−1 where rwk = A r¯λk+1 and ryk =r ¯xk + Qkrwk . Applying the eigenvalue decomposition (3.17) in (6.18)

      ∆w ∆w V  k  k+1  wk    = Dk   +   rk, (6.19) ∆yk ∆yk+1 Vyk

¯ ¯ ¯ ¯ where rk = [rwk , ryk ], Vwk = V11k , V12k and Vyk = V21k , V22k . In order to derive the method for MPC we follow section 4.3 step by step, more in particular from relation (4.19) to (4.38). This part considers matrix A to be non-singular. For the opposite case, the extension presented in section 3.3.2 can be easily implemented.

6.3.1 Algorithm

Algorithm 6.2 encompasses the eigenvalue factorization method to compute the step direction of the Newton method in MPC (presented in Poupard & Heath [38]).

70 Algorithm 6.2 Eigenvalue factorization method for the step direction computation

1: function eigen-mpc(Qi, Ri, PN , A, B, bmpc)

2: *Residuals: r¯x,r ¯u,r ¯λ, rµ, rt; . Predictor (5.11) or Corrector (5.13)

3: *Initial conditions: Tk+1 = PN and πk+1 =r ¯xN

4: for k = N − 1 to 0 do . Factorization of (6.11)

k −1 5: Hm = VkDkVk . Eigendecomposition (3.17)

6: Twk = (V22k − Tk+1V12k );

7: Tyk = (V21k − Tk+1V11k );

8: T = −M −1 T −1T M −1; k k wk yk k x λ 9: Tk = (V11k + V12k Tk),Tk = (V21k + V22k Tk); λ x −1 10: Tk = Tk (Tk ) ; . Matrix Sequence w ¯ ¯ y ¯ ¯ 11: Sk = V11k rwk + V12k ryk , Sk = V21k rwk + V22k ryk ; y 12: υ = −T Sw + M −1 T −1π + S ; k k k k wk k+1 k x λ 13: υk = V12k υk, υk = V22k υk; x λ 14: πk = −Tkυk + υk ; . Vector sequence

15: end for

16: *Initial condition given: ∆x0 = rλk ;

17: ∆λ0 = T0∆x0 + π0; −1 18: −1 x −1 x ∆µx0 = Mx0 Tx0 Ω∆x0 + rµ0 − Mx0 rt0 ;

19: −1 x ∆tx0 = −Mx0 Tx0 ∆µx0 + rt0 ;

20: for k = 0 to N − 1 do . Forward substitution

| −1 ¯ 1 21: ∆λk+1 = (A ) Qk∆xk + ∆λk − r¯k ;

¯−1 | 2 22: ∆uk = −Rk B ∆λk+1 +r ¯k ; 3 23: ∆xk+1 = A∆xk + B∆uk + rk+1; −1 24: ∆µ = M−1 T Ω∆x + rx − M−1 rx ; xk+1 xk+1 xk+1 k+1 µk+1 xk+1 tk+1 −1 25: ∆µ = M−1T Φ∆u + ru − M−1ru ; uk uk uk k µk uk tk 26: ∆t = −M−1 T ∆µ + rx ; xk+1 xk+1 xk+1 xk+1 tk+1

27: ∆t = −M−1 T ∆µ + ru ; uk uk uk uk tk

28: end for

29: return ∆p = [∆x, ∆u, ∆λ, ∆µ, ∆t]|;

30: end function

71 6.4 Splitting method implementation

In this section, we address some necessary modifications for the efficient implementation of Algorithm 6.2, specifically avoiding the computation of a sequence of eigenvalue decompositions (step 5) at every sample time, which would raise the computational cost

signiﬁcantly. The ﬁrst step is to decompose matrix Hmk into two parts (Poupard et al. [42])

    −1 12 12 A Hc 0 Hv H =   +  k  . (6.20) mk     H21 H22 H21 H22 c c vk vk | {z } | {z } Hc Hvk

12 The matrix Hc contains data from the system that are independent of k where Hc =

−1 21 −1 22 | −1 A G, Hc = QA and Hc = A + QA G. Matrix Hvk contains solely the variable terms related to the interior-point algorithm (i.e., the Lagrange multipliers for the inequality constraints, µ and the slack variables, t) where H12 = A−1GL, H21 = QLA−1 vk k vk k and H22 = Q A−1G − QA−1G. Regarding the weighting matrices (6.7), they are divided vk k k into

−1 | −1 Qk = Q + Ω Mxk Txk Ω,

| {zL } Qk −1 | −1 PN = P + Ω MxN TxN Ω,

| {zL } PN and the inverse of Rk gives

−1 −1 −1 −1 | −1 −1 L Rk = R + Φ Muk Tuk Φ = R + Rk ,

L where the inverse of Rk is the part that contains the variable part of Rk after applying the Sherman-Morrison-Woodbury formula. Hence

−1 −1 | L | Gk = BR B + B R B . | {z } k G | {zL } Gk

72 Therefore Dmpc from (6.11) can be also expressed as

  −I     −I 0 A−1 H12   c     0 −IH21 H22   c c     −1 12   −I 0 A Hc     21 22 ..   0 −IHc Hc .  D =   + mpc  . . . . .   ......       ......   . . . . .     .   .. −I 0 A−1 H12  c     21 22  0 −IHc Hc    P −I | {z } Mc   0     0 0 0 H12   vk    0 0 H21 H22   vk vk     0 0 0 H12   vk+1     21 22 ..   0 0 Hv Hv .   k+1 k+1  ,  . . . . .   ......       ......   . . . . .     .   .. 0 0 0 H12   vN−1     0 0 H21 H22   vN−1 vN−1    L PN 0 | {z } Mv and thus

(Mc + Mv) zmpc = bmpc. (6.21)

We will propose a classical additive splitting method in system (6.21) that can naturally exploit the decomposition of Dmpc into constant (Mc) and variable (Mv) parts. Example of splitting methods are Jacobi, Gauss-Seidel and successive over-relaxation (SOR) [72, 73].

System (6.21) can be written in the form of

Mczmpci = Mvzmpci−1 + bmpc, (6.22)

73 where zmpc0 is the starting point. Each iteration requires the factorization of Mc, which in this case, as it does not depend on k, can be computed offline. Matrix Mc has the structure of a two-point boundary-value problem (TPBVP). Its factorization can be carried out by dividing the factorization-for-loop snippet of Algorithm 6.2 in two parts, the offline and the online segments. The offline part absorbs the most expensive bit of the algorithm, which are the matrix-matrix computations; whereas the online part takes on matrix-vector computations, which accounts for the right hand side vector of the system.

6.4.1 Algorithms

The eigenvalue factorization method (section 6.3) can then be split into oﬄine and online segments for matrix Mc. The general algorithm (for time-variant systems) for the oﬄine part is

Algorithm 6.3 Eigenvalue factorization for Mc oﬄine part (time-variant systems)

1: function eigen-mpc-offline-tv(Ak, Bk, Q, R, P )

2: *Initial conditions: Tk+1 = PN

3: for k = N − 1 to 0 do . Offline factorization

k −1 4: Hm = VkDkVk . Eigendecomposition (3.17)

5: Twk = (V22k − Tk+1V12k );

6: Tyk = (V21k − Tk+1V11k );

7: T = −M −1 T −1T M −1; k k wk yk k x λ 8: Tk = (V11k + V12k Tk),Tk = (V21k + V22k Tk); λ x −1 9: Tk = Tk (Tk ) ; . Matrix Sequence

10: end for

11: return *Sequences: T , M , V , T −1; k k k wk

12: end function

For time-invariant systems, Algorithm 6.3 basically reduces to Algorithm 3.2 presented in section 3.3.1, regarding non-recursive unconstrained free-ﬁnal state MPC. The sole diﬀerence is that instead of computing for k = 1, the setting of iteration is k = 0, as we are aiming to compute ∆λ0. Namely,

74 Algorithm 6.4 Eigenvalue factorization for Mc oﬄine part (time-invariant systems) 1: function eigen-mpc-offline-ti(A, B, Q, R, P )

−1 2: Hm = VDV . Eigendecomposition (3.17)

−1 −1 3: Tw = (V22 − PV12)

4: Ty = (V21 − PV11)

−N −1 −N 5: T0 = −M Tw Ty M −1 6: T0 = (V21 + V22T0)(V11 + V12T0)

−1 7: return *Matrices: T0, M, V, Tw ;

8: end function

The online part of the factorization of Mc is required twice in the splitting algorithm,

−1 as shown in (6.22). The vector dmpc = Mc bmpc is computed only once before starting.

Vector gmpci = Mvzmpci is obtained at every splitting iteration, i, until convergence is achieved. For time-invariant systems (for time-variant the corresponding part of Algo- rithm 6.2 can be used) the algorithm is

Algorithm 6.5 Eigenvalue factorization for Mc online part (time-invariant systems) −1 1: function eigen-mpc-online-ti(T0, M, V , Tw , dmpc (or gmpc )) w y 2: *Initial conditions: S0 = 0, S0 = 0;

3: for k = 0 to N − 1 do ¯ ¯ 4: r¯wk = V11rwk + V12ryk ; ¯ ¯ 5: r¯yk = V21rwk + V22ryk ; w −(k+1) w 6: S0 = M r¯wk + S0 ; y −(N−k−1) y 7: S0 = M r¯yk + S0 ;

8: end for ¯ w −N −1 y 9: υ0 = −T0S0 + M Tw r¯xN + S0 ; x λ 10: υ0 = V12υ0, υ0 = V22υ0; x λ 11: π0 = −T0υ0 + υ0 ; . Vector at k = 0

12: return ∆λ0 = T0∆x0 + π0;

13: end function

6.4.2 Numerical experiment

An important aspect of advanced controllers that require the use of numerical methods such as MPC is the distinction between oﬄine and online computations. The former are

75 all parameters that can be introduced into the controller as data, as long as they are well balanced and do not consume a signiﬁcant amount of memory. The latter are related to the parameters that are dependent on the iteration itself and thus they need to be computed at every sample time. In this experiment we want to highlight this distinction.

To test the feasibility of this method we carry out a numerical experiment which compares the computational cost and also shows the convergence of the splitting method. The same second-order system (time-invariant) used in section 6.2.2 will be deployed here. We used a computer with ubuntu 16.04 LTS 64-bit operating system and Intel®Core i7-4770CPU

-3.40GHz processor. The ﬁrst simulation is for comparing the computational cost between diﬀerent N values using the Riccati and eigenvalue methods.

×10 -3 5

4.5

3.5

2.5

2 Time (sec)

1.5

0.5

0 0 20 40 60 80 100 Prediction Horizon (N)

(a) Eigenvalue (blue) and Riccati (red) for online factorization of matrix-matrix operations.

×10 -3 4

3.5

2.5

Time (sec) 1.5

0.5

0 0 20 40 60 80 100 Prediction Horizon (N)

(b) Eigenvalue (blue) and Riccati (red) for online factorization of matrix-vector operations.

Figure 6.2: Comparison of the computational time during the simulation of the online part of the MPC controller using the Riccati recursion and eigenvalue method.

In Figure 6.2a, Algorithm 6.1 was used for the Riccati part, from steps 5 to 8. It is

76 clear that the computational online cost is linearly dependent on the horizon N as the

Riccati equation cannot be decoupled as the Hamiltonian system (6.20). However, with the eigenvalue method, as Algorithm 6.4 is computed entirely offline, there is no online cost attributed. In addition, there exist Riccati-based methods with better computational complexity than the one used here [21], such is the case of [7] and [22]. The latter is derived for active-set solvers. Algorithm 6.3 could also have been used here, which would be offline as well, but it is not necessary as the system is time-invariant. The only difference between these algorithms is that for the time-variant case more data is created, i.e. matrix sequences of T , M , V and T −1 (where k = 0,...,N −1), as opposed to the four matrices k k k wk −1 required for the time-invariant case, T0, M, V and Tw . Figure 6.2b shows the online computation that accounts for the residuals of the Newton system (right hand side vector). For the Riccati method, steps 9 to 10 of Algorithm 6.1 were computed. For the eigenvalue method, Algorithm 6.5 was used. The computational cost of both is linearly dependent on the horizon N. The eigenvalue case differs slightly, as its cost is almost halved since some of the computations are vector-vector operations related to matrix M, which is diagonal and contains the eigenvalues of the Hamiltonian system.

Up to this point, the factorization of the Newton system is done for the Riccati method, but for the eigenvalue method another step remains to be computed, namely the system

−1 zmpci = Mc Mvzmpci−1 + dmpc, (6.23)

−1 −1 where vector dmpc = Mc bmpc and part of the factorization Mc are already available. In order to determine the potential and feasibility of the eigenvalue factorization approach, fast convergence should be guaranteed. In this sense, an experiment that applies a naive implementation of the splitting method is carried out, using data obtained by the simulation of section 6.2.2. In particular, we use the sequence of matrices Mc, Mv and vector dmpc encompassing outer iterations corresponding to the sample time ﬁxed at t = 4s; and the iterations corresponding to the Mehrotra’s algorithm, k. The test is set by forcing the splitting iterations up to i = 30, so as to observe either convergence, divergence or stagnation. Figure 6.3 shows the results of this test, where the vertical axis is the splitting residual norm after the i−iterations. The horizontal axis is the iterations related to the

Mehrotra’s algorithm.

77 10 5

10 0

10 -5

10 -10 residual norm

10 -15

10 -20 0 5 10 15 20 iterations

Figure 6.3: Convergence of a naive implementation of the splitting method on system (6.23). The number of iterations on the splitting method was set at i = 30. Each line represents a vector whose length is the number of k−iterations of the Mehrotra’s algorithm (Algorithm 5.1). The elements of the vector correspond to the residual norm after i−iterations.

It is clear from Figure 6.3 that convergence is not guaranteed for all the spectrum of iterations as there are some zones with stagnation. A more profound analysis is required to relate convergence to the structure of matrices Mc and Mv, and to determine whether or not it is bounded. Furthermore, it is known that convergence of any splitting method

−1 is related to the spectral radius of the F = Mc Mv, i.e. the largest modulus eigenvalue of that matrix [48, p. 613]. Therefore, there is potential to guarantee and improve convergence using this method should proper preconditioners be provided. Likewise, although splitting methods have been already applied to eﬃcient MPC algorithms, such as in [74] and [75], and to embedded MPC in a PLC in [76] for the augmented system (6.1); the splitting system (6.21) is novel, as it comes from the Hamiltonian system (6.8) proposed in this thesis (and published in [38]).

78 Chapter 7

Fast Hamiltonian/GMRES solver

In the previous section, we presented an approach that employs the iterative splitting method to solve the Newton system (6.11) of the interior point method (IPM). Research of iterative methods for IPM is very active, as current real-time advanced controllers require eﬃcient and rapid optimization solvers. For instance, work regarding the general quadratic programming structure can be found in [77] and [78]; and for a more inherently optimal-control type structure, such as with MPC, in [1] and [79].

In this section, we will present a method that takes advantage of the special structure of

MPC when using quadratic programming. This novel method consists of the formulation of two preconditioners (left and right), which resemble the two-point boundary-value problem structure. Multiplying them onto system (6.11) gives rise to a companion-like matrix, whose last column is computed as a forward recursive sequence. The factorization of this matrix is then used to solve the Newton system. This factorization could be carried out either by using a direct method or a Krylov iterative method (such as GMRES).

Application of the GMRES method to nonlinear MPC can be found in [80]. The following methods and results will be part of a journal paper which is currently in preparation.

79 7.1 Incomplete LU factorization

To derive the preconditioners, system (6.11) is ﬁrst slightly rearranged

      IGk 0 ∆xk+1 r¯λ      k+1         0 A| −I ∆λ   r¯     k+1  xk         −A 0 IG 0  ∆x  r¯   k+1   k+2  λk+2   .       | ..      Qk+1 −I 0 A 0  ∆λk+2 r¯xk+1         ......   .   .   ......   .  = −  .  . (7.1)        .   .   .   ......   .   .   ......   .   .         ..       . −A 0 IGN−1 0   ∆xN  r¯λ       N−1         Q −I 0 A| 0   ∆λ  r¯   N−1   N   xN−1        PN −I 0 ∆λk r¯xN | {z } | {z } | {z } Dhqr zhqr bhqr

The matrix Dhqr is of size (2N + 1)nx×(2N + 1)nx, and the vectors zhqr and bhqr are of dimension (2N + 1)nx. System (7.1) is equivalent to (6.11), diﬀering only in size due to two aspects: ﬁrst, ∆xk was eliminated since it is known (∆xk = rλk , where in fact rλk = 0); and second, the column corresponding to ∆λk was shifted to the last position.

7.1.1 Tailor-made preconditioners

Let Hhqr = HQHR be non-singular and apply it as a preconditioner of the system

Dhqrzhqr = bhqr (7.1), namely

˜ D˜hqrz˜hqr = bhqr, (7.2) where

˜ −1 −1 ˜ −1 Dhqr = HQ DhqrHR , bhqr = HQ bhqr.

Matrices HQ and HR are called the left and right preconditioners, respectively. In order to recover the original vector zhqr, system HRzhqr =z ˜hqr has to be solved. In a Krylov method, system (7.2) would converge faster than system (7.1) if matrix D˜hqr looks like the identity matrix. Therefore, in order to achieve this, the preconditioner Hhqr should meet the following criteria [48, sec. 11.5].

80 1. The structure of matrix Hhqr should resemble the original matrix Dhqr in some −1 −1 sense that Hhqr ≈ Dhqr. Hence, HQ DhqrHR will resemble the identity matrix I,

D˜hqr ≈ I.

2. Likewise, the matrices HQ and HR should be easy to factorize (like a tridiagonal matrix), so the algorithm would take on solely matrix-vector operations.

We will demonstrate that our selection of preconditioners fulﬁll the above criteria with some additional advantages:

a) The matrix D˜hqr is a companion-like matrix, which emulates the identity matrix in all its columns but the last one.

b) Due to the special structure of D˜hqr, the GMRES method is guaranteed to converge

to the exact solution in a small number of nx + 1 iterations, independent of the time horizon N.

c) The preconditioners, HQ and HR, are tridiagonal matrices, which makes their factorization trivial.

d)A matrix-free algorithm will be proposed that avoids building the matrices prior to

the iterative method.

The above is described by the following result. Let the right preconditioner HR be an upper triangular matrix

  IGk    .   I ..     . .   .. ..       .. ..   . .      HR =  IG  , (7.3)  N−2       I       IGN−1       I      I

81 and the left preconditioner HQ be a lower block-triangular matrix

  I      0 A|         −A AGk I      Qk+1 Fk+1 0 A|     . . . .  HQ =  ......  , (7.4)      ......   . . . .     ..   . −A AGN−2 I       Q F 0 A|   N−1 N−1    PN FN I

so that criterion 1 is met, where Fk+1 = − (Qk+1Gk + I) for k = 0,...,N − 1 and

FN = − (PN GN−1 + I) for k = N. This preconditioner is valid if and only if matrix A is non-singular.

In order for the preconditioner Hhqr to have the same structure as the main matrix Dhqr, subsystems HQ(1 : 2nx, 1 : 2nx)HR(1 : 2nx, 1 : 2nx) = Dhqr(1 : 2nx, 1 : 2nx) and HQ(3nx :

4nx, 1 : 2nx)HR(1 : 2nx, 1 : 2nx) = Dhqr(3nx : 4nx, 1 : 2nx) are solved, i.e.

      IG I −G I 0  k  k   HQ(1 : 2nx, 1 : 2nx) =     =   , 0 A| 0 I 0 A| and

      −A 0 I −G −A AG    k  k  HQ(3nx : 4nx, 1 : 2nx) =     =   . Qk+1 −I 0 I Qk+1 Fk+1

Due to the block-diagonal-type structure of Dhqr, the previous also applies to the subsequent subsystems down to k = N − 1. For k = N, the following subsystem is solved

HQ(2Nnx + 1 : (2N + 1)nx, 2(N − 1)nx : 2Nnx) =   I −GN−1   = , PN −I   PN FN 0 I

and then, the entire structure of HQ (7.4) is now derived.

Lemma 7.1.1. The matrix Hhqr has a factorization Hhqr = HQHR where HR and HQ

82 are deﬁned in (7.3) and (7.4), respectively

  IGk 0      0 A| 0        −A 0 IGk+1 0    ..  Qk+1 −I 0 A| . 0    ......  Hhqr =  ......  . (7.5)    .  ......   ......     ..   . −A 0 IGN−1 0      Q −I 0 A| 0  N−1    PN −II

Remark 7.1.2. The only diﬀerence between matrices Hhqr and Dhqr (7.1), lies in the last nx−block column. Matrix Dhqr has only an identity matrix, −I, lying in the second nx−block row. Matrix Hhqr has only an identity matrix, I, in the last nx−block row.

Hence, HQ and HR are in fact the L and U factors, respectively, of the incomplete LU factorization of matrix Dhqr, whose residual matrix is R = HQHR − Dhqr6=0 as opposed to a standard LU factorization in which R = 0 [72, sec. 10.3].

7.2 Companion-like matrix

Since the factorization of matrix HR (7.3) is trivial, it is easy to see that the product of −1 matrices Dhqr and HR is

  I 0      0 A| −I        −A AGk I 0      Qk+1 Fk+1 0 A| 0    −1  . . . . .  DhqrH =  ......  , (7.6) R    .   ......   . . . . .     ..   . −A AGN−2 I 0       Q F 0 A| 0   N−1 N−1    PN FN 0

83 which nearly resembles the preconditioner HQ (7.4) in all its columns except for the last one. The resulting matrix after the multiplication of the preconditioners HQ (pre- multiplied) and HR (post-multiplied) in matrix Dhqr is depicted in the following result. ˜ −1 −1 Theorem 7.2.1. Let the matrix Dhqr = HQ DhqrHR be

  IWk      IY   k       IWk+1       IYk+1    ˜  . .  Dhqr =  .. .  , (7.7)    .   .. .   . .       IWN−1      IY   N−1    ZN a companion-like matrix, then the last column is represented by the following recursive sequence from k = 1,...,N − 1

      Wk+1 −A AGk Wk   =     , (7.8)    − −    Yk+1 A |Qk+1 A |Fk+1 Yk and for k = N

ZN = PN WN−1 + FN YN−1, (7.9)

− with initial conditions Wk = 0 and Yk = −A |.

Proof. Since, matrices (7.4) and (7.6) are equivalent except for the last nx−block column; it is simple to prove the structure of (7.7), from the ﬁrst nx−block column to the second last −1 −1 nx−block column. In this sense, the product HQ DhqrHR corresponding to such column produces a diagonal of identity matrices I of size nx × nx. Regarding the last nx−block −1 column of the companion matrix, it can be observed that the only elements of HQ that will be aﬀected by the product are the ones located at the second nx−block column, since −1 the last nx−block column of DhqrHR is almost empty except for the identity matrix in

84 its second nx−block row. Namely

    C(0,2) −C(0,N)          C   −C   (1,2)   (1,N)           C(2,2)   −C(2,N)           C(3,2)   −C(3,N)       .   .   .  × −I =  . , (7.10)      .  | {z }  .   .  −1  .    DhqrH (2nx, Nnx)     R       C(N−2,2) −C(N−2,N)         C  −C   (N−1,2)  (N−1,N)     C(N,2) −C(N,N) | {z } | {z } −1 ˜ HQ (:, 2nx) Dhqr (:, Nnx)

th th where i and j, from C(i,j), are the i nx−block row and the j nx−block column respectively. −1 The next step is to derive the coeﬃcients of HQ (:, 2nx), which can be done by solving | the system HQX = B with respect to solely these columns (:, 2nx). If X = [X0,..., XN ]

| and B = [B0,..., BN ] , it derives from (7.4) that the coeﬃcients of B1 are described by the following recursion

X0 = [0] B1, |{z} C(0,2)

−| X1 = A B1, | {z } C(1,2) X2 = (A) C(0,2) − (AG0) C(1,2) B1, | {z } C(2,2)

−| −| X3 = − A Q1 C(0,2) − A F1 C(1,2) B1, | {z } C(3,2) . . . . XN = − (PN ) C(N−2,2) − (FN ) C(N−1,2) B1, | {z } C(N,2) which resembles precisely the recursive sequence of (7.8) and the structure of the companion- like matrix (7.7) is now proved.

85 7.2.1 Algorithm

The form of the companion-like matrix, deﬁned in Theorem 7.2.1, allows for the direct factorization of system (7.1), paving the way to obtain the solution of the general Newton system (5.8). This direct method is depicted in Algorithm 7.1 (see code in Appendix A).

Algorithm 7.1 Direct factorization of system (7.1) via companion-like matrix D˜hqr

1: function comp-dire-mpc-online(Qi, Ri, PN , A, B, bhqr) ˜ −1 1) Computation of bhqr = HQ bhqr: . Matrix-vector operations y 2: *Initial conditions: ˜bw = −r¯ , ˜b = −A−|r¯ ; hqrk λk+1 hqrk xk 3: for k = 0 to N − 1 do         ˜w ˜w b A −AGk b r¯λ  hqrk+1     hqrk   k+2  4:   =     −  ; ˜bw −A−|Q −A−|F ˜by r¯ hqrk+1 k+1 k+1 hqrk xk+1 5: end for

y 6: *Final condition (k=N): ˜b = −P ˜bw − F ˜b − r¯ hqrN N hqrN−1 N hqrN−1 xN 2) Recursion computation: . Matrix-matrix operations

− 7: *Initial conditions: Wk = 0, Yk = A |;

8: for k = 0 to N − 1 do       Wk+1 −A AGk Wk 9:   =    ;    − −    Yk+1 A |Qk+1 A |Fk+1 Yk 10: end for

11: *Matrix ﬁnal condition (k=N): ZN = PN WN−1 + FN YN−1; −1˜ 12: *Vector ﬁnal condition (k=N): z˜hqrN = ZN bhqrN ; ˜ 3) Solving system D˜hqrz˜hqr = bhqr: . Matrix-vector operations

13: for k = 0 to N − 1 do       w ˜w z˜hqr bhqr Wk 14:  k  =  k  −   z˜ ;       hqrN z˜y ˜by Y hqrk hqrk k 15: end for

−1 4) Computation of zhqr = HR z˜hqr: . Matrix-vector operations

16: *Initial conditions: ∆λ0 =z ˜hqrN ;

17: for k = N to 1 do

y 18: ∆x =z ˜ ; k hqrk−1 19: ∆λ = −G ∆x +z ˜w ; k k−1 k hqrk−1 20: end for

| 21: return zhqr = [∆x, ∆λ] ; . Solution vector of system (7.1)

22: end function

86 We call this algorithm, the Hamiltonian recursion method. To complete the step direction vector ∆p of the Newton system (5.8), Algorithm 7.2 computes the remaining part,

∆˜ p = [∆u, ∆µ, ∆t]|.

Algorithm 7.2 Remaining step direction computation ∆˜ p

1: function comp-dire2-mpc-online(∆x, ∆λ, Ri, M, T , B,r ¯u, rµ, rt) −1 2: −1 x −1 x ∆µx0 = Mx0 Tx0 Ω∆x0 + rµ0 − Mx0 rt0 ;

3: −1 x ∆tx0 = −Mx0 Tx0 ∆µx0 + rt0 ;

4: for k = 0 to N − 1 do . Forward substitution

−1 | 5: ∆uk = −Rk (B ∆λk+1 +r ¯uk ) −1 6: ∆µ = M−1 T Ω∆x + rx − M−1 rx ; xk+1 xk+1 xk+1 k+1 µk+1 xk+1 tk+1 −1 7: ∆µ = M−1T Φ∆u + ru − M−1ru ; uk uk uk k µk uk tk 8: ∆t = −M−1 T ∆µ + rx ; xk+1 xk+1 xk+1 xk+1 tk+1

9: ∆t = −M−1 T ∆µ + ru ; uk uk uk uk tk

10: end for

11: return ∆˜ p = [∆u, ∆µ, ∆t]|;

12: end function

7.2.2 Recursion sequence structure on the companion-like matrix

Part of the matrix in system (7.8) has a symplectic structure, as the Hamiltonian system

(3.4). Its inversion is guaranteed throughout the entire recursion k = 0,...,N − 1, as we will demonstrate with the following result.

Lemma 7.2.2. The inversion of the system matrix (from 7.8)

 −1        A −IGk  K =     ,  −     A | Qk+1 − (Qk+1Gk + I)    | {z } | {z } KA KS is guaranteed provided A is non-singular.

Proof. The inversion of KA exists under the assumption that A is non-singular, i.e.

  A−1 K−1 =   . A   A|

87 Regarding the case of KS, a more elaborated analysis is required. As demonstrated in

[21, sec. 3.2], matrices from k = 1,...,N − 1 on sequence Qk are symmetric positive semideﬁnite; whereas matrices on sequence Rk are symmetric positive deﬁnite, and so are −1 Gk = BRk B|. Likewise, matrix KS is symplectic [81], satisfying

| KSJKS = J , where

  0 I   J =   . −I 0

The inverse of the symplectic matrices is simple to ﬁnd

| −1 KSJ = JKS ,

−1 | −1 J KSJ = KS , and since J −1 = −J

−1 | KS = −J KSJ .

After multiplications

  − (Qk+1Gk + I) −Gk K−1 =   , S   −Qk+1 −I therefore, matrix K exists at every iteration k.

7.2.3 Nonsingular matrix ZN guarantee

Algorithm 7.1 is consistent if and only if matrix ZN of step 12 is non-singular. We justify this with the following statement.

Remark 7.2.3. First, by assuming that matrix Dhqr of the Newton system (7.1) is non- singular, and the inversion of preconditioners HQ and HR is guaranteed, the product of these matrices D˜hqr (companion-like matrix (7.7)) is eventually non-singular [47, p.121].

Since D˜hqr is a block upper triangular matrix, its determinant is the product of the de-

88 terminants of the diagonal blocks [82, p.25]. All this implies that these determinants are non-zero, including matrix ZN .

7.2.4 Flop count

The arithmetic complexity of Algorithm 7.1 is divided in four parts, three of them are computed by matrix-vector operations, and only the part related to the recursion of the companion-like matrix contemplates matrix-matrix operations. Table 7.1 summarizes such cost.

Table 7.1: Computational cost dependent on N of the Hamiltonian recursion Algorithm 7.1

Part Arithmetic complexity

˜ −1 2 2 Solution of bhqr = HQ bhqr N 6nx + 4nxnu + 5nx + 2nu

3 2 2 Recursion computation N 6nx + nx (3 + 4nu) + 2nunx

˜ ˜ 2 Solution of Dhqrz˜hqr = bhqr (N − 1) 4nx + 2nx

−1 2 Solution of zhqr = HR z˜hqr (N − 1) 4nxnu + nx + 2nu

For comparison, the computational cost of the Riccati recursion method [21], presented in Algorithm 6.1 (subsection 5.2.2) is also summarized in Table 7.2.

Table 7.2: Computational cost dependent on N of the Riccati recursion Algorithm 6.1

Part Arithmetic complexity

3 3 2 nu 2 Riccati equation N 4nx + 3nx (1 + 2nu) + 3 + nu (1 + 4nx)

2 2 Residual computation N 4nx + 6nxnu + 2nu + 4nu

2 Computation of ∆x and ∆λ N 4nx + nx (3 + 2nu)

3 Skimming through both algorithms, the coeﬃcient of the most expensive factor, i.e. nx, is smaller on the Riccati recursion than on the Hamiltonian recursion algorithm. However,

89 it is interesting to observe the special case when nx ≈ nu

43 Riccati recursion −→ n3 + (22) n2 + (7) n , 3 x x x

3 2 Hamiltonian recursion −→ (12) nx + (23) nx + (8) nx.

This shows that when in a given application there are as many (or roughly) control inputs u as there are states x, the computational cost of the Hamiltonian recursion algorithm is slightly smaller than the Riccati recursion. Thus, it renders this algorithm competitive and attractive, opening the possibility of it being used and/or analyzed by the optimization- solvers research community.

7.2.5 Direct method comparison

To prove the viability of Algorithm 7.1 in a real MPC simulation, we set it up for the numerical example presented in subsection 6.2.2 with the same parameters, except the horizon N = 30. We use the result to compare this algorithm with the Riccati recursion one (Algorithm 6.1).

×10 -9

1 relative error

0 14

10 40 30 6 20 10 2 k-iteration t-sample time

Figure 7.1: Relative error corresponding to the optimal solution vector, p = [x, u, λ, µ, t], during a closed-loop MPC simulation on a second order plant. The x axis corresponds to the sample time t. The y axis represents the interior-point iterations, k. The relative error (z axis) is computed by ρ = kpcom − prick∞/kprick∞ at each (t, k).

90 Figure 7.1 shows the relative error computation between the solution vector of the

Hamiltonian recursion method, pcom, and the Riccati recursion approach, pric. It is easy to observe with a suitable degree of accuracy that both methods are equivalent. Figure

7.2 shows the computational time between these two methods. The algorithms are implemented in Matlab R2015a running with Ubuntu 16.04 LTS 64-bit operating system and

Intel®Core i7-4770CPU -3.40GHz processor

×10 -3 9.5

8.5

7.5

time (s) 6.5

5.5

4.5 0 50 100 150 200 250 300 iterations

Figure 7.2: Comparison of the computational time during the simulation of the MPC controller on system (7.1) between the Hamiltonian recursion (blue) and the Riccati recursion (red) methods.

As it was shown in the ﬂop counts tables 7.1 and 7.2, the Riccati approach shows that it is slightly faster than the Hamiltonian recursion method.

7.3 GMRES method application

The implementation in the previous section of the companion-like matrix (7.7) to factorize the Newton system on the interior-point algorithm is carried out as a direct method.

Another attractive possibility is to consider an iterative method for such a task. We will show that the special structure of the companion-like matrix ensures rapid and bounded convergence on the GMRES method. GMRES stands for generalized minimal residual and

91 it was proposed in [83]. This Krylov method is used for solving general (non-symmetric) systems of linear equations such as the companion-like matrix itself, as opposed to the

MINRES method which is used for symmetric matrices, for instance the augmented system

(6.1). An eﬃcient application of MINRES for MPC can be found in [84].

7.3.1 Convergence

For direct methods, the arithmetic complexity is always fixed. Namely, the factorization of a given system is carried out by a predetermined recursive sequence (such as LU, QR or Cholesky). Computing it gives rise to the solution (only a for-loop needed). On the contrary, for iterative methods [48] the solution of the system is obtained after a sequence of approximate solutions converges (a while-loop is needed). From a practical viewpoint, ensuring fast convergence is important for efficient optimization solvers in MPC. Recently, there have been several works that have addressed this topic for different types of iterative methods. To name just a few of them:

1. In [85], the fast-gradient method is used. Regarding convergence, it gives a predeﬁned

upper bound for the number of iterations. A setback of this proposal is that it is

only applicable for input-constrained MPC

2. In [84], the MINRES method is used. By setting up a special structure of the

system that splits into two parts the active and inactive constraints, it improves the

condition number of the matrix in (6.1). In doing so, the rate of convergence is

increased.

3. In [79], the conjugate gradient method (CG) is used, where the matrix factors of

the augmented system (or the normal equations) are used as preconditioners for the

CG method. The authors introduce a regularization parameter (which is a common

practice for iterative methods to improve the condition number of the main matrix)

that is quite sensitive and has an important weight on the rate of convergence.

4. In [1], diﬀerent Krylov methods are used, GMRES, QMR and BiCG. The authors

compare the convergence of some numerical results for systems with and without pre-

conditioners, using a method that intertwines the approach in [84] with a condensing

procedure in the main matrices.

All the previous works related to interior-point solvers lack theoretical proof for a bound

92 on the number of iterations in their corresponding methods. We will demonstrate that the companion-like matrix (7.7) applied to the GMRES method has a defined convergence that depends on the size of the system, nx. Definition 7.3.1. To characterize the convergence of the GMRES method applied to solve the linear system in (7.1) with our proposed preconditioner Hhqr (7.5), we examine the degree of the minimal polynomial of the preconditioned system matrix D˜hqr, defined in ˜ (7.7). Recall that the minimal polynomial pD˜ of Dhqr is defined as the monic (that is, ˜ with leading coefficient 1) polynomial of smallest degree such that pD˜ Dhqr = 0. Saad & Schultz [83] and Campbell et al. [86] proved that, in exact arithmetic, GMRES requires at most as many iterations as the degree of pD˜ .

It is easy to see that the degree of pD˜ is bounded by nx + 1, where nx is the number of states x and is also the dimension of matrix ZN deﬁned in (7.9). Let pZN denote the minimal polynomial of ZN , and consider the polynomial p (z) = (z − 1) pZN (z). Then

˜ ˜ ˜ p Dhqr = Dhqr − I pZN Dhqr     0 Wk pZ (I) ∗    N       0 Y   p (I) ∗  k   ZN   . .   . .  .. .   .. .  .   . =          0 WN−1   pZN (I) ∗          0 Y   p (I) ∗  N−1   ZN      (ZN − I) 0

= 0.

Hence, p is a monic polynomial of at most nx + 1 degree that satisﬁes p D˜hqr = 0. The minimal polynomial of D˜hqr cannot be of higher degree, thus we conclude the following result.

Theorem 7.3.2. Assume that GMRES is applied to solve (7.1) with a nonsingular system matrix having blocks of size nx × nx, and that the left and right preconditioners HQ (7.4) and HR (7.3) are used. Then, in exact arithmetic, GMRES will compute the exact solution in at most nx + 1 iterations.

The numerical experiments in Section 7.4 will conﬁrm that this termination property of GMRES is also valid in the presence of rounding errors.

93 7.3.2 Algorithm

Algorithm 7.3 (see code in Appendix A.5) has the necessary modiﬁcations of Algorithm

7.1 so as to house the GMRES method. The main diﬀerence between both algorithms is that, with the implementation of the iterative methods, only matrix-vector operations are required, the so-called BLAS2-level operations.

7.3.3 Flop count

The arithmetic cost of Algorithm 7.3 is shown in Table 7.3.

Table 7.3: Computational cost of the fast Hamiltonian/GMRES Algorithm 7.3, dependent on horizon N

Part Arithmetic complexity

˜ −1 2 2 Solution of bhqr = HQ bhqr N 6nx + 4nxnu + 5nx + 2nu

2 2 GMRES method with companion-like matrix N 6nx + 4nxnu + 4nx + 2nu ngmres

−1 2 Solution of zhqr = HR z˜hqr (N − 1) 4nxnu + nx + 2nu

Here, the arithmetic complexity of the preconditioners HQ and HR is the same as that of the direct method. The most expensive part corresponding to the GMRES depends on

2 matrix-vector operations, nx. It depends also on the i−GMRES iterations, ngmres; that, as proved in Theorem 7.3.2, has a bound of ngmres = nx + 1.

Taking the special case of nx ≈ nu, the arithmetic cost of the GMRES becomes

3 2 Hamiltonian/GMRES method −→ (12) nx + (34) nx + (6) nx.

3 This has in fact the same cost on the relevant term, nx, as that of the direct method. This comes about by taking into account the theoretical bound of ngmres = nx + 1. However, we will demonstrate in the numerical experiments (Section 7.4) that when this iterative method is numerically implemented, convergence sometimes is lower than this bound.

94 Algorithm 7.3 Fast Hamiltonian/GMRES solver for the solution of system (7.1) using the companion-like matrix D˜hqr

1: function comp-gmres-mpc-online(Qi, Ri, PN , A, B, bhqr) ˜ −1 1) Computation of bhqr = HQ bhqr: . Matrix-vector operations y 2: *Initial conditions: ˜bw = −r¯ , ˜b = −A−|r¯ ; hqrk λk+1 hqrk xk 3: for k = 0 to N − 1 do         ˜w ˜w b A −AGk b r¯λ  hqrk+1     hqrk   k+2  4:   =     −  ; ˜bw −A−|Q −A−|F ˜by r¯ hqrk+1 k+1 k+1 hqrk xk+1 5: end for

y 6: *Final condition (k=N): ˜b = −P ˜bw − F ˜b − r¯ hqrN N hqrN−1 N hqrN−1 xN 2) GMRES implementation: . Matrix-vector operations ˜ 7: function gmres-algorithm(@companion-iteration, bhqr);

8: *GMRES algorithm iteration: in−→ rgmres, out−→ qgmres;

9: function companion-iteration(qgmres); . Matrix-free computation

10: −| *Initial conditions: wi = 0, yi = A qgmresN ; w w y y 11: ri = qi − wi, ri = qi − yi;

12: for k = i to N − 1 do       wk+1 A −AGk wk 13:   =    ;    − −    yk+1 −A |Qk+1 −A |Fk+1 yk w w y y 14: rk+1 = qk+1 − wk+1, rk+1 = qk+1 − yk+1;

15: end for

16: *Final condition (k=N): zN = −P5wN−1 − FN yN−1;

17: rgmresN = qgmresN − zN ;

18: return rgmres; . Residual vector for i−GMRES iteration

19: end function

20: return z˜hqr; . Solution vector of system (7.2)

21: end function

−1 3) Computation of zhqr = HR z˜hqr: . Matrix-vector operations

22: *Initial conditions: ∆λ0 =z ˜hqrN ;

23: for k = N to 1 do

y 24: ∆x =z ˜ ; k hqrk−1 25: ∆λ = −G ∆x +z ˜w ; k k−1 k hqrk−1 26: end for

| 27: return zhqr = [∆x, ∆λ] ; . Solution vector of system (7.1)

28: end function

95 7.4 Numerical experiments

In order to demonstrate the feasibility and performance of the previous method, a MPC numerical example is set up. To avoid the double factorization of the Mehrotra predictor- corrector Algorithm 5.1, which renders an iterative method ineﬃcient, other algorithms proposed in the literature can be used that only require a single factorization of the Newton system, such is the case of the Exact/Inexact Infeasible IPM Algorithm 1 presented in [78].

7.4.1 Experiment 1: Feasibility

A well-known testbench for MPC is the oscillating-masses example available in [87], that has been used in [66] and [88]. This consists of a predeﬁned number of masses connected by springs to each other in series, and to walls on either side. Actuators between the masses exert tensions. The masses have value of 1 and all spring constants are set at 1 with no

nx damping. The state vector xk ∈ R encompasses the displacements and velocities of the masses

| x = q1, q2, . . . , qnx/2;q ˙1, q˙2,..., q˙nx/2 ,

nu and the control inputs are represented by the actuators uk ∈ R

| u = [f1, f2, . . . , fnu ] ,

nx where nu = 2 − 1. We carried out the necessary modiﬁcations on the code to make it versatile, so as to allow setting the number of states nx as needed. It also contains a discretization procedure.

For the feasibility and convergence test, Algorithm 7.3 is coded in Matlab® (see Appendix

A.5) with a sample time of t = 0.5s; the number of masses is set at m = 20 (hence nx = 2m = 40), inputs nu = 19 and horizon N = 5. Regarding the inequalities constraints, the control can exert a maximum force of ±0.5 and the displacement of the states cannot exceed ±3.8. The weighting matrices Q = I, R = I are set; and terminal cost P is the solution to the inﬁnite horizon LQR.

96 4

0 states -2

-4 0 10 20 30 40

0.5

0 inputs

-0.5 0 10 20 30 40 time (s)

Figure 7.3: MPC simulation for the oscillating-masses example with nx = 40 states using the Hamiltonian/GMRES solver within an interior-point algorithm. First plot shows the evolution of the states xk subject to constraints, with initial conditions xk = 3.5 for the displacements, and xk = 0 for the velocities. Second plot shows the proﬁle of the control inputs subject to constraints.

Figure 7.3 demonstrates the simulation of the oscillating-masses model connected in a closed-loop fashion to an MPC controller that uses Algorithm 7.3 for the optimization solver.

7.4.2 Experiment 2: Convergence veriﬁcation under rounding errors

An important point is the convergence of the GMRES method. According to Theorem

7.3.2 for this specific example, the maximum number of iterations should not surpass the limit nx + 1 = 41. Figure 7.4 shows that this is in fact the case since one can observe that the profile of GMRES iterations fluctuates between 20 up to 41. The profile of the relative residual is shown in Figure 7.5. Figure 7.6 shows the profile of the GMRES iterations for a simulation with the same setting as Figure 7.4, except with nx = 160. Here, iterations fluctuate between 100 up to 150.

97 40

25 GMRES iterations 20 14 10 80 60 6 40 20 2 k-iteration time/t-sample

Figure 7.4: Convergence of the Hamiltonian/GMRES solver for the oscillating-masses example with nx = 40 states. The x axis corresponds to the sample time t throughout the MPC simulation. The y axis corresponds to the iterations of the interior-point algorithm at every sample time/(t−sample). The z axis shows the proﬁle of the number of iterations of the GMRES method that solves system (7.1).

×10 -7

0 GMRES relative residual 14 10 80 60 6 40 20 2 k-iteration time/t-sample

Figure 7.5: Relative residual of the Hamiltonian/GMRES solver for the oscillating-masses example. The x axis corresponds to the sample time t throughout the MPC simulation. The y axis corresponds to the iterations of the interior-point algorithm at every sample time/(t − sample). The z axis shows the proﬁle of the relative residual of the GMRES method that solves system (7.1).

98 150

140

130

120

110 GMRES iterations 100 14 10 80 60 6 40 20 2 k-iteration time/t-sample

Figure 7.6: Convergence of the Hamiltonian/GMRES solver for the oscillating-masses example with number of masses m = 80 (nx = 160 states). The x axis corresponds to the number of samples time/(t − sample) throughout the MPC simulation. The y axis corresponds to the iterations of the interior-point algorithm. The z axis refers to the number of iterations of the GMRES method that solves system (7.1).

7.4.3 Experiment 3: Convergence subject to diﬀerent horizons

The validation of the convergence for Algorithm 7.3 is easily observed in Figure 7.4 and

7.7. However, another interesting experiment is to vary the value of the horizon N in order to see how the distribution of the number of GMRES iterations looks like with respect to the total of IPM iterations. Figure 7.7 shows four simulations of the oscillating-masses example with the same setup of parameters (such as nx = 40) as previous experiments, except for N = 5, 10, 20, 40. Although all four tests reach the maximum number of GMRES iterations at 41 (conﬁrming Theorem 7.3.2), their distributions are rather diﬀerent. For instance, regarding test N = 5 (Figure 7.7a), the iterations are roughly equally distributed from iteration 21 to 41; but as the horizon increases, for instance in tests N = 20 and

N = 40 (Figures 7.7c and 7.7d respectively), their distributions are assembled around the maximum number of iterations. Therefore, from this speciﬁc example we can conclude that the longer the horizon N, the closer the total number of iterations (in the presence of rounding errors) will be to its theoretical bound of nx + 1 in each run of the interior-point algorithm.

99 150 200

180

160

140 100 120

100

80 50 60

40 Recurrence (IPM iterations) Recurrence (IPM iterations)

0 0 20 25 30 35 40 29 30 31 32 33 34 35 36 37 38 39 40 41 GMRES iterations GMRES iterations

(a) Number of GMRES iterations for a simulation (b) Number of GMRES iterations for a simulation with horizon N = 5. with horizon N = 10.

500 400

450 350 400 300 350 250 300

250 200

200 150 150 100 100 Recurrence (IPM iterations) Recurrence (IPM iterations) 50 50

0 0 30 32 35 36 37 38 39 40 41 30 31 32 34 35 36 37 38 39 40 41 GMRES iterations GMRES iterations

(c) Number of GMRES iterations for a simulation (d) Number of GMRES iterations for a simulation with horizon N = 20. with horizon N = 40.

Figure 7.7: Four diﬀerent MPC simulations for the oscillating-masses example with nx = 40 states using the Hamiltonian/GMRES solver within an interior-point algorithm. With four diﬀerent values of the horizon, N = 5, 10, 20, 40. The x axis corresponds to the GMRES iterations and the y axis represents the number of recurrences that happen throughout all the IPM iterations, without distinction of the sample time in which a given GMRES run occurred.

7.4.4 Experiment 4: Comparison

It is necessary to carry out a comparison test in order to give a reasonable perspective about the feasibility and potential that the Hamiltonian/GMRES solver might have. For instance, one of the most recent works on this regard was presented in [1]. The authors carry out several experiments with heuristically designed preconditioners applied to different configurations of the Newton method, using different Krylov methods such as GMRES,

BiCG and QMR. Likewise, they also apply the approach presented in [78], which splits the inequality constraints into two sets (active and inactive). By doing so the condition number of the relevant matrix (either with the augmented system or the normal equations)

100 is improved and the convergence of such iterative method is sped up. We will address this ill-conditioned issue in the next subsection 7.4.5.

GMRES iterations 2

0 0 5 10 15 IPM iterations (a) GMRES method convergence using the (b) Convergence plot of three diﬀerent Krylov companion-like matrix (7.7). Iterations (blue) and methods published recently in [1]. The GMRES the theoretical bound nx + 1 (red). result is represented by the blue line.

Figure 7.8: Comparison between the Hamiltonian/GMRES solver (Algorithm 7.3) and the GMRES preconditioned method proposed in [1]. All the parameters for the MPC simulation of the oscillating-masses example resembled exactly those in [1]. The x axis corresponds to the IPM iterations and the y axis represents the number of GMRES iterations.

The testbench used in [1] is the same as the one we used in our previous experiments.

To match up this setup, the parameters of the oscillating-masses example that need to be changed are the states nx = 12 (number of masses m = 6), the control inputs nu = 3 and the horizon at N = 30. The simulation will be run for only one sample time. Although we know in advance that the maximum convergence that this simulation will have is nx + 1 = 13 (due to Theorem 7.3.2), we replicate the same experiment so as to show a fair comparison of both proposals.

It is interesting to notice from Figure 7.8 the following:

1. In Figure 7.8a, the Hamiltonian/GMRES solver has a maximum number of GMRES

iterations of 13, going down sometimes to 2 and 1. Whereas with the GMRES

method from [1] (Figure 7.8b), the minimum number of iterations is considerably

above ours, around 19. It also varies greatly, reaching a maximum limit of around

38. The authors do not establish any bounds for these iterations since their proposal

lacks a proof for convergence.

2. Another interesting point is the total number of IPM iterations needed during the

single sample time. With the Hamiltonian/GMRES solver, the total number of

101 iterations is 15, whereas with the other GMRES method it is 20.

7.4.5 Experiment 5: Numerical issue

It is well known that the application of primal-dual interior-point algorithms causes ill- conditioning of the Newton system (5.8) (or the augmented system (6.1)). This is due to the fact that some elements of M−1T −1 tend to zero, while others tend to inﬁnity. A rigorous analysis of this issue can be found in [89], [21] and [70]. Likewise, computational experience has demonstrated that direct methods normally deliver an accurate and fast solution regardless of the ill-conditioned issue [90]. Nonetheless, there still might be some direct factorization methods that can encounter some diﬃculties [91]. Iterative approaches are likely to underperform or entirely fail unless appropriate preconditioners are applied.

Analysis of diﬀerent preconditioners can be found in [92], [93], [77] and [78].

The Hamiltonian/GMRES solver of Algorithm 7.3 is no stranger to this ill-conditioned problem, as we will demonstrate with the next experiment. The second order system

(6.17) used in subsection 6.2.2 will also be used for this example. The main changes are,

state x1k will be subject to constraints, i.e. −0.2 ≤ x1k ≤ 10 and the control input is subject to −0.04 ≤ uk ≤ 0.85. Other minor changes are R = 1 and the horizon N = 5. Figure 7.9 shows an MPC simulation of the system (6.17) by two optimization solvers,

Hamiltonian/GMRES and the Riccati recursion. Double precision u = 1 × 10−16 of IEEE standard arithmetic was used. It is clear that, regarding the Hamiltonian/GMRES solver, a flag was triggered that supervises the size of the step direction norm at every interior- point iteration. When this norm blows up, the flag breaks up the while loop. This is represented by the flat line of the input u, which causes the violation of the state x1 with respect to its lower limit −0.2 during the simulation.

−1 −1 −1 −1 Figure 7.10 shows the inﬁnity norms proﬁles of the vectors Mx Tx and Mu Tu , corresponding to states x and inputs u respectively. With the Hamiltonian/GMRES solver, the maximum value of the norm regarding states x reaches the order of magnitude of

1020. This is considerably higher than the 1016 that appeared with the Riccati recursion solver. This discrepancy might conﬁrm that there exists some cancellations due to the ill-conditioning of the Newton system (5.8) in the Hamiltonian/GMRES solver, [94]. This happens exactly when the supervisory ﬂag is triggered. In order to verify it, the precision on the code can be increased. To do so, we use the Multiprecision Computing Toolbox [2].

102 1.4 1.4 x x 1 1 x x 1.2 2 1.2 2

1 1

0.8 0.8

0.6 0.6

0.4 0.4 states (x) states (x) 0.2 0.2

0 0

-0.2 -0.2

-0.4 -0.4 0 1 2 3 4 0 1 2 3 4 time (s) time (s) (a) Hamiltonian/GMRES solver. Evolution of the (b) Riccati recursion solver. Evolution of the states, x1 and x2. x1 is subject to a lower con- states, x1 and x2. x1 is subject to a lower constraint −0.2 (dashed red line). straint −0.2 (dashed red line).

0.9 0.9

0.8 0.8

0.7 0.7

0.6 0.6

0.5 0.5

0.4 0.4

input (u) 0.3 input (u) 0.3

0.2 0.2

0.1 0.1

0 0

-0.1 -0.1 0 1 2 3 4 0 1 2 3 4 time (s) time (s) (c) Hamiltonian/GMRES solver. Proﬁle of the (d) Riccati recursion solver. Proﬁle of the control control input u (blue) subject to the constraints input u (blue) subject to the constraints −0.04 ≤ −0.04 ≤ uk ≤ 0.85 (dashed red line). uk ≤ 0.85 (dashed red line).

Figure 7.9: Closed-loop simulation of the second order system (6.17). Double precision, u = 1 × 10−16, was used (which is standard for MATLAB®). The MPC controller is tested by two optimization solvers, the Hamiltonian/GMRES and the Riccati recursion.

103 X: 18 Y: 10 X: 14 Z: 1.693e+16 10 20 Y: 7 Z: 3.875e+20

10 10

10 0 10 0 infinity norm infinity norm

15 15 X: 1 40 X: 1 40 10 Y: 10 10 Y: 10 Z: 3.408e-06 30 Z: 3.408e-06 30 5 20 5 20 10 10 k-iteration time/t-sample k-iteration time/t-sample

(a) Hamiltonian/GMRES solver. Infinity norm (b) Riccati recursion solver. Infinity norm profile −1 −1 −1 −1 profile of the vector Mx Tx , which corre- of the vector Mx Tx , which corresponds to sponds to the states x. the states x.

X: 25 Y: 11 Z: 1.058e+06 X: 1 Y: 10 Z: 6.317e+06

10 0 10 0 infinity norm infinity norm

15 X: 5 15 Y: 13 40 X: 9 40 10 Z: 6.498e-07 10 Y: 10 30 Z: 2.928e-08 30 5 20 5 20 10 10 k-iteration time/t-sample k-iteration time/t-sample

(c) Hamiltonian/GMRES solver. Infinity norm (d) Riccati recursion solver. Infinity norm profile −1 −1 −1 −1 profile of the vector Mu Tu , which corre- of the vector Mu Tu , which corresponds to sponds to the inputs u. the inputs u.

−1 −1 Figure 7.10: Inﬁnity norm k·k∞ proﬁle of the vector M T . For clarity, every plot also highlights the maximum and minimum norm. Double precision, u = 1 × 10−16, was used. The x axis corresponds to the number of samples time/(t − sample) throughout the MPC simulation. The y axis represents the iterations of the interior-point algorithm.

104 1.4 0.9 x 1 x 0.8 1.2 2 0.7 1 0.6 0.8 0.5 0.6 0.4 0.4 input (u)

states (x) 0.3 0.2 0.2 0 0.1

-0.2 0

-0.4 -0.1 0 1 2 3 4 0 1 2 3 4 time (s) time (s) (a) Hamiltonian/GMRES solver. Evolution of the (b) Hamiltonian/GMRES solver. Proﬁle of the states, x1 and x2. x1 is subject to a lower con- control input u (blue) subject to the constraints straint −0.2 (dashed red line). −0.04 ≤ uk ≤ 0.85 (dashed red line).

Figure 7.11: Closed-loop simulation of the second order system (6.17) by an MPC controller with the Hamiltonian/GMRES solver. High precision arithmetic was set up at u = 1 × 10−45, using the Multiprecision Computing Toolbox [2].

The result of using this toolbox on the Hamiltonian/GMRES solver is shown in Figure

7.11, with a high precision of u = 1 × 10−45. This simulation shows the same behavior as the one using the Riccati recursion solver. There are no longer violations of the constraints.

This conﬁrms that there exist cancellations when using double precision.

7.5 The case of A being singular

In real-time applications, the systems are normally subject to time delays. Namely, the matrix A of the state space representation is singular. When this happens the inverse of the left preconditioner HQ (7.4) does no longer exist and hence, the companion-like matrix (7.7) cannot be formed. Consequently, Algorithms 7.1 and 7.3 are no longer valid. In addition, the bound on the maximum number of GMRES iterations (presented in Theorem

7.3.2) no longer holds.

Therefore, we will propose a change on the left preconditioner HQ in order to deal with matrix A being singular. Likewise, we will show with a numerical example the consequences that such a change produces on the GMRES convergence.

7.5.1 Formulation

We propose the following workaround,

105 Proposition 7.5.1. Let Dhqr and HR be the matrices deﬁned in (7.1) and (7.3) respectively. Therefore, the left preconditioner HQ (a lower block-triangular matrix) is

  I      0 A|   α       −A AGk I     |  Qk+1 Fk+1 0 Aα     . . . .  HQ =  ......  , (7.11) α      ......   . . . .     ..   . −A AGN−2 I       Q F 0 A|   N−1 N−1 α    PN FN I so that not only does its inverse exist but also D˜ = H−1D H−1. Matrix A| = hqrα Qα hqr R α (A| + αI) is non-singular where α > 0.

The direct method presented in Algorithm 7.1 can no longer be used since it requires the explicit form of the companion-like matrix D˜hqr (7.7), which is not available when the

α parameter is introduced into the diagonal of preconditioner HQ. On the contrary, the GMRES method can still be employed with its matrix-vector operations. It is simple to observe that the best selection of the α parameter (ideally) should be such that matrix ˜ ˜ Dhqrα resembles as close as possible to matrix Dhqr, but without compromising the increase on its condition number κ.

We will present next some numerical experiments that aim to propose some heuristic rules on how to set up (tune) this α parameter.

7.5.2 Numerical example

We use the following state-space model of a ﬁfth order system with time delay found in

[95, p.41]

xk+1 = Axk + Buk,

106 where

    1.4 −0.48 0 0 0 1          1 0 0 0 0 0         A =  0 1 0 0 0 ,B = 0 . (7.12)              0 0 1 0 0 0     0 0 0 1 0 0

The matrix A is evidently singular. We will ﬁrst show the variation of the condition ˜ number κ of matrix Dhqrα when α parameter varies. It is necessary that α > 0, as

κ = ∞ otherwise. Matrices HQ (7.11), HR (7.3) and Dhqr (7.1) are built with parameters: horizon N = 2, Q = I, R = I and P being the solution of the Riccati equation on the inﬁnite-horizon LQR.

10 10

10 8

10 6

10 4 condition number κ 10 2

10 0 0 0.2 0.4 0.6 0.8 1 α parameter

Figure 7.12: Condition number κ of matrix D˜ = H−1D H−1 subject to variations hqrα Qα hqr R of the α parameter. Experiment set with horizon N = 2.

Figure 7.12 shows a simulation where the α parameter varies from 0.01 to 1. It is obvious that when α → 0, the condition number κ → ∞. Conversely, when α increases, κ decreases. This gives some evidence that there will be a trade-oﬀ between the condition number κ and the number of GMRES iterations. Namely, the main objective is to emulate ˜ ˜ as much as possible Dhqrα ≈ Dhqr, which means α → 0, so that we can be close to the convergence established in Theorem 7.3.2. However, a big condition number should be avoided as it is a main factor for cancellations within the algorithm (as presented in

107 subsection 7.4.5).

7.5.3 Eigenvalue trajectories

The eigenvalues of the companion-like matrix D˜hqr (7.7) are divided in two parts. First,

2Nnx eigenvalues are exactly 1 and second, the eigenvalues of matrix ZN (7.9), which are arbitrary. Hence, the value of the α parameter has to be such that the 2Nnx eigenvalues ˜ of matrix Dhqrα are clustered as close as possible to 1. Consequently, its convergence will be near to the one in Theorem 7.3.2, while using the GMRES method.

10 4

3 10 3

2.5

10 2 2

10 1 1.5 eigenvalue trajectory eigenvalue trajectory 10 0 1

10 -1 0.5 0.2 0.4 0.6 0.8 1 0.2 0.4 0.6 0.8 1 α parameter α parameter

(a) Eigenvalues of ZN -like matrix (7.9). The num- (b) 2Nnx eigenvalues. Since horizon N = 2 and ber of these eigenvalues is nx = 5. nx = 5, the number of these eigenvalues is 20.

Figure 7.13: Eigenvalue trajectories of matrix D˜ = H−1D H−1. Each trajectory hqrα Qα hqr R (y axis) is the absolute value |µ| of an eigenvalue µ = a + bi. The x axis represents the variation of the α parameter.

Figure 7.13a shows the nx eigenvalues corresponding to the ZN -like matrix. These eigenvalues are arbitrary and depend on the parameters of the problem shown in Algo- rithm 7.1. The important result is the one depicted in Figure 7.13b regarding the 2Nnx ˜ eigenvalues of matrix Dhqrα . For small values of α → 0, these eigenvalues are clearly clustered around 1 and as α increases this cluster starts dispersing. This might be evidence of what was mentioned before that if this cluster is close to 1, the GMRES convergence ˜ ˜ for Dhqrα is likely to be close to Dhqr. Figure 7.14 shows the locus of the eigenvalues lying in the imaginary plane for four diﬀer- ent values of α = 0.01, 0.10, 0.50, 1.00. It shows that for small values of α, i.e. 0.01 and

0.10, the 2Nnx eigenvalues (in blue) are clustered very close to one, as opposed to larger values of α, i.e. 0.50 and 1.00, where these eigenvalues are now dispersed.

108 15 15

10 10

5 5

0 0 imaginary imaginary -5 -5

-10 -10

-15 -15 -10000 -8000 -6000 -4000 -2000 0 -150 -100 -50 0 real real

(a) The locus of 2Nnx eigenvalues (blue) and nx (b) The locus of 2Nnx eigenvalues (blue) and nx eigenvalues (red). Parameter α = 0.01. eigenvalues (red). Parameter α = 0.10.

3 6

2 4

1 2

0 0

imaginary -2 imaginary -1

-4 -2

-6 -3 -6 -4 -2 0 2 -3 -2 -1 0 1 real real

(c) The locus of 2Nnx eigenvalues (blue) and nx (d) The locus of 2Nnx eigenvalues (blue) and nx eigenvalues (red). Parameter α = 0.50. eigenvalues (red). Parameter α = 1.00. ˜ Figure 7.14: Imaginary plane for the eigenvalues µ = a + bi of matrix Dhqrα = H−1D H−1. Experiment set with horizon N = 2. Qα hqr R

7.5.4 Convergence experiment

We now carry out an experiment that includes the solution of the preconditioned system ˜ −1 −1 (7.2), with Dhqrα = HQ DhqrHR for the discrete-time system 7.12. The initial conditions

(2N+1)nx are set x0 = −0.2 ∗ [1, 1, 1, 1, 1], and bhqr = [Ax0, −Qx0, 0,..., 0] ∈ R . Figure 7.15 shows the result of this experiment. The GMRES convergence increases as the horizon N and α parameter rise. Nonetheless, for small values of α convergence is close to that of the companion-like matrix (7.7). For a ﬁfth-order system this would be nx +1 = 6. Figure 7.16 shows the consequences of using small values of α. Namely, the condition number κ of ˜ Dhqrα increases signiﬁcantly, which is very likely to cause cancellations on the algorithm.

109 16 26

24 15 22 14 20

13 18

12 16

GMRES iterations 11 GMRES iterations 12 10 10

9 8 0.2 0.4 0.6 0.8 1 0.2 0.4 0.6 0.8 1 α parameter α parameter (a) GMRES iterations with horizon N = 2. (b) GMRES iterations with horizon N = 5.

35 50

45 30

25 35

30 20

GMRES iterations GMRES iterations 25

15 20

10 15 0.2 0.4 0.6 0.8 1 0.2 0.4 0.6 0.8 1 α parameter α parameter (c) GMRES iterations with horizon N = 10. (d) GMRES iterations with horizon N = 20.

Figure 7.15: GMRES experiment with diﬀerent values of the horizon N. The x axis represents the α parameter and the y axis is the number of GMRES iterations.

110 10 10 10 20

10 8 10 15

10 6

10 10

10 4 condition number condition number

κ κ 10 5 10 2

10 0 10 0 0 0.2 0.4 0.6 0.8 1 0.2 0.4 0.6 0.8 1 α parameter α parameter (a) Condition number κ with horizon N = 2. (b) Condition number κ with horizon N = 5.

10 25 10 25

10 20 10 20

10 15

10 10 condition number condition number

κ κ 10 10 10 5

10 0 10 5 0.2 0.4 0.6 0.8 1 0.2 0.4 0.6 0.8 1 α parameter α parameter (c) Condition number κ with horizon N = 10. (d) Condition number κ with horizon N = 20.

Figure 7.16: GMRES experiment with diﬀerent values of the horizon N. The x axis ˜ represents the α parameter and the y axis is the condition number κ of matrix Dhqrα = H−1D H−1. Qα hqr R

111 Chapter 8

Conclusions of part I and II

We heretofore explored eﬃcient methods to solve the two-point boundary-value (TPBV) problem within the optimal control framework. In part I, our focus was on the Linear

Quadratic Regulator (LQR) and the Linear Quadratic Tracking problem. In part II, we paid attention to Model Predictive Control (MPC), which is applicable to industry, as opposed to the LQR. The summary of the novel methods obtained from this thesis are highlighted in the next sections, along with the next steps (or open questions).

8.1 Comments on novel methods (part I)

In subsection 3.3.1, we proposed a derivation of the eigenvalue decomposition to solve the ﬁnite-horizon LQR problem. This receding horizon idea could be attractive for unconstrained online MPC on linear time-invariant systems, when constant changes on the tuning parameters (weighting matrices) are required. Algorithm 3.2 is computationally eﬃcient for such cases, since there is no need for a backward recursion as opposed to the

Riccati equation when computing P1, as shown in Figure 3.1. In section 4.3, we extended the eigenvalue decomposition, initially only for ﬁnite-horizon

LQR problems, to the LQ tracking problem. Likewise, the receding horizon idea was implemented for this extension, rendering the unconstrained tracking MPC computation

(on linear time-invariant systems) very eﬃcient for applications where online changes are required on the tuning parameters. With this approach, the computation of P1 and π1 are computationally less expensive than the one computed by the Riccati equation, as shown in Figure 4.1.

112 8.2 Further work on novel methods (part II)

Comments on the limitations (that are in fact opportunities for future research) of the methods presented during part II are detailed in the next sections.

8.2.1 On the splitting methods

In section 6.3, the eigenvalue decomposition presented in part I was extended to the constrained MPC framework so as to factorize the Newton system, shown in Algorithm 6.2.

This algorithm is computationally unsuitable for online implementations, as an eigenvalue decomposition is required at every k iteration. Necessary modiﬁcations were done in section 6.4 in order to remove the k-dependency on the eigenvalue decomposition. The structure obtained from this modiﬁcation suits naturally well the iterative splitting method.

Here, the cost of the constant matrix factorization becomes data when dropped oﬄine (see

Figure 6.2a), and the remaining variable part is then computed by the splitting method.

However, the result depicted in ﬁgure 6.3 shows that sometimes the method is unable to reach convergence after iteration i = 30. Therefore, there exists an interesting opportunity

−1 for further research to study the structure of the matrix F = Mc Mv, so as to propose methods to speed up and ensure convergence. In doing so, the eigenvalue decomposition with splitting method will become attractive for real-time implementations.

8.2.2 On the Hamiltonian recursion solver

We demonstrated that the Newton system can be decomposed as an ILU factorization by using the Hamiltonian structure of the two-point boundary-value (TPBV) problem. The result of this ILU factorization is a companion-like matrix (7.7) that has some interesting properties, in the sense that it can be factorized by either direct or iterative methods

(described in the next section). Regarding the former, Algorithm 7.1 carries out a forward substitution (as opposed to the backward substitution with the Riccati equation) to compute a sequence of matrices, Wk and Yk, required to factorize the system. Although computationally speaking, there exist more eﬃcient proposals than the Hamiltonian recursion method (such as the factorized Riccati recursion proposed in [96] or the regularized method proposed in [88]) this is a novel structure. Hence, it remains to be analyzed further to determine its entire value and to propose eﬃcient algorithms. Likewise, the

113 Hamiltonian recursion method requires the inversion of matrix A, and therefore, alternatives are needed so that the algorithm can address this special case. One option could be to incorporate the generalized eigenvalue decomposition method presented in subsection

3.3.2, which is valid when matrix A is singular.

8.2.3 On the Hamiltonian/GMRES solver

As presented in section 7.3, the factorization of the companion-like matrix (7.7) can be carried out by the GMRES method, which is a Krylov iterative method. Contrary to other iterative methods proposed in the literature to solve MPC problems, we proved that the convergence of the GMRES method (in Theorem 7.3.2), applied to the companion-like matrix, is bounded. Namely, the maximum number of iterations is dependent on the numbers of states, nx + 1. Figure 7.8 shows that apart from this bound, the method turns out to be superior in terms of the total number of iterations, compared to the state-of-the-art literature, where convergence is not speciﬁed (refer to [1]).

Furthermore, the major challenge for the Hamiltonian/GMRES solver is the ill-conditioning issue reported in subsection 7.4.5. This is caused when the system inherits the ill- conditioning nature that has an impact on all the interior-point based algorithms. The eﬀects of this issue are substantially more visible when iterative methods are used. An option to address the issue is to use the partition proposed in [78] and [84]. The authors of these works propose dividing the Newton system into two subsystems by introducing a parameter that divides the inequality constraints into two sets, active and inactive. The

ﬁrst subsystem contains the structure of the two-point boundary-value (TPBV) problem with the active constraints, which is less ill-conditioned than the original system. And the second subsystem contains only the inactive constraints, where its factorization is simple to carry out since the main matrix is diagonal. Another option to address the ill-conditioning problem is to implement some of the extensions for the GMRES method that deal with nearly nonsingular systems, such as the method proposed in [97]. It is likely that by improving the condition number of the companion-like matrix, the number of iterations will be more equally distributed, i.e. away from the upper bound (see Figure

7.4.3), requiring fewer iterations than the theoretical limit (Theorem 7.3.2). It might be also favorable when matrix A is singular, as there exists a trade-oﬀ between the condition number and the parameter α, as shown in Figures 7.15 and 7.16.

114 Part III

Embedded Model Predictive Control

115 Chapter 9

Festo AG & Co.KG Collaboration

The motivation for the development and study of the eﬃcient optimization solvers for

Model Predictive Control (MPC) presented in previous chapters, is its implementation on real-time applications using embedded platforms. In this sense, the project was spon- sored by a company called Festo (www.festo.com) throughout its entire duration. In the following sections, we will present the details of this collaboration and its objectives.

9.1 The Company

Festo is a German company that develops and provides technical support for ground- breaking control equipment (software and hardware), focused on a wide variety of industries. Some recent details of the company are,

Headquarters: Esslingen am Neckar, Germany.

Founded: 1925.

Employees: 20,100.

Turnover: D3.1 billion in 2017.

R&D investment: Around 8% of turnover.

This collaboration has the following goals:

Aim: Implementation of on-line Model Predictive Control (MPC) on Festo Pro-

grammable Logic Controllers (PLCs), intended for small and medium-scale process-

industry applications.

116 Objectives: Development of an in-house Quadratic Programming (QP) engine for

Festo platform devices, CPX-CODESYS.

Figure 9.1: A Bionic Kangaroo designed by Festo Bionic Learning Network. The ﬂight phase control uses an optimal state feedback controller [3].

This collaboration arose due to a professional relationship that dates back to 2007. Specif- ically, we are currently working along with the Future Concepts division of the Process

Automation department in Festo.

9.2 Why Model Predictive Control (MPC)?

Over the last 25 years, Model Predictive Control (MPC) has been successfully applied to a wide variety of process-industry applications [16]. MPC is an advanced control technique that can include, explicitly within its framework, inequality constraints for the states x and the inputs u (as seen in part II), which is not achievable with more traditional control techniques such as PID controllers [35]. Likewise, the MPC structure naturally allows the incorporation of multivariable systems [4]. Traditionally, these advantageous features led the MPC controllers to be implemented on large-scale applications, such as the ones found in the petrochemical industry.

These type of applications are normally slow-sampled, which means that the control input update is required every few minutes or even hours. This facilitates the implementation of MPC controllers on these applications that although computationally expensive, gives plenty of time for the optimization problem to be solved. In addition, the nature of these

117 large-scale applications means that the MPC controller used to reside in control rooms with plenty of available computational resources.

Plant Optimization Management Operation (hour)

MPC MPC

Local Loop Controllers PhD Project (PID)

Actuator and Sensor Actuator and Sensor (Valves or motors) (Valves or motors) Typical Use Current Trend Figure 9.2: MPC hierarchy found in industrial applications (diagram from [4]).

Speciﬁcally, the typical location of the MPC controller used to lie between the management operation software and the local loop controllers, which normally requires faster samples (left hand side of Figure 9.2). Nonetheless, technological innovations in the ﬁeld of embedded platforms (i.e. more powerful processors and the increase of memory capabilities); and the need for self-contained (modularization) small-scale applications (concep- tualized within the Industry 4.0 trend) have recently brought down the MPC controller to the local loop level in the hierarchy. It is exactly here (right hand side of Figure 9.2) where the main goal of this collaboration lies. Namely, the on-line implementation of

MPC controllers on Festo embedded platforms in order to take their advantages to local and modular systems.

9.2.1 MPC oﬀered by major control-systems vendors

As mentioned previously, MPC software has been around for the past few decades. Besides the beneﬁts mentioned above, another attractive advantage, from a customer viewpoint, is that the implementation of MPC on process-industry applications can entail energy savings. Some interesting applications achieving this can be found in the following report

[98]. This is why, major control-systems vendors have long been oﬀering MPC software, known as APC software (Advanced Process Control), for specially large-scale applications

118 (see Figure 9.3).

Figure 9.3: Some of the major control-systems vendors oﬀering MPC software in their corresponding product portfolio. Within the industrial jargon, MPC software is known as APC which stands for Advanced Process Control. All these brands are registered trademarks.

Most of these vendors offer the kind of off-the-shelf MPC software designed for these demanding systems (several thousands of signals) and hence its software is normally installed on the customer’s large computational systems. However, there are a few of them that have started to offer MPC software embedded on their small-scale (and normally inexpensive) proprietary devices, such as the so-called Programmable Logic Controllers or PLCs. This way the MPC software can be implemented on small and modular systems. The PLCs are the key rugged industrial computers for the control and automation of all kind of processes. They can be found practically everywhere in industry; for instance, aerospace, petrochemical, automotive, waste-water treatment plants, amongst others. Therefore, this is a niche that Festo wants to exploit (MPC on PLCs). In the next section, we will present the state of the art and a feasibility study for the proprietary PLCs from Festo, so as to properly implement MPC.

119 9.3 Programmable Logic Controller platform (PLC)

PLCs are very versatile embedded platforms that are widespread in industry. The market growth of these devices is certainly ensured for years to come [99], due to the current trend of digitalization and automation of even more modular processes. In the case of Festo, who has a wide variety of different PLCs, the one suited best for an MPC algorithm is the CPX terminal (Figure 9.4). The CPX terminal is a modular PLC with the option for different configurations depending on, for example, the number of digital and analog signals I/Os or the communication protocols needed. It also has the possibility to include a pneumatic interface that allows different types of electrovalves.

Figure 9.4: CPX platform. Festo proprietary PLC.

9.3.1 Embedded MPC on PLCs

In recent years, there have been a couple of works that implement MPC on PLCs. Figure

9.5 depicts a diagram divided in two parts depending on the method used to compute and solve the optimization problem. On the left hand side, [5] and [6] use a method called explicit MPC. This approach, presented in [71], takes the computational effort offline by solving a multi-parametric quadratic program mp-QP, which generates different piecewise affine controllers that are saved as a look-up table. Conversely, online MPC solves the optimization problem at every sample time. A work based on this approach using the Riccati recursion method is in [7], presented in section 6.2. Fast-gradient methods implemented on PLCs can be found in [10] and [8]. Likewise, an interesting MPC comparative study between the so called PACs, which stands for Programmable Automation Controllers (a more powerful device but less widespread in industry) and PLCs can be found in [9]. The

120 optimization solvers used in this work are, the Hildreth QP Algorithm [100] and qpOASES which is a an active-set algorithm [61].

Research on Embedded MPC for PLCs

Explicit MPC Online MPC

Optimal PLC Optimal PLC Authors Authors Technique Equipment Technique Equipment -G. Valencia & Allen-Bradley -G. Frison ABB AC500 1 mp-QP 3 HPMPC Rossiter (2011) PLC SCL500 et al. (2014) PM592 -M. Kvasnica Only -D. Kufoalor ABB AC500 2 mp-QP 4 FiOrdOs et al. (2010) simulated et al. (2014) PM592 -B. Huyck Siemens qpOASES et al. (2012)5 S7-300 -M. Pereira Schneider 6 FG et al. (2015) Electric BMX PhD Project

Figure 9.5: Research overview of the implementation of MPC algorithms on PLCs. Ref- erences: 1. G. Valencia-Palomo & J.A. Rossiter [5], 2. M. Kvasnica et al. [6], 3. G. Frison et al. [7], 4. D.K.M. Kufoalor et al. [8], 5. B. Huyck et al. [9] and 6. M. Pereira et al. [10].

9.3.2 Comparison between diﬀerent PLC platforms

Since MPC algorithms are computationally demanding, it is important to consider the memory size and processor capabilities of low-cost devices, such as PLCs. To determine whether the Festo PLC is suitable for running MPC algorithms, it will be compared against diﬀerent PLC data sheets reported in the MPC-on-PLCs research (Figure 9.5).

Table 9.1: Comparative PLC properties between diﬀerent manufacturers

Brand PLC Memory Processor Floating-point Software

32MB Flash CPX-CEC-S1-V3 800 MHz 0.43µs CODESyS 256MB RAM

AC500 4MB Data 400 MHz 0.004µs CODESyS PM592-ETH PLC 4MB RAM

S7-319 8MB Data Step 7 undeﬁned 0.04µs CPU319-3 PN/DP 700KB RAM Professional

SLC 5/05 64KB Data Control Logix undeﬁned 0.37µs 1747-L543 undeﬁned RAM 5000

Modicon-M340 4MB Data undeﬁned 1.16µs Unity Pro BMXP3420302 256KB RAM

121 Table 9.1 shows a brief summary of diﬀerent properties of PLCs, in addition to the

Festo PLC, where MPC algorithms have already been implemented. The Festo PLC has a signiﬁcant larger memory and a more powerful processor than the others. The average cycle time for a ﬂoating-point operation is within the same order of magnitude 10−6 (µs) for all the PLCs.

Although the software needed to program diﬀerent automated routines has diﬀerent names depending on the manufacturer, it is based on a standard called IEC 61131-3 that serves as a guideline for PLC programming. This standard was established by the international standardisation organisation IEC (International Electrotechnical Commission). Proper references for the IEC 61131-3 standard are [101] and [102].

9.3.3 Some commercial embedded APC software for PLCs

Although most of the APC vendors depicted in Figure 9.3 have been focused exclusively on large-scale applications, more recently some of them have also started oﬀering built-in

APC function blocks in order to be implemented on their low-cost platforms (PLCs). Table

9.2 shows some of these manufacturers and their software, hardware, and some external tools for modeling (mathematical representation of the system).

Table 9.2: APC software embedded on PLCs currently oﬀered by some manufacturers

Brand APC Software PLC Modeling tools

APROL-APC PLC X20 System Predeﬁned models

SIMATIC PCS 7 SIMATIC APC Engineering

Function Blocks CPU 410 E Tools (External)

DeltaV Predict DeltaV system DeltaV Predict

DeltaV PredictPro (more a DCS) Oﬄine procedure

Studio5000 1756 Logix MPC builder

PlantPAx MPC Dedicated MPC mod Oﬄine procedure

In the next chapters, we will translate the MPC algorithm coded hitherto to CODESyS language (Festo PLC software) for implementation on a Hardware-in-the-loop (HIL) test.

122 Chapter 10

Optimization Solver Engine

As presented in Chapter 4, an essential part of a feedback controller in real-time applications is to drive the measured outputs (some of the states) to a speciﬁed reference.

Moreover, most of the systems in industry are subject to exogenous disturbances, either via the input or the output. For Model Predictive Control (MPC) there exists a formulation that contemplates these additional parameters. The MPC framework presented in Chapter 6 is easily scalable to this formulation, and so are the optimization methods presented in Chapters 6 and 7: the Riccati-recursion method (section 6.2) and the novel

Hamiltonian recursion method for both the direct (section 7.2) and iterative (section 7.3) approaches.

In addition to the presentation of this extended formulation, we will ﬁrst give the steps to generate the IEC 61131-3 function blocks of Algorithm 7.1, which was originally programmed in Matlab®, for a speciﬁc industrial application (the Flotation-cell system).

This will pave the way to incorporate Algorithm 7.1 into a real-time controller embedded on a Festo PLC, and hence to test its applicability and feasibility (Chapter 11).

10.1 Tracking and Disturbances Framework on MPC

The following description was mainly taken from [103], [11] and [13]. The free-ﬁnal state

MPC (regulator) problem depicted in (5.1) drives the states x from some predeﬁned initial conditions x(0) to the origin (equilibrium point). In order to include setpoint tracking parameters into the formulation, it is necessary to carry out a coordinate transformation.

123 10.1.1 Coordinate transformation and steady-state target problem

We deﬁne an extension of the time-invariant system in (5.1)

xk+1 = Axk + Buk,

yk = Cxk, (10.1)

ny where yk ∈ R is the measured vector with sequence Y = (y0, y1, . . . , yN−1) and C ∈

Rny×nx is the output matrix. C = I, when the states are measured entirely. It is assumed that the pair (A, C) is observable.

The steady state setpoint of the system (10.1) can be deﬁned as (xs, us). For the unconstrained setpoint tracking problem, the requirement is that the steady state setpoint satisﬁes the following system

      (I − A) −B x 0    s       =   , (10.2) C 0 us ysp | {z } Fs where matrix Fs is of size (nx + ny)×(nx + nu) and ysp is the goal of the setpoint tracking problem. When this system has a solution, some deviation variables can be deﬁned as

x˜k = xk − xs,

u˜k = uk − us, (10.3) that satisfy the dynamic system (10.1), i.e.

x˜k+1 = Axk + Buk − (Axs + Bus) ,

x˜k+1 = Ax˜k + Bu˜k.

The deviation variables (˜x, u˜) satisfy the same dynamic model as the original problem

(10.1). For the system (10.2) to have a solution for all ysp, it is required that ny≤nu

(which means that the rows of matrix Fs are linearly dependent). In real applications, it is common that ny>nu (more measured outputs than inputs). Consequently, a set of outputs is chosen such that nr≤nu. This gives rise to a new variable, namely r = Hy, known as the controlled variable, which is of size nr. To handle inequality constraints on

124 the steady state setpoint (xs, us), the following optimization problem is set

1 min |u − u |2 + |Cx − y |2 , (10.4) s sp Rs s sp Qs xs,us 2 subject to

      (I − A) −B x 0    s       =   , (10.5) HC 0 us rsp

Φus≤υu, (10.6)

ΩCxs≤υx, (10.7)

where Φ, Ω, υu and υx were already deﬁned in (5.4). Vector rsp contains the assigned setpoints of r.

10.1.2 Dynamic regulation problem

Given the solution of the previous steady state optimization problem, the transformed free-ﬁnal state MPC problem is

N−1 1 2 1 X 2 2 min |x˜N |P + |x˜k|Q + |u˜k|R X ,U 2 2 i=0

s.t. x˜k+1 = Ax˜k + Bu˜k,

Φ˜u≤υu − Φus,

ΩCx˜≤υx − ΩCxs, (10.8) where the initial condition isx ˜0 = xbk − xs and xbk is the estimated state, deﬁned in subsection 10.1.3. Since the receding horizon control requires the ﬁrst input of the decision vector, i.e.u ˜0 (as shown in subsection 3.3.1), the original control input is u0 =u ˜0 + us.

10.1.3 Constant disturbances and zero oﬀset

A simple method to compensate oﬀsets on the controlled variables caused by unmeasured disturbances is to augment the system (10.1) with a new state, depicted as an integral

125 disturbance with white noise, namely

dk+1 = dk + wd.

It is established by the works of [104], [105] and [106], that for nonzero and constant disturbances the following augmented system

        x AB x B  k+1  d  k     =     +   uk + w, dk+1 0 I dk 0   xk y =   + v, (10.9) k CCd   dk can cancel the effect of such disturbances and therefore eliminate the offset on the controlled variables. In a real controller, the states and the disturbances of system (10.9) are estimated by the Kalman filter equations (simplified notation version)

             xk+1 ABd xk B Lx xk b  =   b  +   u +   y − b  . (10.10)         k    k CCd   dbk+1 0 I dbk 0 Ld dbk

The next step is to modify the steady state optimization problem to include the augmented system, i.e.

1 min |u − u |2 + |Cx + C d − y |2 , (10.11) s sp Rs s d bs sp Qs xs,us 2 subject to

      (I − A) −B xs Bddbs           =   , (10.12) HC 0 us rsp − HCddbs

Φus≤υu, (10.13)

ΩCxs≤υx − ΩCddbs, (10.14)

In Lemma 2 of [11], it is proved that when the number of disturbances nd is equal to the number of measured outputs, nd = ny, and provided that (i) the outputs yk go to the steady state, (ii) the constraints are not active, and (iii) the closed-loop system is stable; then there is zero oﬀset in the controlled variables. The matrices Bd and Cd are of size

126 nx×ny and ny×ny, respectively. The choice of these matrices will be such that

  (I − A) −Bd   rank   = nx + ny. (10.15) CCd

The closed-loop diagram that contemplates the steady-state QP (10.11), the free-ﬁnal state

QP (10.8), and the Kalman ﬁlter (10.10) is depicted in Figure 10.1.

Disturbances and Noise

*Optimal Inputs Process-Industry Application

Real Plant

QP Solver for DAC Kalman Filter dynamic for Estimation HOLD h regulation Initial and Noise *Sensors Values Estimated Disturbance

QP Solver for *Setpoints Reference steady states h Values

MPC Digital Controller h=Sample Time DAC=Digital-to-Analog Converter

Figure 10.1: Closed-loop system diagram with MPC controller for tracking reference and zero oﬀset.

10.2 Case Study: Flotation-Cell System

In order to test the MPC controller shown in Figure 10.1, we will consider an industrial process that can be regularly encountered in the mining industry called the Flotation-cell system [107]. The layout of this system is an array of tanks connected in series (the total number depends on the speciﬁc conﬁguration), such as the one shown in Figure 10.2, which is an array of three tanks.

Loosely speaking, the main objective of a Flotation-cell system is to separate the valuable ore from the raw mixture (slurry or pulp). When this mixture is introduced to the ﬁrst tank (Fd in Figure 10.2), the ore material, acting as hydrophobic particles (i.e. repelling water molecules), attaches to the air bubbles, which are fed from the bottom of the tank.

The air bubbles burdened with the ore material go to the top, creating a froth layer that is then recovered. Likewise, the remaining slurry is led to the next cell by gravity, and the

127 process itself is repeated until as much mineral as possible has been recovered [108].

10.2.1 Single-tank model

In this part, the focus will be on mathematically describing the Flotation-Cell system for a single-tank array. We will use the balance equation, based on an instantaneous rate-of- change under the assumption that the system is perfectly mixed, i.e. the density of the slurry leaving is equal to the density of the slurry within the tank [12]. Thereafter, we will linearize the model. A single-tank array system with three tanks is shown in Figure 10.2.

hi1

qi1

s qi

qi1

Figure 10.2: Flotation-cell system for three tanks connected in series.

By conservation of mass, the relationship between its level hi and the input ﬂow qi−1 and output ﬂow qi is

Ash˙i = qi−1 − qi, (10.16)

˙ dhi where hi = dt and As is the area of the tanks. The input qi−1 and output ﬂow qi are controlled by valves, and their relationship with the system is explained by Bernoulli’s equation

s h − h + λ q = C v i−1 i s , i−1 v i−1 s.g. s h − h + λ q = C v i i+1 s , (10.17) i v i s.g.

where Cv is the valve coefficient, s.g. is the specific gravity, λs is the physical difference of the level between tanks and v is the control input of the valve, which in this case we

128 will consider linear (refer to [12, p.48]). It is interesting to notice that the pressure drop across the valve (normally deﬁned as ∆Pv) is the entire term in the numerator within the square root of both expressions in (10.17). Therefore, equation (10.16) becomes

s s Cvvi−1 hi−1 − hi + λs Cvvi hi − hi+1 + λs h˙i = − . (10.18) As s.g. As s.g.

Equation (10.18) is a nonlinear first-order differential equation that we will now linearize around an operating point. Let h˙i be defined as

h˙i = f (hi, vi, vi−1) , and deﬁning a new set of coordinates

x = hi − h¯i State ui = vi − v¯i Input ui−1 = vi−1 − v¯i−1 Input Disturbance yi = hi − h¯i Output (Level Sensor)

Therefore, we now carry out the ﬁrst order approximation of the Taylor expansion of f (hi, vi, vi−1) around the operating point h¯i, v¯i, v¯i−1

  ¯ ¯ ¯ ui x˙ = ∂f(hi,v¯i,v¯i−1) x + ∂f(hi,v¯i,v¯i−1) ∂f(hi,v¯i,v¯i−1)   , ∂hi ∂vi ∂vi−1   ui−1

yi = x. (10.19)

Computing the Jacobians, system (10.19) becomes

x˙ = −ax − Kiui + Ki−1ui−1,

yi = x, (10.20)

129 where

! Cv v¯i−1 v¯i−1 a = − √ p − p , (10.21) 2As s.g. h¯i−1 − h¯i + λs h¯i − h¯i+1 + λs

Cv q Ki = √ h¯i − h¯i+1 + λs, (10.22) As s.g. Cv q Ki−1 = √ h¯i−1 − h¯i + λs. (10.23) As s.g.

Finally, applying Laplace Transform to system (10.20) (for clarity we consider null initial conditions)

sx(s) + ax(s) = Ki−1ui−1(s) − Kiui(s),

since yi = x, the system in the frequency-domain is

K K y (s) = i−1 u (s) − i u (s). (10.24) i s + a i−1 s + a i

A similar analysis to the above for another kind of array, such as the double-tank system, can be found in Appendix B. In the next section we will set up the closed-loop MPC controller from Figure 10.1 for the Flotation-cell system using the state-space representation model just derived (10.20).

10.3 Numerical experiment: Flotation-cell system

To carry out the numerical experiment, a new variable is introduced that limits the rate of change of the input urc = uk − uk−1. In real-time applications, the change of state on a given input is subject to physical constraints (e.g. in a motor or a pneumatic valve), i.e. there cannot be an immediate change on an input after a controller’s command (sending a signal to close an open valve will not close it immediately).

We apply two changes to the optimization problem (10.8), ﬁrst, the rate of change of

| the inputs can be penalized by introducing a quadratic term urck Surck into the objective

nu×nu function, where S ∈ R is a weighting matrix. Second, to handle urc properly in the

130 decision vector (5.3), the following change of variable is proposed (refer to [13] and [21])

    x x  bj   k  x˜0 ←   , x˜k ←   , u˜k←u˜k, uj−1 u˜k−1

whereu ˜j−1 is the selected control inputu ˜0 from the previous sample time. Consequently, on the system

    A B     A ←   ,B ←   , 0 I and on the objective function

      Q P 0       Q ←   ,P ←   ,R← (R + S) ,D ←   . S 0 −S

We can also apply this change of variable to the Newton system (7.1), and therefore to the companion-like matrix D˜hqr (7.7) with preconditioners HQ (7.4) and HR (7.3). It is easy to verify that subsystem (6.9) becomes

          − A − BR−1D| 0 ∆x IG ∆x r¯  k   k  k   k+1  λk+1      +     = −   , −1 | | −1 | Qk − DRk D −I ∆λk 0 A − DRk B ∆λk+1 r¯x˜k (10.25)

−1 | | −1 with ∆uk = −Rk (D ∆xk + B ∆λk+1 +r ¯uk ) andr ¯x˜k =r ¯xk − DRk r¯uk .

10.3.1 Control design objective and settings

The main control objective for a Flotation-cell system is to keep the level in each tank near to its setpoint at all times. The challenge of this application is that there is an input disturbance (main inlet flow Fd from Figure 10.2) flowing downstream from the first tank to the last one, rendering the control of this system a difficult task. To address the problem, a feedforward control approach has been reported within the literature in [109] and [110]. We will apply an MPC controller (from Figure 10.1) using the optimization solver proposed in Algorithm 7.1.

First, for the MPC controller, we will use the linearized version (10.20) of the single-tank

131 system (10.18) for an array of three tanks as in Figure 10.2. A sample time of t = 1s and the following parameters, shown in Table 10.1, are set up.

Table 10.1: Physical parameters for the Flotation-cell system, refer to [12].

Parameter Description Value

Ds Nominal diameter (m) 6.860

2 2 As Cross sectional area m , As = π(Ds/2) 36.961

λs Diﬀerence between tanks (m) 0.975

s.g. Speciﬁc gravity 2.750

hmax Maximum pressure drop to avoid overﬂow (m) 1.830

3 qmax Maximum ﬂow rate m /s 1.550 p Cv Linear valve coeﬃcient, Cv = qmax s.g./hmax 1.900

Hence, the linearized system for three tanks connected in series is

      0.9987 0.0013 0.0000 −0.0376 0.0000 0.0000 1 0 0             A = 0.0013 0.9975 0.0013 ,B =  0.0376 −0.0376 0.0000  ,C = 0 1 0 ,             0.0000 0.0013 0.9977 0.0000 0.0376 −0.0490 0 0 1

regarding the steady-state operating conditions: xss = h¯1, h¯2, h¯3 = [3.5m, 3.0m, 2.5m] and sample time of t = 1s. Since there are equal number of measured outputs and inputs ny = nu, the additional matrices required for the augmented system (10.9) are easily established, i.e. Bd = I, Cd = I and H = I. It is simple to validate the condition of (10.15), namely rank = nx + ny = 6. The weighting matrices for the

3 4 3 steady-state optimization problem (10.11) are Qs = diag 1×10 , 1×10 , 1×10 and

2 2 2 Rs = diag 1×10 , 1×10 , 1×10 . The weighting matrices corresponding to the free- ﬁnal state MPC problem (10.8) are

Q = diag 9×103, 7×103, 7×103 ,

R = diag 1×103, 1×103, 1×103 ,

S = diag 19×103, 1×104, 1×104 ,

132 and

  8.8159 1.9881 0.5587   5   P = 1×10 · 1.9881 3.3499 0.6138 ,     0.5587 0.6138 1.4216 which is the solution to the inﬁnite-horizon LQR. Also, lower bounds are set for the inputs

0≤uk.

10.3.2 Simulation

For the simulation, we developed our code in Simulink®. This environment facilitates the interconnection between a digital controller and a plant, using a zero-order hold function.

This function emulates the digital-analog converter (DAC) encountered in real-time digital controllers connected to industrial processes. The program is divided into two parts (see

Figure 10.3):

1. Digital MPC controller function. This block contains three main functions

(Figure 10.1):

Kalman ﬁlter: This block contains the Kalman ﬁlter equation (10.10) that

receives the measured outputs y and then computes the estimated state and

disturbance xb and db.

Steady-state optimization solver: This part contains the code for the optimiza-

tion problem (10.11), based on the Mehrotra’s Predictor-Corrector algorithm

(Algorithm 5.1). The supervisory ﬂags and stopping criteria for this code were

taken from [28]. This block receives xb and db, and then computes the steady

state setpoints (xs, us).

Dynamic regulator optimization solver: This is the main block of the MPC

controller since it contains the code for the optimization problem (10.8). As

previously mentioned, the outer algorithm is based on the Mehrotra’s Predictor-

Corrector method (Algorithm 5.1). The inner algorithm, which relates to the

factorization of the Newton system, is based on the Hamiltonian recursion

method presented in Algorithm 7.1. For the diﬀerent parts of this code (corre-

sponding to the computation of HQ, HR and D˜hqr) see Appendix A.

133 2. Flotation-cell system S-function. This block has the nonlinear model for three

tanks based on (10.18). This code can be seen in Appendix A.6.

Figure 10.3: Closed-loop digital MPC controller for a Flotation-cell system with three tanks. The MPC controller is based on the control diagram in Figure 10.1 using a Matlab® function. For the plant, an S-function is coded for three nonlinear ﬁrst-order diﬀerential equations (10.18).

The simulation runs for T = 660s (10min), with a sample time of t = 1s and with a horizon of N = 5. The input disturbance (main inlet ﬂow Fd) is emulated as a square function with an amplitude of 0.5m3/s and a period of 10s. It has also an oﬀset of 1.0m3/s.

The change of setpoints are: (i) Tank 1: 4.2m @150s, (ii) Tank 2: 3.5m @350s and (iii)

Tank 3: 2.0m @500s. The result of this simulation is shown in Figure 10.4.

134 tank 1 4.5 tank 2 tank 3

3.5

level tanks (m) 2.5

1.5 0 100 200 300 400 500 600 time (s)

(a) Measured outputs y corresponding to the level sensors.

1.5 valve 1 valve 2 valve 3

0.5 control signals (u)

0 0 100 200 300 400 500 600 time (s)

(b) Manipulated variables u corresponding to the process valves.

Figure 10.4: Closed-loop MPC simulation for a Flotation-cell system with three tanks using the optimization solver presented in Algorithm 7.1.

Figure 10.4 shows that the MPC controller eﬀectively attenuates the disturbance coming into the system from the main inlet Fd, specially for tanks 1 and 2. Likewise, the controller drives the levels to their new setpoints when required. Figure 10.4b shows the control signals, which represent the process valves placed at the outlet of each tank. Their oscillatory behaviour is expected, as each valve opens and closes so as to compensate the inlet disturbance. This MPC controller function design is a prior step to generating (trans- lating) a suitable code for its implementation on the Festo PLC (Figure 9.4). To facilitate the code generation for this function, some aspects were considered: (i) structure arrays

(data type) were used in order to organize diﬀerent types of data throughout the algorithm and to use them as inputs. (ii) The main MPC function was coded without using nested

135 functions, as the code generation tool (more about this in section 10.4) does not allow them. (iii) As we are using our own handmade program portraying novel methods such as

Algorithm 7.1, the entire function only contains very basic and non-proprietary operations

(i.e. matrix-matrix or matrix-vector operations). Therefore it subscribes to the extended capabilities for code generation by MATLAB®.

10.4 Optimization Code Generation

The next step is to translate the MPC code used in the previous section to the language that Festo PLC uses (or talks). Figure 10.5 shows the process required to generate the code. There are four main parts:

Simulink PLC Coder 1 Toolbox 2

*Matlab Software: *Code generation: - Optimization Engine - From Matlab Code to Development Structured Text (IEC 61131-3)

4 3 *Embedded MPC on CPX: -Memory-size analysis -Processing cycle time -Feasibility Test -Hardware-in-the-loop (HIL)

*CODESYS design:

- From XML ¡ le to the creation of the MPC function block

Figure 10.5: Code generation process diagram for an MPC algorithm, from MATLAB® code to Structured-text language (IEC 61131-3 environment).

136 1. Optimization Engine Development. This part refers to the development of the MPC

algorithm from Figure 10.1, whose feasibility has been proved in the simulation

presented in the previous section. Furthermore, although there exist state-of-the-art

optimization solvers (e.g. based on the interior point method [24, 28] or the active

set method [111]), the idea behind the development of an own proprietary solver is

that Festo would obtain the know-how of the tailor-made solver and it would also

be able to design a more competitive oﬀ-the-shelf library for their PLCs.

2. Simulink PLC Coder toolbox. The toolbox generates Structured-text code (IEC

61131-3 environment) from MATLAB® code. A work applying this toolbox to a nonlinear MPC algorithm can be found in [112].

3. CODESYS Function Block. As shown in Table 9.1, the Festo PLC is programmed by

the CODESYS environment, which is based on the IEC 61131-3 standard. This environment oﬀers six diﬀerent programming languages. Most of them (such as Ladder

diagram) are unsuitable to carry out matrix operations except for the Structured-

text code. An XML ﬁle (generated by the toolbox) containing the MPC algorithm is

uploaded and then adjusted as a function block with inputs (tuning parameters such

as Q, R and S) and outputs (the optimal control input u∗). In addition, since the

IEC 61131-3 standard allows data structures [101], the structures arrays deﬁned in the MATLAB® code are recognized by this environment, with no need for further

change.

4. PLC Capabilities. The Festo PLC has a similar processor and a much more powerful

memory compared to the other PLCs (Table 9.1) reported in literature that have

already housed MPC algorithms. This might be suﬃcient to show feasibility on Festo

PLC. Nonetheless, to prove this we will carry out in the next chapter a Hardware-in-

the-loop (HIL) test for two diﬀerent plants. Likewise, it is important to highlight that

the processor, which is an ARM Cortex-A8, supports double-precision ﬂoating-point

numbers (64 bit IEEE Standard 754).

137 Chapter 11

Hardware-in-the-loop Test (HIL)

In order to test the feasibility of the Hamiltonian recursion method (presented in Algorithm

7.1) in real-time controllers, we set up a rig which contains two Festo PLCs (Figure 9.4) and additional control equipment. In this chapter, we ﬁrst describe the details of such equipment and its interconnection, followed by two experiments from two diﬀerent process- industry applications.

11.1 PLCs Test Rig

The control equipment for the experiment is shown in Figure 11.1

Festo PLC CPX-CEC-S1-V3

Connection between equipment: Ethernet TCP/IP Protocol

Festo HMI Touch Screen CDPX-X-A-W-13

Festo PLC CPX-CEC-S1-V3

Figure 11.1: Test rig for an MPC controller experiment. It has two PLCs and a touch screen (HMI). Physical connection between equipment is done through Ethernet TCP/IP protocol.

138 It has two PLCs in order to run experiments in parallel and an HMI touch screen, which serves as an interactive visual interface, sending commands and receiving data in a user-friendly manner. The physical connection between this equipment is done through

Ethernet TCP/IP protocol, which is a well-known and widespread protocol in industry.

11.1.1 HIL test diagram

For the real-time experiment, Figure 11.2 shows the interconnection between controller and plant. It can be summarized as follows,

The OPC software is used to exchange data between control systems. OPC stands

for Open Platform Communications, and it can be deﬁned as a data platform that

receives, organizes and distributes data between equipment. It is standard in indus-

try [113], and its advantage lies in the fact that it facilitates the interconnection,

and hence the interaction among control equipment from diﬀerent brands.

The nonlinear plant (mathematical model) will be emulated in MATLAB® with a

S-function (see code in Appendix A.6). For the interconnection with the PLC, we

use the MATLAB OPC toolbox.

The MPC controller is embedded in the PLC as a function block (see Figure 10.5).

It is based on the Hamiltonian recursion method proposed in Algorithm 7.1. The

PLC acts as an OPC server.

Mathematical model Embedded MPC on Signals CPX and HMI of the application on Matlab

OPC Connection

Sensors

Figure 11.2: Hardware-in-the-loop (HIL) diagram. The MPC controller is implemented on the PLC and the model is running in MATLAB®. All the equipment is connected through OPC protocol (Open Platform Communications).

139 11.2 Industrial experiment 1: Flotation-cell system

Everything is now set to carry out the Hardware-in-the-loop (HIL) test. For the plant, we will use the nonlinear Flotation-cell system applied in the simulation of subsection 10.3.2

(Figure 10.3). First, the viability of the Hamiltonian recursion method running in a real- time controller will be tested, and then a comparative experiment will be set between this method and the Riccati recursion approach (section 6.2).

Figure 11.3: Image captured from CODESYS environment. It shows the code size in the PLC after the code generation process (Figure 10.5).

11.2.1 Feasibility of the Hamiltonian recursion method

In order to facilitate the interaction between the parameters and the user, several screens were developed on the Festo HMI touch screen, such as the one shown in Figure 11.4. The experiment is set as follows,

The parameters of the system and tuning weighting matrices are kept the same as

the simulation presented in subsection 10.3.1.

The real-time test runs for T = 1040s, with a sample time of t = 1s and a horizon

of N = 5. The input disturbance (main inlet ﬂow Fd), introduced at t = 160s, is emulated as a square function with an amplitude of 0.2m3/s and a period of 10s. It

also has an oﬀset of 1.0m3/s. The change of setpoints are: (i) Tank 1: 4.0m @355s,

(ii) Tank 2: 3.5m @552s and (iii) Tank 3: 2.0m @823s. The result of this test is

shown in Figure 11.5.

140 Figure 11.4: Image captured from Festo HMI touch screen device showing real-time performance of the main tailor-made screen for the Flotation-cell system.

4.5

3.5

level tanks (m) 2.5

1.5 0 100 200 300 400 500 600 700 800 900 1000 time (s) (a) Measured outputs y corresponding to the level sensors.

1.5

0.5 control inputs (u)

0 0 100 200 300 400 500 600 700 800 900 1000 time (s) (b) Manipulated variables u corresponding to the process valves.

Figure 11.5: Real-time closed-loop MPC controller for a Flotation-cell system with three tanks using the optimization solver presented in Algorithm 7.1.

141 Figure 11.6: Image captured from CODESYS already logged into the Festo PLC (online). It shows the MPC function block.

PLC's Cycle Time: 0.0524sec

Figure 11.7: Image captured from CODESYS already logged into the Festo PLC (online). It shows the cycle time for the solution of the optimization problem (5.1).

This test shows that the MPC controller successfully drives the controlled variables y to their corresponding setpoints and it also attenuates the disturbance coming from the inlet

ﬂow. This conﬁrms that the Hamiltonian recursion method is feasible, i.e. it delivers the optimal control input u∗ throughout the entire experiment. Hence, it renders this method a viable option for optimization solvers of real-time MPC controllers embedded in small-scale platforms, such as PLCs.

In addition, Figure 11.6 shows the function block, within the CODESYS environment, based on the Hamiltonian recursion method of Algorithm 7.1; and also Figure 11.7 depicts the time that the PLC takes to solve the optimization problem (5.1) based on the

Hamiltonian recursion method in single cycle.

142 11.2.2 Hamiltonian Recursion vs. Riccati Recursion

Here, two PLCs will be set up to run in parallel, one with the Hamiltonian recursion method and the other one with the Riccati recursion approach. As in the previous test, several tailor-made screens were developed on the Festo HMI touch screen, such as the one shown in Figure 11.8.

Figure 11.8: Image captured from Festo HMI touch screen device showing real-time performance of the main tailor-made screen for the Flotation-cell system with two PLCs. Comparative test between Hamiltonian recursion and Riccati recursion methods.

The experiment is set as follows,

The parameters of the system, the tuning weighting matrices, the stopping criteria

and the outer algorithm (Mehrotra’s Predictor-Corrector Algorithm 5.1) are exactly

the same for both approaches.

The real-time test runs for T = 1228s, with a sample time of t = 1s and horizon

of N = 5. The plant is activated at T = 56s and the input disturbance (main inlet

ﬂow Fd) is introduced at T = 225s, and is emulated as a square function with an amplitude of 0.2m3/s and a period of 10s. It also has an oﬀset of 1.0m3/s. The

change of setpoints are: (i) Tank 1: 4.0m @407s, (ii) Tank 2: 3.5m @612s and (iii)

Tank 3: 2.0m @908s. The result of the measured outputs y is shown in Figure 11.9

and the manipulated variables u (process valves) are depicted in Figure 11.10.

143 4.5

3.5

level tanks (m) 2.5

1.5 0 200 400 600 800 1000 1200 time (s)

Figure 11.9: Measured outputs y corresponding to the level sensors. Two real-time closed- loop MPC controllers running in parallel for two Flotation-cell systems, with three tanks each. Two optimization solvers used: (i) the Hamiltonian recursion method, in blue (Algorithm 7.1) and (ii) the Riccati recursion approach, in red (Algorithm 6.1).

Some comments regarding these results:

1. Both MPC controllers successfully drive their corresponding controlled variables y

to their setpoints. They also attenuate the disturbance coming from the inlet ﬂow.

2. The Riccati recursion approach causes a more oscillating performance than the

Hamiltonian recursion method. The reason for this is still unclear, since as men-

tioned before, all the parameters in both controllers were kept the same. It might

be caused due to a mistranslation during the code generation that leads the round-

ing error to accumulate and then propagate diﬀerently throughout the algorithms.

From a control viewpoint, this outcome seems to be favorable to the Hamiltonian

recursion method (i.e. less oscillating behavior).

3. Additionally, we carried out intensive simulations in MATLAB® for both approaches,

and have obtained results with relative errors similar to the ones shown in Figure 7.1.

This might conﬁrm that there exists a variation between both methods developed

during the code generation process.

144 1.4

1.2

0.8

0.6

control input (u1) 0.4

0.2

0 0 200 400 600 800 1000 1200 time (s) (a) Control inputs u corresponding to tank 1.

1.4

1.2

0.8

0.6

control input (u2) 0.4

0.2

0 0 200 400 600 800 1000 1200 time (s) (b) Control inputs u corresponding to tank 2.

1.8

1.6

1.4

1.2

0.8

control input (u3) 0.6

0.4

0.2

0 0 200 400 600 800 1000 1200 time (s) (c) Control inputs u corresponding to tank 3.

Figure 11.10: Manipulated variables u corresponding to the process valves. Two real-time closed-loop MPC controllers running in parallel, for two Flotation-cell systems with three tanks each. Two optimization solvers used: (i) the Hamiltonian recursion method, in blue (Algorithm 7.1) and (ii) the Riccati recursion approach, in red (Algorithm 6.1).

145 11.3 Industrial experiment 2: Continuous stirred tank re-

actor (CSTR)

To test the versatility of the Hamiltonian recursion method, another well-known industrial application will be set up for a Hardware-in-the-loop Test (HIL). This time, a chemical process known as Continuous stirred tank reactor (CSTR) will be used. It is depicted in

Figure 11.11.

Figure 11.11: Diagram of a Continuous stirred tank reactor (CSTR), from [11].

We will take the following irreversible ﬁrst order reaction (from [114])

A −→k1 B.

The mass and energy balance equations (model) are

dc F0 (c0 − c) − E = − k e( RT ), dt πr2h 0 dT F0 (T0 − T ) ∆H E 2Uh (− RT ) = 2 − k0e + (T0 − T ) , dt πr h ρCp rρCp dh F − F = 0 , (11.1) dt πr2 where

h is the level of the tank,

146 c is the molar concentration of A,

T is the reactor temperature,

F is the outlet ﬂow rate,

Tc is the coolant liquid temperature,

F0 is the inlet ﬂow rate (disturbance).

11.3.1 Control design objective and settings

For the MPC controller, we will linearize the model with the following steady-state operating points (from [11] and [13])

cs = 0.878kmol/m3,T s = 324.5K, hs = 0.659m

s s 3 Tc = 300K,F = 0.1m /min and its parameters are shown in Table 11.1.

Table 11.1: Physical parameters for the Continuous stirred tank reactor (CSTR), refer to [13].

Parameter Units Value

3 F0 m /min 0.1

T0 K 350

3 c0 kmol/m 1.0

r m 0.219

−1 10 k0 min 7.2 × 10

E/R K 8750

U kJ/min ∗ m2 ∗ K 54.94

ρ kg/m3 1000

Cp kJ/kg ∗ K 0.239

∆H kJ/kmol −5 × 104

147 Therefore the linearized system with a sample time of t = 1s is

      0.9811 −0.0002 −0.0030 −2.8781×10−6 0.0002 1 0 0             A = 0.4668 0.9832 −0.6499 ,B =  0.0347 0.0360  ,C = 0 1 0 ,             0.0000 0.0000 1.0000 0.0000 −0.1106 0 0 1 where the states, inputs (MV), controlled outputs (CV) and disturbance (DV) are respectively

  c − cs       s s Tc − Tc c − c  s s x = T − T  , u =   , r =   , p = F0 − F0 .    s   s   F − F h − h h − hs

Here, contrary to the Flotation-cell system, there are fewer inputs nu = 2 than measured outputs ny = 3. Hence, the additional matrices required for the augmented system (10.9) are deﬁned as (from [13])

    0.0000 0.0000 0.0002 1 0 0           1 0 0 Bd = 0.0000 0.0000 0.0360  ,Cd = 0 0 0 ,H =   .           0 0 1 0.0000 0.0000 −0.1106 0 1 0

It is simple to validate the condition of (10.15), namely rank = nx +ny = 6. The weighting matrices for the steady-state optimization problem (10.11) are Qs = diag ([1.0, 0.0, 1.0]) and Rs = diag ([1.0, 1.0]). The weighting matrices corresponding to the free-ﬁnal state MPC problem (10.8) are

Q = diag ([50, 0, 50]) ,

R = diag ([500, 500]) ,

S = diag 2×104, 2×104 , and

  1.2153 −0.0049 −0.0103   3   P = 1×10 · −0.0049 47.85×10−6 −0.0003 ,     −0.0103 −0.0003 1.4608

which is the solution of the inﬁnite-horizon LQR. Also, the lower bound inputs are 0≤uk.

148 11.3.2 Simulation

The ﬁrst step prior to the MPC implementation on the real-time controller is to carry out a simulation. We use the same controller as the one designed in subsection 10.3.2, the only diﬀerence is the plant. The CSTR system is programmed in a S-function (see Appendix

A.7). The closed-loop diagram is depicted in Figure 11.12.

Figure 11.12: Closed-loop digital MPC controller for a CSTR reactor. The MPC controller is based on the control diagram in Figure 10.1 using a Matlab® function. For the plant, an S-function is coded for the nonlinear third-order diﬀerential equations (11.1).

The simulation runs for T = 50min with a horizon of N = 5. For the MPC design

3 3 criteria, we ﬁrst introduce a disturbance (inlet ﬂow F0) from 0.1m /min to 0.11m /min at 15min. Then, at 30min, two setpoint changes are carried out: (i) the concentration c to 0.9248kmol/m3 and (ii) the level tank h to 0.5931m. The result is shown in Figure

11.13.

149 ) 0.95 3

0.9

c (kmol/m 0.85 0 5 10 15 20 25 30 35 40 45 50

325 T (K)

320 0 5 10 15 20 25 30 35 40 45 50

0.7

h (m) 0.6

0 5 10 15 20 25 30 35 40 45 50 time (min)

(a) Measured outputs y.

300

298

296 Tc (K)

294

0 5 10 15 20 25 30 35 40 45 50

0.14

0.12 /min) 3 0.1 F (m

0.08 0 5 10 15 20 25 30 35 40 45 50 time (min)

(b) Manipulated variables u.

Figure 11.13: Closed-loop MPC simulation for a Continuous stirred tank reactor (CSTR) using the optimization solver presented in Algorithm 7.1. The measured outputs y correspond to the molar concentration c, the reactor temperature T and the level tank h. The inputs u are the outlet ﬂow F and the temperature Tc of the jacketed cooling system.

The result shows that the MPC controller based on the Hamiltonian recursion method attenuates the disturbance for the controlled variables c and h (the reactor temperature T is not controlled), and also drives them to their new setpoints when required. Therefore, the next step is to generate the code for the PLC in order to run a Hardware-in-the-loop

(HIL) test.

150 11.3.3 Real-time controller test

For this experiment, we ﬁrst carry out the code generation process presented in section

10.4. Once the MPC function is developed, the CODESYS program is set up; and then the run-time ﬁles are created, as depicted in Figure 11.14, which shows the total code size.

Figure 11.14: Image captured from CODESYS showing the size of the code after generating the runtime ﬁles for the MPC controller.

In addition, tailor-made screens were developed on the Festo HMI touch panel for the

Continuous stirred tank reactor (CSTR), such as the one shown in Figure 11.15.

Figure 11.15: Image captured from the Festo HMI touch screen device showing real- time performance of the main tailor-made screen for the Continuous stirred tank reactor (CSTR).

The real test experiment runs for T = 3104s∼51min with a horizon of N = 5. For the

3 MPC design criteria, the disturbance (inlet ﬂow F0) is introduced at 659s from 0.1m /min to 0.11m3/min. Then, a change of the concentration c setpoint is carried out at 1676s, from 0.8808kmol/m3 to 0.925kmol/m3. Finally, a change of the level tank h setpoint is

151 carried out at 2527s, from 0.6591m to 0.5929m. The result is shown in Figure 11.16.

) 0.95 3

0.9

c (kmol/m 0.85 0 500 1000 1500 2000 2500 3000

326

322 T (K) 318 0 500 1000 1500 2000 2500 3000

0.7

h (m) 0.6

0 500 1000 1500 2000 2500 3000 Time (sec)

(a) Measured outputs y.

300

298

296 Tc (K)

294

0 500 1000 1500 2000 2500 3000

0.14

0.12 /min) 3 0.1 F (m

0.08 0 500 1000 1500 2000 2500 3000 Time (sec)

(b) Manipulated variables u.

Figure 11.16: Real-time closed-loop MPC controller experiment for a Continuous stirred tank reactor (CSTR) using the optimization solver presented in Algorithm 7.1. The measured outputs y correspond to the molar concentration c, the reactor temperature T and the level tank h. The inputs u are the outlet ﬂow F and the temperature Tc of the jacketed cooling system.

This test shows that the MPC controller successfully drives the controlled variables r

(c and h) to their corresponding setpoints and it also attenuates the disturbance coming from the inlet ﬂow. This conﬁrms that the Hamiltonian recursion method is versatile, i.e. it delivers the optimal control input u∗ throughout the entire experiment as with the

Flotation-cell system test. Thus, it renders the method a good option for optimization solvers of real-time MPC controllers embedded in small-scale platforms, such as PLCs.

152 Chapter 12

Conclusions of part III

During this last part of the thesis, we validated the implementation of the novel Hamil- tonian recursion method (presented in chapter 7) within a real-time MPC algorithm on small-scale devices, named Programmable Logic Controllers (PLCs). Such platforms were chosen because, as mentioned previously, they are found from small-scale (normally inexpensive) to large-scale applications in all kind of industries. As opposed to more expensive platforms, such as FPGAs (which require a large and time-consuming development) that are barely widespread in industry.

Likewise, this topic has been rarely explored in research (see Figure 9.5). Therefore, we believe that there certainly exists an interesting niche for the research community to analyze, develop and provide innovation on MPC algorithms and optimization solver tools exclusively tailored for applications where PLCs are in charge of the control task. This is something that the Industry 4.0 will eventually demand.

12.1 Further work

Even though the feasibility of the MPC controller on PLCs was shown in this part, there exist a couple of points that remain to be further analyzed towards the implementation on real industrial applications. These points are explained as follows,

There are some downsides to using the code generation process shown in this part.

First, it depends on a proprietary toolbox (Simulink PLC Coder); and second, the

development engineer is unable to have control on how the code is translated, thus

producing variations among equivalent methods (the Hamiltonian recursion and the

153 Riccati recursion) presented in subsection 11.2.2. Therefore, the next step would

be to use a lower level language such as C++, which would enable a more ad hoc

object-oriented programming for the quadratic programming framework (see [28]).

In addition to this, since the size and eﬃciency of the MPC algorithms are important

for PLCs, employing a low level language would allow the direct use of algebra

libraries, such as BLAS and LAPACK, rendering the code more eﬃcient. In the

case of the BLAS libraries, its implementation would speed up the matrix-matrix

and matrix-vector operations that are required throughout the entire optimization

solver. In the case of the LAPACK libraries, they would provide well-tested and

reliable (and also open-source) routines for the factorization of a linear system of

equations. Although the optimization solvers presented in the thesis require mostly

basic matrix operations, in some parts this factorization is needed. In the case of the

Riccati recursion, the term (R + B|Pk+1B) is factorized every k iteration; whereas with the Hamiltonian recursion, for tracking problems only, the factorization of the

−1 | term A − BRk D is needed for every iteration.

The code generated by the toolbox in the previous examples was horizon-ﬁxed N.

Namely, the toolbox requires to set the size of the problem prior to the generation.

This renders the commissioning and testing of the algorithm very time consuming.

A next step should be the design of an algorithm in which the horizon N is able to

change rapidly. This is something that the use of low-level languages might facilitate.

This will deﬁnitely cause a compromise between the size of the PLC memory and

the horizon N. Therefore, an additional code that can apply a dynamic memory

allocation strategy will be required.

It is well-known that one of the reasons putting oﬀ the widespread use of MPC

controllers in industry is that they depend on a reliable and good mathematical

representation of the system. Therefore, it is also important to develop tools that

can provide such models. System identiﬁcation tools are essential to provide oﬄine

representations that are based on input/output data. We also believe that the

implementation of supervised machine-learning algorithms is an interesting niche

in parallel with real-time MPC controllers. These algorithms are able to analyze

real-time data and adjust for possible discrepancies on the model used by the MPC

controller. A recent work on machine learning and MPC can be found in [115].

154 Appendix A

Relevant code for parts II and III

A.1 HQ preconditioner function

Function that computes the ﬁrst part of Algorithm 7.1. It carries out a forward substitution of the preconditioner HQ (7.4).

1 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

2 %% Function for forward substitution ofHQ preconditioner

3 %% Version1.0 August 2018

4 %% Programmed by Eduardo Poupard

5 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

7 function pmq=fun hq preconditioner(pmq,sze)

9 for k=coder.unroll(1:sze.N+1)

10 %------

11 %Extracting Data

12 ifk < sze.N+1

13 pmq.bk0=-pmq.rwv((k-1)*pmq.nxu+1:k*pmq.nxu,1);

14 pmq.bk1=-pmq.ryv((k-1)*pmq.nxu+1:k*pmq.nxu,1);

15 else

16 pmq.bk1=-pmq.rx(sze.N*(pmq.nxu)+1:(sze.N+1)*(pmq.nxu),1);

17 end

18 %------

19 %Forward substitution forHQ preconditioner

20 if k == 1

21 pmq.qk0=pmq.bk0;

155 22 pmq.qk1=pmq.Ainvt*pmq.bk1;

23 else

24 pmq.Rinvk=pmq.Rinvm((k-2)*(pmq.nud)+1:(k-1)*(pmq.nud),:);

25 ifk < sze.N+1

26 pmq.Qk=pmq.Qm((k-1)*(pmq.nxu)+1:k*(pmq.nxu),:);

27 pmq.gbk1a=pmq.Ba'*pmq.qkp1;

28 pmq.gbk1b=pmq.Rinvk*pmq.gbk1a;

29 pmq.gbk1=pmq.Ba*pmq.gbk1b;

30 pmq.qk0a=pmq.qkp0-pmq.gbk1;

31 pmq.qk0b=pmq.Aa*pmq.qk0a;

32 pmq.qk0=pmq.bk0+pmq.qk0b;

33 pmq.qk1a=pmq.qkp0-pmq.gbk1;

34 pmq.qk1b=-pmq.Qk*pmq.qk1a;

35 pmq.qk1c=pmq.bk1+pmq.qkp1;

36 pmq.qk1d=pmq.qk1b+pmq.qk1c;

37 pmq.qk1=pmq.Ainvt*pmq.qk1d;

38 else

39 pmq.Pk=pmq.PN;

40 pmq.gbk1a=pmq.Ba'*pmq.qkp1;

41 pmq.gbk1b=pmq.Rinvk*pmq.gbk1a;

42 pmq.gbk1=pmq.Ba*pmq.gbk1b;

43 pmq.qk1a=pmq.qkp0-pmq.gbk1;

44 pmq.qk1b=-pmq.Pk*pmq.qk1a;

45 pmq.qk1c=pmq.bk1+pmq.qkp1;

46 pmq.qk1=pmq.qk1b+pmq.qk1c;

47 end

48 end

49 %------

50 %Saving Data

51 pmq.qkp0=pmq.qk0;

52 pmq.qkp1=pmq.qk1;

53 ifk < sze.N+1

54 pmq.gm(2*(k-1)*(pmq.nxu)+1:2*k*(pmq.nxu),:)=[pmq.qk0;pmq.qk1];

55 else

56 pmq.gm(2*(k-1)*(pmq.nxu)+1:(2*sze.N+1)*pmq.nxu,:)=pmq.qk1;

57 end

58 end

60 end

156 A.2 D˜ hqr companion-like matrix recursion function

Function that computes the matrix recursion (7.8), which corresponds to the second part of Algorithm 7.1.

1 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

2 %% Function for companion-like matrix recursion(novel method)

3 %% Version1.0 August 2018

4 %% Programmed by Eduardo Poupard

5 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

7 function pmq=fun dc companion handle(pmq,sze)

9 for k=coder.unroll(1:sze.N+1)

10 %------

11 %Main matrix computation

12 ifk < sze.N+1

13 %------

14 %Extracting Data

15 if k == 1

16 %------

17 %Coefficient Computation

18 pmq.ck0=pmq.zx;

19 pmq.ck1=pmq.Ainvt;

20 %Vector Elements Computation

21 pmq.wk0=-pmq.ck0;

22 pmq.wk1=-pmq.ck1;

23 else

24 %------

25 %Extracting Data

26 pmq.Qk=pmq.Qm((k-1)*(pmq.nxu)+1:k*(pmq.nxu),:);

27 pmq.Rinvk=pmq.Rinvm((k-2)*pmq.nud+1:(k-1)*pmq.nud,:);

28 pmq.ckp0=pmq.ck0;

29 pmq.ckp1=pmq.ck1;

30 %------

31 %Coefficient Computation

32 pmq.gbk1am=pmq.Ba'*pmq.ckp1;

33 pmq.gbk1bm=pmq.Rinvk*pmq.gbk1am;

34 pmq.gbk1m=pmq.Ba*pmq.gbk1bm;

35 pmq.ck0a=pmq.ckp0-pmq.gbk1m;

157 36 pmq.ck0=pmq.Aa*pmq.ck0a;

37 pmq.ck1a=pmq.ckp0-pmq.gbk1m;

38 pmq.ck1b=pmq.Qk*pmq.ck1a;

39 pmq.ck1c=pmq.ck1b-pmq.ckp1;

40 pmq.ck1=-pmq.Ainvt*pmq.ck1c;

41 %Vector Elements Computation

42 pmq.wk0=-pmq.ck0;

43 pmq.wk1=-pmq.ck1;

44 end

45 else

46 %------

47 %Extracting Data

48 pmq.Pk=pmq.PN;

49 pmq.Rinvk=pmq.Rinvm((k-2)*pmq.nud+1:(k-1)*pmq.nud,:);

50 pmq.ckp0=pmq.ck0;

51 pmq.ckp1=pmq.ck1;

52 %------

53 %Coefficient Computation

54 pmq.gbk1am=pmq.Ba'*pmq.ckp1;

55 pmq.gbk1bm=pmq.Rinvk*pmq.gbk1am;

56 pmq.gbk1m=pmq.Ba*pmq.gbk1bm;

57 pmq.ck1a=pmq.ckp0-pmq.gbk1m;

58 pmq.ck1b=-pmq.Pk*pmq.ck1a;

59 pmq.ck1=pmq.ck1b+pmq.ckp1;

60 %Vector Elements Computation

61 pmq.wk1=-pmq.ck1;

62 end

63 %------

64 %Saving Data

65 ifk < sze.N+1

66 pmq.weg(2*(k-1)*(pmq.nxu)+1:2*k*(pmq.nxu),:)=[pmq.wk0;pmq.wk1];

67 else

68 pmq.weg((2*sze.N)*pmq.nxu+1:(2*sze.N+1)*pmq.nxu,:)=pmq.wk1;

69 end

70 end

71 end

158 A.3 D˜ hqr companion-like vector recursion function

Function that computes the vector recursion, which corresponds to the third part of Al- gorithm 7.1.

1 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

2 %% Function for computation of companion matrix for GMRES(own method)

3 %% Version1.0 August 2018

4 %% Programmed by Eduardo Poupard

5 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

7 function pmq=fun dc companion residuals(pmq,sze)

9 %Computing final condition

10 pmq.wk1=pmq.weg((2*sze.N)*pmq.nxu+1:(2*sze.N+1)*pmq.nxu,:);

11 pmq.ekN=pmq.gm((2*sze.N)*pmq.nxu+1:(2*sze.N+1)*pmq.nxu,1);

12 pmq.zkN=pmq.wk1\pmq.ekN;

13 pmq.dyNEW((2*sze.N)*pmq.nxu+1:(2*sze.N+1)*pmq.nxu,1)=pmq.zkN;

14 %Forward substitution

15 for k=coder.unroll(1:sze.N)

16 %------

17 %Extracting Data

18 pmq.ekd=pmq.gm(2*(k-1)*(pmq.nxu)+1:2*k*(pmq.nxu),:);

19 pmq.ek0=pmq.ekd(1:pmq.nxu,:);

20 pmq.ek1=pmq.ekd(pmq.nxu+1:2*pmq.nxu,:);

21 pmq.wegd=pmq.weg(2*(k-1)*(pmq.nxu)+1:2*k*(pmq.nxu),:);

22 pmq.wegk0=pmq.wegd(1:pmq.nxu,:);

23 pmq.wegk1=pmq.wegd(pmq.nxu+1:2*pmq.nxu,:);

24 %------

25 %Coefficient Computation

26 pmq.zk0=pmq.ek0-pmq.wegk0*pmq.zkN;

27 pmq.zk1=pmq.ek1-pmq.wegk1*pmq.zkN;

28 %------

29 %Saving Data

30 pmq.dyNEW(2*(k-1)*(pmq.nxu)+1:2*k*(pmq.nxu),:)=[pmq.zk0;pmq.zk1];

32 end

33 end

159 A.4 HR preconditioner function

Function that computes the fourth part of Algorithm 7.1. It carries out a backward substitution of the preconditioner HR (7.3).

1 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

2 %% Function for backward substitution ofHR preconditioner

3 %% Version1.0 August 2018

4 %% Programmed by Eduardo Poupard

5 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

7 function pmq=fun hr preconditioner(pmq,sze)

9 for k=coder.unroll(1:sze.N+1)

10 %------

11 %Extracting Data

12 if k == 1

13 pmq.bk1=pmq.dyNEW((2*sze.N)*pmq.nxu+1:(2*sze.N+1)*pmq.nxu,:);

14 else

15 pmq.bkd=pmq.dyNEW(2*(sze.N-k+1)*pmq.nxu+1:2*(sze.N-k+2)*pmq.nxu,:);

16 pmq.bk1=pmq.bkd(pmq.nxu+1:2*pmq.nxu,:);

17 pmq.bk0=pmq.bkd(1:pmq.nxu,:);

18 end

19 %------

20 %Backward substitution forHR preconditioner

21 if k == 1

22 pmq.qk1=pmq.bk1;

23 else

24 pmq.Rinvk=pmq.Rinvm((sze.N-k+1)*pmq.nud+1:(sze.N-k+2)*pmq.nud,:);

25 pmq.qk1=pmq.bk1;

26 pmq.gbk1a=pmq.Ba'*pmq.qk1;

27 pmq.gbk1b=pmq.Rinvk*pmq.gbk1a;

28 pmq.gbk1=pmq.Ba*pmq.gbk1b;

29 pmq.qk0=pmq.bk0-pmq.gbk1;

30 end

31 %------

32 %Saving Data

33 if k == 1

34 pmq.dxlNEW((2*sze.N)*pmq.nxu+1:(2*sze.N+1)*pmq.nxu,:)=pmq.qk1;

35 else

160 36 pmq.dxlNEW(2*(sze.N-k+1)*pmq.nxu+1:2*(sze.N-k+2)*pmq.nxu,:)=...

37 [pmq.qk0;pmq.qk1];

38 end

39 end

40 end

A.5 Application of GMRES method for companion-like matrix

D˜ hqr

Function that computes second part of Algorithm 7.3, which corresponds to the GMRES method with matrix-free operations.

1 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

2 %% Function for computation of companion matrix for GMRES(own method)

3 %% Version1.0 August 2018

4 %% Programmed by Eduardo Poupard

5 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

7 function pmq=fun dc companion handle polished(pmq,sze)

9 %GMRES funtion

10 weg=ourgmres(@symplectic mpc,pmq.gm,[],1e-30,pmq.nxu+1,[],[],[]);

12 pmq.dyNEW=weg;

14 function weg=symplectic mpc(x)

16 for k=coder.unroll(1:sze.N+1)

17 %------

18 %Main matrix computation

19 ifk < sze.N+1

20 %------

21 %Extracting Data

22 pmq.ekd=x(2*(k-1)*(pmq.nxu)+1:2*k*(pmq.nxu),:);

23 pmq.ek0=pmq.ekd(1:pmq.nxu,:);

24 pmq.ek1=pmq.ekd(pmq.nxu+1:2*pmq.nxu,:);

25 if k == 1

26 %------

161 27 %Extracting Data

28 pmq.ekN=x((2*sze.N)*pmq.nxu+1:(2*sze.N+1)*pmq.nxu,1);

29 %------

30 %Coefficient Computation

31 pmq.ck0=pmq.zx*pmq.ekN;

32 pmq.ck1=pmq.Ainvt*pmq.ekN;

33 %Vector Elements Computation

34 pmq.wk0=pmq.ek0-pmq.ck0;

35 pmq.wk1=pmq.ek1-pmq.ck1;

36 else

37 %------

38 %Extracting Data

39 pmq.Qk=pmq.Qm((k-1)*(pmq.nxu)+1:k*(pmq.nxu),:);

40 pmq.Rinvk=pmq.Rinvm((k-2)*pmq.nud+1:(k-1)*pmq.nud,:);

41 pmq.ckp0=pmq.ck0;

42 pmq.ckp1=pmq.ck1;

43 %------

44 %Coefficient Computation

45 pmq.gbk1a=pmq.Ba'*pmq.ckp1;

46 pmq.gbk1b=pmq.Rinvk*pmq.gbk1a;

47 pmq.gbk1=pmq.Ba*pmq.gbk1b;

48 pmq.ck0a=pmq.ckp0-pmq.gbk1;

49 pmq.ck0=pmq.Aa*pmq.ck0a;

50 pmq.ck1a=pmq.ckp0-pmq.gbk1;

51 pmq.ck1b=pmq.Qk*pmq.ck1a;

52 pmq.ck1c=pmq.ck1b-pmq.ckp1;

53 pmq.ck1=-pmq.Ainvt*pmq.ck1c;

54 %Vector Elements Computation

55 pmq.wk0=pmq.ek0-pmq.ck0;

56 pmq.wk1=pmq.ek1-pmq.ck1;

57 end

58 else

59 %------

60 %Extracting Data

61 pmq.Pk=pmq.PN;

62 pmq.Rinvk=pmq.Rinvm((k-2)*pmq.nud+1:(k-1)*pmq.nud,:);

63 pmq.ckp0=pmq.ck0;

64 pmq.ckp1=pmq.ck1;

65 %------

66 %Coefficient Computation

67 pmq.gbk1a=pmq.Ba'*pmq.ckp1;

162 68 pmq.gbk1b=pmq.Rinvk*pmq.gbk1a;

69 pmq.gbk1=pmq.Ba*pmq.gbk1b;

70 pmq.ck1a=pmq.ckp0-pmq.gbk1;

71 pmq.ck1b=-pmq.Pk*pmq.ck1a;

72 pmq.ck1=pmq.ck1b+pmq.ckp1;

73 %Vector Elements Computation

74 pmq.wk1=-pmq.ck1;

75 end

76 %------

77 %Saving Data

78 ifk < sze.N+1

79 weg(2*(k-1)*(pmq.nxu)+1:2*k*(pmq.nxu),1)=[pmq.wk0;pmq.wk1];

80 else

81 weg((2*sze.N)*pmq.nxu+1:(2*sze.N+1)*pmq.nxu,1)=pmq.wk1;

82 end

83 end

84 end

85 end

A.6 S-Function for Flotation-cell system

Nonlinear model for a Flotation-cell system for three tanks using an S-function from Matlab®.

1 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

2 %%S-Function for flotation-cell system(Non-linear model for three-tanks)

3 %% Version 02.00(August 2018)

4 %% Programmed by Eduardo Poupard

5 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

6 function [sys,x0,str,ts] = sfun flotation system 02(~,x,u,flag,par)

7 switch flag

8 case0%Initialization

9 sizes = simsizes;

10 sizes.NumContStates = 3;

11 sizes.NumDiscStates = 0;

12 sizes.NumOutputs = 3;

13 sizes.NumInputs = 4;

14 sizes.DirFeedthrough = 0;

163 15 sizes.NumSampleTimes = 1;

16 sys=simsizes(sizes);

17 x0=par.xs;

18 str=[];

19 ts=[0,0];

20 case1%Derivatives

21 %States

22 ht1=x(1);%Current tank level 1(m)

23 ht2=x(2);%Current tank level 2(m)

24 ht3=x(3);%Current tank level 3(m)

25 %Inputs

26 ut3=u(1);%Valve command for tank3

27 ut2=u(2);%Valve command for tank2

28 ut1=u(3);%Valve command for tank1

29 qmi=u(4);%Main inlet flowrate(mˆ3/s)--Input Disturbance--

31 %Flotation-Cell System Nonlinear model(single tank)

33 dht1dt=(1/par.As)*(qmi-(par.Cv*ut1*sqrt((ht1-ht2+par.lm)/par.sg)));

36 dht2dt=(1/par.As)*((par.Cv*ut1*sqrt((ht1-ht2+par.lm)/par.sg))-...

37 (par.Cv*ut2*sqrt((ht2-ht3+par.lm)/par.sg)));

39 dht3dt=(1/par.As)*((par.Cv*ut2*sqrt((ht2-ht3+par.lm)/par.sg))-...

40 (par.Cv*ut3*sqrt(ht3/par.sg)));

42 sys=[dht1dt;dht2dt;dht3dt];

43 case3%Outputs

45 sys = [x(3);x(2);x(1)];

46 case {2,4,9}

47 sys=[];

48 otherwise

49 error(['unhandled flag=',num2str(flag)]);

50 end

164 A.7 S-Function for Continuous stirred tank reactor (CSTR)

S-function for the nonlinear model of a well-stirred Continuous stirred tank reactor (CSTR).

1 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

2 %%S-Function runs CSTR non-linear model

3 %% Model from Rawlings's book p. 53

4 %% Version 02.00(November 2017)

5 %% Programmed by Eduardo Poupard

6 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

8 function [sys,x0,str,ts] = sfun cstr 02(~,x,u,flag,par)

10 switch flag

12 case0%Initialization

13 sizes = simsizes;

14 sizes.NumContStates = 3;

15 sizes.NumDiscStates = 0;

16 sizes.NumOutputs = 3;

17 sizes.NumInputs = 3;

18 sizes.DirFeedthrough = 0;

19 sizes.NumSampleTimes = 1;

20 sys=simsizes(sizes);

21 x0=par.xs;

22 str=[];

23 ts=[0,0];

25 case1%Derivatives

26 %States

27 c=x(1);%Molar concentration inA(kmol/mˆ3)

28 T=x(2);%Reactor temperature(K)

29 h=x(3);%Level of the tank(m)

30 %Inputs

31 Fo=u(1);%Coolant liquid temperature(K)

32 Tc=u(2);%Outlet flowrate(mˆ3/min)

33 Fi=u(3);%Inlet flowrate(mˆ3/min)-Disturbance-

35 %CSTR Nonlinear model

165 36 dcdt=((Fi*(par.c0-c))/(pi*(par.rˆ2)*h))-(par.k0*c*exp(-par.ER/T));

37 dTdt=((Fi*(par.T0-T))/(pi*(par.rˆ2)*h))+...

38 ((-par.dH/(par.rho*par.Cp))*par.k0*c*exp(-par.ER/T))+...

39 ((2*par.U*(Tc-T))/(par.r*par.rho*par.Cp));

40 dhdt=((Fi-Fo)/(pi*(par.rˆ2)));

42 sys=[dcdt;dTdt;dhdt];

44 case3% Outputs

45 sys=[x(3);x(2);x(1)];

47 case {2,4,9}

48 sys=[];

50 otherwise

51 error(['unhandled flag=',num2str(flag)]);

52 end

166 Appendix B

Double-tank model

In industry, apart from the single-tank array for ﬂotation-cell systems, there exists another type of system, the double-tank array depicted in Figure B.1

hi1 1 2 hi hi

hi1

qi1 qi s q qi1

Figure B.1: Flotation-cell system for three tanks connected in series (a single-tank array in each side and a double-tank array in the middle).

We take the double-tank array in the middle for the analysis of the mathematical

1 2 model. By conservation of mass, the relationship between its levels hi , hi , the input ﬂow qi−1 and output ﬂow qi is

˙1 Ashi = qi−1 − qF ,

˙2 Ashi = qF − qi, (B.1)

1 2 ˙1 dhi ˙2 dhi where qF is the ﬂow across the ﬂange that connects both tanks, hi = dt , hi = dt , and

As is the area of the tanks. As in the single-tanks, here the input ﬂow qi−1 and the output

ﬂow qi are controlled by valves. Their relationship with the system, as well as qF , is also

167 explained by Bernoulli’s equation

s h − h1 + λ q = C v i−1 i s , i−1 v i−1 s.g. q 1 2 qF = αc 2g hi − hi , s h2 − h + λ q = C v i i+1 s , (B.2) i v i s.g.

where g is the gravitational acceleration and αc is the cross-sectional area of the ﬂange. Everything else holds as in the single-tank. Therefore, system (B.2) becomes

s 1 q ˙1 Cvvi−1 hi−1 − hi + λs αc 1 2 hi = − 2g hi − hi , As s.g. As s q 2 ˙2 αc 1 2 Cvvi hi − hi+1 + λs hi = 2g hi − hi − . (B.3) As As s.g.

System (B.3) is a nonlinear second order diﬀerential equation that we will now linearize

˙1 ˙2 around an operating point. Let hi and hi be deﬁned as

˙1 1 2 hi = f1 hi , hi , vi, vi−1 ,

˙2 1 2 hi = f2 hi , hi , vi, vi−1 , and also deﬁning a new set of coordinates

 1 ¯1 x1 = hi − hi  States 2 ¯2  x2 = hi − hi  ui = vi − v¯i Input ui−1 = vi−1 − v¯i−1 Input Disturbance 2 ¯2 Output (Level Sensor) yi = hi − hi

¯1 ¯2 Therefore, we now carry out around the operating point hi , hi , v¯i, v¯i−1 , the ﬁrst order

168 1 2 1 2 approximation of Taylor expansion of f1 hi , hi , vi, vi−1 and f2 hi , hi , vi, vi−1

   ¯1 ¯2 ¯1 ¯2    ∂f1(hi ,hi ,v¯i,v¯i−1) ∂f1(hi ,hi ,v¯i,v¯i−1) x˙1 ∂h1 ∂h2 x1    i i    = ¯1 ¯2 ¯1 ¯2 (B.4)    ∂f2(hi ,hi ,v¯i,v¯i−1) ∂f2(hi ,hi ,v¯i,v¯i−1)   x˙2 1 2 x2 ∂hi ∂hi  ¯1 ¯2 ¯ ¯2    ∂f1(hi ,hi ,v¯i,v¯i−1) ∂f1(hi,hi ,v¯i,v¯i−1) ui  ∂vi ∂vi−1    + ¯1 ¯2 ¯ ¯2 ,  ∂f2(h ,h ,v¯i,v¯i−1) ∂f2(hi,h ,v¯i,v¯i−1)   i i i u ∂vi ∂vi−1 i−1   x  1 yi = 0 1   . (B.5) x2

Computing the Jacobians, system (B.5) becomes

          x˙ −a a x 0 K u  1  11 12   1  i−1  i    =     +     , x˙2 a21 −a22 x2 −Ki 0 ui−1   x  1 yi = 0 1   , (B.6) x2 where

  s Cv v¯i−1 αc 2g a = √ + , (B.7) 11 q  ¯1 ¯2 2As s.g. ¯ ¯1 2As h − h hi−1 − hi + λs i i s α 2g a = c , (B.8) 12 2A ¯1 ¯2 s hi − hi s α 2g a = c , (B.9) 21 2A ¯1 ¯2 s hi − hi   s Cv v¯i−1 αc 2g a = √ + , (B.10) 22 q  ¯1 ¯2 2As s.g. ¯2 ¯ 2As h − h hi − hi+1 + λs i i q Cv ¯ ¯1 Ki−1 = √ hi−1 − hi + λs, (B.11) As s.g. q Cv ¯2 ¯ Ki = √ hi − hi+1 + λs. (B.12) As s.g.

We now derive the Laplace Transform for system (B.6), with null initial conditions. In order to do so, it is well-known that the state-space representation (B.6) can be expressed in frequency domain (in terms of the output, yi, as a function of the inputs, ui and ui−1),

169 using the following relationship [116]

    −1     s 0 −a a 0 K u (s)     11 12   i−1  i  yi(s) =  0 1   −       , (B.13) 0 s a21 −a22 −Ki 0 ui−1(s) and after a few algebraic operations so as to simplify previous expression, the double-tank system in frequency domain is

K a K (s + a ) y (s) = i−1 21 u (s) − i 11 u (s), (B.14) i s2 + ρs + µ i−1 s2 + ρs + µ i where

ρ = a11 + a22,

µ = a11a22 − a12a21.

170 Bibliography

[1] A. Malyshev, R. Quirynen, and A. Knyazev, “Preconditioned Krylov iterations and condensing in interior point MPC method,” Proceedings of the 6th IFAC Conference on Nonlinear Model Predictive Control, pp. 478–483, 2018. [2] Pavel Holoborodko, Advanpix, “Multiprecision Computing Toolbox,” ver 4.5.1.12833. Yokohama, Japan. [3] K. Graichen, S. Hentzelt, A. Hildebrandt, N. Karcher, N. Gaissert, and E. Knubben, “Control design for a bionic kangaroo,” Control Engineering Practice, vol. 42, pp. 106–117, 2015. [4] J. Maciejowski, Predictive Control with Constraints. Prentice Hall, 2002. [5] G. V. Palomo and J. Rossiter, “Efficient Suboptimal Parametric Solutions to Predictive Control for PLC Applications,” Control Engineering Practice, pp. 732–743, 2011. [6] M. Kvasnica, I. Rauova, and M. Fikar, “Automatic Code Generation for Real-Time Implementation of Model Predictive Control,” 2010 IEEE International Symposium on Computer-Aided Control System Design, pp. 993–998, 2010. [7] G. Frison, D. K. M. Kufoalor, L. Imsland, and J. B. Jørgensen, “Efficient Implementation of Solvers for Linear Model Predictive Control on Embedded Devices,” Proceedings of IEEE Conference on Control Applications, pp. 1954–1959, 2014. [8] D. K. M. Kufoalor, S. Richter, L. Imsland, T. A. Johansen, M. Morari, and G. O. Eikrem, “Embedded Model Predictive Control on a PLC using a primal-dual first-order Method for a Subsea Separation Process,” Proceedings of the 22nd Mediterranean Conference on Control and Automation, pp. 368– 373, 2014. [9] B. Huyck, H. J. Ferreau, M. Diehl, J. D. Brabanter, J. F. M. V. Impe, B. D. Moor, and F. Logist, “Towards Online Model Predictive Control on a Programmable Logic Controller: Practical Consid- erations,” Hindawi Publishing Corporation, vol. Mathematical Problems in Engineering, 2012. [10] M. Pereira, D. Limon, D. Muñoz,and T. Alamo, “MPC implementation in a PLC based on Nes- terovs fast gradient method,” Proceedings of the 23rd Mediterranean Conference on Control and Automation, pp. 371–376, 2015. [11] G. Pannocchia and J. B. Rawlings, “Disturbance models for offset-free model-predictive control,” AIChE Journal, vol. 49, pp. 426–437, 2003. [12] B. W. Bequette, Process Control: Modeling, Design and Simulation. Prentice Hall, First Edition ed., 2003. [13] J. B. Rawlings and D. Q. Mayne, Model Predictive Control: Theory and Design. Nob Hill Publishing, LLC, 2009. [14] J. Richalet, A. Rault, J. Testud, and J. Papon, “Model algorithmic control of industrial processes,” IFAC Proceedings Volumes, vol. 10, no. 16, pp. 103–120, 1977. [15] C. R. Cutler and B. L. Ramaker, “Dynamic matrix control -a computer control algorithm,” AICHE national meeting, 1979. [16] S. J. Qin and T. A. Badgwell, “A survey of industrial Model Predictive Control technology,” Control Engineering Practice, pp. 733–764, 2003. [17] D. Deenen, E. Maljaars, L. Sebeke, B. de Jager, E. Heijman, H. Grüll,and W. Heemels, “Offset-free Model Predictive Control for enhancing MR-HIFU hyperthermia in cancer treatment,” Proceedings of the 6th IFAC Conference on Nonlinear Model Predictive Control, pp. 223–228, August 2018. [18] M. Diehl, H. J. Ferreau, and N. Haverbeke, Efficient Numerical Methods for Nonlinear MPC and Moving Horizon Estimation, vol. 384 of Lecture Notes in Control and Information Sciences. Springer, 2009. [19] M. Diehl, H. Bock, J. P. Schlöder,R. Findeisen, Z. Nagy, and F. Allgöwer, “Real-time optimization and nonlinear Model Predictive Control of processes governed by differential-algebraic equations,” Journal of Process Control, vol. 12, no. 4, pp. 577–585, 2002.

171 [20] M. Katliar, J. Fischer, G. Frison, M. Diehl, H. Teufel, and H. H. Bülthff,“Nonlinear Model Predictive Control of a Cable-robot-based motion simulator,” Proceedings of the 20th IFAC World Congress, vol. 50, pp. 9833–9839, July 2017. [21] C. Rao, S. Wright, and J. Rawlings, “Application of Interior-point Methods to Model Predictive Control,” Optimization Theory and Applications, vol. 99, pp. 723–757, 1998. [22] I. Nielsen and D. Axehill, “Low-Rank Modifications of Riccati Factorizations for Model Predictive Control,” IEEE Transactions on Automatic Control, vol. 63, no. 3, pp. 872–879, 2018. [23] J. V. Frasch, S. Sager, and M. Diehl, “A parallel quadratic programming method for dynamic optimization problems,” Mathematical Programming Computation, vol. 7, no. 3, pp. 289–329, 2015. [24] A. Domahidi and J. Jerez, “FORCES Professional.” embotech GmbH (http://embotech.com/ FORCES-Pro), July 2014. [25] B. Stellato, G. Banjac, P. Goulart, A. Bemporad, and S. Boyd, “OSQP: An operator splitting solver for quadratic programs,” ArXiv e-prints, Nov. 2017. [26] A. Domahidi, A. U. Zgraggen, M. N. Zeilinger, M. Morari, and C. N. Jones, “Efficient interior point methods for multistage problems arising in receding horizon control,” Proc. IEEE Conf. Decision Control, pp. 668–674, 2012. [27] B. Houska, H. Ferreau, and M. Diehl, “ACADO Toolkit-An Open Source Framework for Automatic Control and Dynamic Optimization,” Optimal Control Applications and Methods, vol. 32, no. 3, pp. 298–312, 2011. [28] E. M. Gertz and S. J. Wright, “Object-oriented software for quadratic programming,” ACM Trans- action on Mathematical Software, vol. 29, pp. 58–81, 2003. [29] M. G. Forbes, R. S. Patwardhan, H. Hamadah, and R. B. Gopaluni, “Model Predictive Control in Industry: Challenges and Opportunities,” 9th IFAC Symposium on Advanced Control of Chemical Processes ADCHEM 2015, pp. 531–538, June 2015. [30] K. Ling, S. Yue, and J. Maciejowski, “A FPGA implementation of Model Predictive Control,” IEEE Proceedings of the 2006 American Control Conference, pp. 1930–1935, June 2006. [31] J. L. Jerez, P. J. Goulart, S. Richter, G. A. Constantinides, E. C. Kerrigan, and M. Morari, “Em- bedded Online Optimization for Model Predictive Control at Megahertz Rates,” IEEE Transactions on Automatic Control, pp. 3238–3251, 2014. [32] B. Khusainov, E. C. Kerrigan, and G. A. Constantinides, “Automatic Software and Computing Hardware Codesign for Predictive Control,” IEEE Transactions on Control Systems Technology, pp. 1–10, 2018. [33] G. A. Constantinides, “Tutorial paper: Parallel architectures for Model Predictive Control,” Euro- pean Control Conference (ECC), pp. 138–143, 2009. [34] D. Mayne, J. Rawlings, C. Rao, and P. Scokaert, “Constrained Model Predictive Control: Stability and Optimality,” Automatica, vol. 36, pp. 789–814, 2000. [35] F. Borrelli, A. Bemporad, and M. Morari, Predictive Control for Linear and Hybrid Systems. Cam- bridge University Press, 2017. [36] F. L. Lewis, D. L. Vrabie, and V. L. Syrmos, Optimal Control. John Wiley and Sons INC, Third ed., 2012. [37] D. R. Vaughan, “A nonrecursive algebraic solution for the discrete Riccati equation,” IEEE Trans- actions on Automatic Control, pp. 597–599, 1970. [38] E. Poupard and W. P. Heath, “Hamiltonian Decomposition for Model Predictive Control,” Pro- ceedings of the 6th IFAC Workshop on Lagrangian and Hamiltonian Methods for Nonlinear Control, vol. 24, pp. 193–198, May 2018. [39] P. Y. Papalambros and D. J. Wilde, Principle of Optimal Design: Modeling and Computation. Cambridge University Press, second edition ed., 2000. [40] S. J. Wright, Primal-Dual Interior Point Methods. Society for Industrial and Applied Mathematics, 1997. [41] S. Mehrotra, “On the implementation of a Primal-Dual Interior-Point method,” Society for Industrial and Applied Mathematics, vol. 2:4, pp. 575–601, 1992. [42] E. Poupard, W. P. Heath, and S. Güttel,“A Hamiltonian Decomposition for Fast Interior-Point Solvers in Model Predictive Control,” Proceedings of the 6th IFAC Conference on Nonlinear Model Predictive Control, August 2018. [43] A. Bodrov, E. Poupard, W. P. Heath, and J. Apsley, “Minimum electrical energy use in a multi- actuator system,” 2017 IEEE International Electric Machines and Drives Conference (IEMDC), May 2017. [44] A. E. Bryson and Y. C. Ho, Applied Optimal Control; Optimization, Estimation and Control. Halsted Press Book, 1975.

172 [45] S. Boyd and L. Vandenberghe, Convex Optimization. Cambridge University Press, 2004. [46] J. Nocedal and S. J. Wright, Numerical Optimization. Springer, 2006. [47] C. D. Meyer, Matrix Analysis and Applied Linear Algebra. SIAM, 2004. [48] G. H. Golub and C. F. V. Loan, Matrix Computations. Johns Hopkins, fourth ed., 2013. [49] Kalman, R.E., The Theory of Optimal Control and the Calculus of Variations. University of Cali- fornia Press, Mathematical Optimization Techniques ed., 1963. [50] L. Pontryagin, Mathematical Theory of Optimal Processes. Classics of Soviet Mathematics, Taylor & Francis, 1987. [51] R. Kalman, “Contributions to the theory of optimal control,” Boletin de la Sociedad Mexicana de Matematicas, vol. 5, pp. 102–119, 1960. [52] A. G. J. MacFarlane, “An Eigenvector solution of the Optimal Linear Regulator problem,” Electron- ics and Control, vol. 14:6, pp. 643–654, 1963. [53] D. R. Vaughan, “A negative exponential solution for the linear optimal regulator problem,” Joint Automatic Control Conference preprints, pp. 717–725, 1968. [54] D. Kleinman, “Linear quadratic optimal estimation and control with receding horizon,” Electronics Letters, pp. 19–21, 1975. [55] Y. A. Thomas, “An easy way to stabilize a Linear Constant System,” IEEE Trans. Automat. Control, pp. 692–693, 1970. [56] S. Bittanti, A. J. Laub, and J. C. Willems, The Riccati Equation. Springer-Verlag, 1991. [57] G. Gaalman, “Comments on a nonrecursive algebraic solution for the discrete Riccati equation,” IEEE Transactions on Automatic Control, vol. 25, pp. 610–612, 1980. [58] T. Pappas, A. J. Laub, and N. R. Sandell, “On the numerical solution of the discrete-time algebraic Riccati equation,” IEEE Transactions on Automatic Control, vol. 25, pp. 631–641, 1980. [59] Y. Nesterov, Introductory Lectures on Convex Optimization: A Basic Course. Springer Science and Business Media, 2003. [60] R. Fletcher, Practical Methods of Optimization. John Wiley and Sons, Second ed., 1987. [61] H. J. Ferreau, H. G. Bock, and M. Diehl, “An online active set strategy to overcome the limitations of explicit MPC,” International Journal of Robust and Nonlinear Control, vol. 18, pp. 816–830, 2008. [62] A. G. Wills, D. Bates, A. J. Fleming, B. Ninness, and S. O. R. Moheimani, “Model Predictive Control applied to constraint handling in active noise and vibration control,” IEEE Transactions on Control Systems Technology, vol. 16, pp. 3–12, 2008. [63] S. Boyd, N. Parikh, E. Chu, B. Peleato, and J. Eckstein, “Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers,” Foundations and Trends in Machine Learning, pp. 1–122, 2011. [64] C. Danielson, “An Alternating Direction Method of Multipliers Algorithm for Symmetric MPC,” Proceedings of the 6th IFAC Conference on Nonlinear Model Predictive Control, vol. 51, no. 20, pp. 319–324, 2018. [65] A. G. Wills and W. P. Heath, “Barrier function based Model Predictive Control,” Automatica, vol. 40, pp. 1415–1422, 2004. [66] Y. Wang and S. Boyd, “Fast model predictive control using online optimization,” IEEE Transactions on Control Systems Technology, vol. 18, pp. 267–278, 2010. [67] M. Sznaier and M. J. Damborg, “Suboptimal control of linear systems with state and control inequality constraints,” Proceedings of the 25th Conference on Decision and Control, pp. 761–762, 1987. [68] D. Chmielewski and V. Manousiouthakis, “On constrained inﬁnite-time linear quadratic optimal control,” Systems and Control Letters, vol. 29, pp. 121–129, 1996. [69] P. O. M. Scokaert and J. B. Rawlings, “Constrained linear quadratic regulation,” IEEE Transactions on Automatic Control, vol. 43, pp. 1163–1169, 1998. [70] J. Gondzio, “Interior point methods 25 years later,” European Journal of Operational Research, p. 587601, 2011. [71] A. Bemporad, M. Morari, V. Dua, and E. N. Pistikopoulos, “The explicit linear quadratic regulator for constrained systems,” Automatica, vol. 38, pp. 3–20, 2002. [72] Y. Saad, Iterative Methods for Sparse Linear Systems. Philadelphia, PA, USA: Society for Industrial and Applied Mathematics, 2nd ed., 2003. [73] D. M. Young, “Convergence properties of the symmetric and unsymmetric successive overrelaxation methods and related methods,” Mathematics of Computation, pp. 793–807, 1970.

173 [74] H. A. Blanchard and A. A. Adegbege, “An SOR-like method for fast Model Predictive Control,” Preprints of the 20th World Congress The International Federation of Automatic Control, pp. 14418– 14423, July 2017. [75] H. A. Shukla, B. Khusainov, E. C. Kerrigan, and C. N. Jones, “Software and Hardware Code generation for Predictive Control using Splitting methods,” Preprints of the 20th World Congress The International Federation of Automatic Control, pp. 14386–14391, July 2017. [76] R. M. Levenson, Z. E. Nelson, and A. A. Adegbege, “Programmable Logic Controller for embedded implementation of input-constrained systems,” Preprints of the 20th World Congress The Interna- tional Federation of Automatic Control, pp. 14412–14417, July 2017. [77] J. Gondzio, “Matrix-free interior point method,” Computational Optimization and Applications, vol. 52, pp. 457–480, 2012. [78] A. Shahzad, E. C. Kerrigan, and G. A. Constantinides, “A stable and efficient method for solving a convex quadratic program with application to optimal control,” SIAM Journal on Optimization, vol. 22, pp. 1369–1393, 2012. [79] A. Malyshev, R. Quirynen, and A. Knyazev, “Preconditioning of conjugate gradient iterations in interior point MPC method,” Proceedings of the 6th IFAC Conference on Nonlinear Model Predictive Control, pp. 484–489, 2018. [80] T. Ohtsuka, “A continuation/GMRES method for fast computation of nonlinear receding horizon control,” Automatica, vol. 40, no. 4, pp. 563 – 574, 2004. [81] H. Fassbender, Symplectic Methods for the Symplectic Eigenproblem. Kluwer Academic Plenum Publishers, 2000. [82] R. A. Horn and C. R. Johnson, Matrix Analysis. New York, NY, USA: Cambridge University Press, 2nd ed., 2012. [83] Y. Saad and M. H. Schultz, “GMRES: A generalized minimal residual algorithm for solving non- symmetric linear systems,” SIAM Journal on Scientific and Statistical Computing, pp. 856–869, 1986. [84] A. Shahzad, E. C. Kerrigan, and G. A. Constantinides, “A fast well-conditioned interior point method for Predictive Control,” Proceedings of the IEEE Conference Decision and Control, pp. 508– 513, 2010. [85] S. Richter, C. N. Jones, and M. Morari, “Real-time input-constrained mpc using fast gradient methods,” Proceedings of the 48th IEEE Conference on Decision and Control, pp. 7387–7393, 2009. [86] S. Campbell, I. Ipsen, C.T.Kelley, and C. Meyer, “GMRES and the minimal polynomial,” BIT Numerical Mathematics, vol. 36, pp. 664–675, 1996. [87] “Oscillating Masses example.” http://web.stanford.edu/~boyd/fast_mpc/masses_example.m. Ac- cessed: 2018-10-14. [88] A. Malyshev, R. Quirynen, A. Knyazev, and S. D. Cairano, “A regularized Newton solver for linear Model Predictive Control,” Proceedings of the European Control Conference (ECC), 2018. [89] M. H. Wright, “Ill-conditioning and computational error in interior methods for nonlinear programming,” SIAM Journal on Optimization, vol. 9, pp. 84–111, 1998. [90] E. Andersen, J. Gondzio, C. Mészáros,and X. X, Implementation of interior point methods for large scale linear programming, vol. 5. Springer, 1998. [91] S. J. Wright, “Modified cholesky factorizations in interior-point algorithms for linear programming,” SIAM Journal on Optimization, vol. 9, pp. 1159–1191, 1999. [92] T. Rees and C. Greif, “A preconditioner for linear systems arising from interior point optimization methods,” SIAM Journal on Scientific Computing, vol. 29, pp. 1992–2007, 2007. [93] L. Bergamaschi, J. Gondzio, and G. Zilli, “Preconditioning indefinite systems in interior point methods for optimization,” Computational Optimization and Applications, vol. 28, pp. 149–171, 2004. [94] N. J. Higham, Accuracy and Stability of Numerical Algorithms. Philadelphia, PA, USA: Society for Industrial and Applied Mathematics, 2nd ed., 2002. [95] L. Wang, Model Predictive Control System Design and Implementation Using MATLAB. Springer Publishing Company, Incorporated, 1st ed., 2009. [96] G. Frison and B. Jørgensen, “Efficient Implementation of the Riccati recursion for solving Linear- Quadratic Control problems,” IEEE Conference on Control Applications, pp. 1117–1122, 2013. [97] P. N. Brown and H. F. Walker, “GMRES on (nearly) singular systems,” SIAM Journal on Matrix Analysis and Applications., vol. 18, pp. 37–51, 1997. [98] T. Samad and A. Annaswamy (eds.), “The Impact of Control Technology,” Overview, Success Stories, and Research Challenges, IEEE Control Systems Society, 2011.

174 [99] J. Pereira (ed), “Monitoring and Control,” Todays Market and its Evolution till 2020, European Commission DG Information Society and Media, 2009. [100] C. Hildreth, “A Quadratic Programming Procedure,” Naval Research Logistics Quarterly, vol. 4, pp. 79–85, 1957. [101] K.-H. John and M. Tiegelkamp, IEC 61131-3: Programming Industrial Automation Systems. Springer-Verlag Berlin Heidelberg, 2010. [102] R. Lewis, Programming Industrial Control Systems using IEC 61131-3. Control Engineering Ser. 50, 1998. [103] K. R. Muske and J. B. Rawlings, “Model Predictive Control with Linear Models,” AIChE Journal, vol. 39, pp. 262–287, 1993. [104] B. Francis and W. Wonham, “The internal model principle of control theory,” Automatica, vol. 12, no. 5, pp. 457–465, 1976. [105] E. Davison and H. Smith, “Pole assignment in linear time-invariant multivariable systems with constant disturbances,” Automatica, vol. 7, no. 4, pp. 489–498, 1971. [106] L. Qiu and E. Davison, “Performance limitations of non-minimum phase systems in the servomech- anism problem,” Automatica, vol. 29, no. 2, pp. 337–349, 1993. [107] D. Hodouin, “Methods for automatic control, observation, and optimization in mineral processing plants,” Process Control, vol. 21, pp. 211–225, 2010. [108] I. K. Craig and R. G. D. Henning, “Evaluation of advanced industrial control projects: a framework for determining economic benefits,” Control Engineering Practice, vol. 8, pp. 769–780, 2000. [109] S. L. J. Jounela, M. Dietrich, K. Halmevaara, and O. Tiili, “Control of pulp levels in flotation cells,” Control Engineering Practice, vol. 11, pp. 73–81, 2003. [110] P. Kampjarvi and S. J. Jounela, “Level control strategies for flotation cells,” Minerals Engineering, vol. 16, pp. 1061–1068, 2003. [111] H. Ferreau, A. Potschka, and C. Kirches, “qpOASES webpage.” http://www.qpOASES.org/, 2007- 2014. [112] B. Kapernick and K. Graichen, “PLC Implementation of a Nonlinear Model Predictive Controller,” Proceeding of the 19th World Congress of The International Federation of Automatic Control, pp. 1892–1897, 2014. [113] M. Schwarz and J. Börcsök,“A survey on OPC and OPC-UA,” Proceedings of the 24th IEEE International Conference on Information, Communication and Automation Technologies, November 2013. [114] M. A. Henson and D. E. Seborg, Nonlinear Process Control. Prentice Hall, First ed., 1997. [115] J. Manzano, D. Limon, D. M. de la Peña,and J. Calliess, “Robust Data-Based Model Predictive Control for Nonlinear Constrained systems,” Proceedings of the 6th IFAC Conference on Nonlinear Model Predictive Control, pp. 193–198, August 2018. [116] K. J. Astrom and R. M. Murray, Feedback Systems: An Introduction for Scientists and Engineers. Princeton, NJ, USA: Princeton University Press, 2008.

175