Linear MPC • Linear Time-Varying and Nonlinear MPC • Quadratic Programming (QP) and Explicit MPC • Hybrid MPC, Stochastic MPC, Data-Driven MPC

Model Predictive Control

Alberto Bemporad

imt.lu/ab

Course page: http://cse.lab.imtlucca.it/~bemporad/mpc_course.html Course structure

• Basic concepts of model predictive control (MPC) and linear MPC • Linear time-varying and nonlinear MPC • Quadratic programming (QP) and explicit MPC • Hybrid MPC, stochastic MPC, data-driven MPC

• Numerical examples:

– MPC Toolbox for MATLAB (linear/explicit/parameter-varying MPC)

– Hybrid Toolbox for MATLAB (explicit MPC, hybrid systems)

For additional background on linear systems and numerical optimization:

http://cse.lab.imtlucca.it/~bemporad/intro_control_course.html

http://cse.lab.imtlucca.it/~bemporad/optimization_course.html

optimization prediction model algorithm

model-based optimizer process

set-points inputs outputs r(t) u(t) y(t)

measurements

simplified likely Use a dynamical model of the process to predict its future evolution and choose------the “best” control action a good

past future r(t) N−1 y X k predicted outputs ∥ y − ∥2 ∥ u − ∥2 min W (yk r(t)) 2 + W (uk ur(t)) 2 uk k=0 manipulated inputs s.t. xk+1 = f(xk, uk) prediction model

yk = g(xk) t t+k t+N

umin ≤ uk ≤ umax constraints ymin ≤ yk ≤ ymax

x0 = x(t) state feedback

numerical optimization problem t+1 t+1+k t+N+1

• At each time t: – get new measurements to update the estimate of the current state x(t)

– solve the optimization problem with respect to {u0, . . . , uN−1} ∗ – apply only the first optimal move u(t) = u0, discard the remaining samples

• MPC is like playing chess !

• On-line (event-based) re-planning used in GPS navigation

• You use MPC too when you drive !

(Rafal, Stevens, AiChE Journal, 1968)

(Propoi, 1963) • MPC used in the process industries since the 80’s (Qin, Badgewell, 2003) (Bauer, Craig, 2008)

Today APC (advanced process control) = MPC

3 MPC in industry (Qin, Badgewell, 2003) S.J. Qin, T.A. Badgwell / Control Engineering Practice 11 (2003) 733–764 745

Table 6 • IndustrialSummary of linear survey MPC applications of MPC by areas applications (estimates based on vendor conducted survey; estimates in do mid not include 1999 applications by companies who have licensed vendor technology)a

Area Aspen Honeywell Adersab Invensys SGSc Total Technology Hi-Spec

Reﬁning 1200 480 280 25 1985 Petrochemicals 450 80 — 20 550 Chemicals 100 20 3 21 144 Pulp and paper 18 50 — — 68 Air & Gas — 10 — — 10 Utility — 10 — 4 14 Mining/Metallurgy 8 6 7 16 37 Food Processing — — 41 10 51 Polymer 17 — — — 17 Furnaces — — 42 3 45 Aerospace/Defense — — 13 — 13 Automotive — — 7 — 7 Unclassiﬁed 40 40 1045 26 450 1601 Total 1833 696 1438 125 450 4542 First App. DMC:1985 PCT:1984 IDCOM:1973 IDCOM-M:1987 RMPCT:1991 HIECON:1986 1984 1985 OPC:1987 Largest App. 603 Â 283 225 Â 85 — 31 Â 12 —

a The numbers reflect a snapshot survey conducted in mid-1999 and should not be read as static. A recent update by one vendor showed 80% increase in the number of applications. b Adersa applications through January 1, 1996 are reported here. Since there are many embedded Adersa applications, it is difficult to accurately Estimatesreport their number based or distribution. on vendor Adersa’s survey product literature indicates over 1000 applications of PFC alone by January 1, 1996. c The number of applications of SMOC includes in-house applications by Shell, which are unclassified. Therefore, only a total number is estimated here.

Aspen Target ﬁrst-principles model that can be derived from 8 by PFC MVC NOVA-NLC integrating across the sample time: Process Perfecter xkþ1 ¼ fðxk; u k; vk; wkÞ; ð9aÞ First Empirical Principles y ¼ g ðxk; u kÞþn ; ð9bÞ DMCplus k k HIECON RMPCT although special care should be taken for stiff systems. PFC Connoisseur SMOC 3.2.2. Linear empirical models Linear Linear empirical models have been used in the majority of MPC applications to date, so it is no Fig. 4. Classiﬁcation of model types used in industrial MPC surprise that most of the current MPC products are algorithms. based on this model type. A wide variety of model forms are used, but they can all be derived from 9 by linearizing about an operating point to get:

x þ1 ¼ Ax þ B u þ B v þ B w ; ð10aÞ 3.2.1. Nonlinear first-principles models k k u k v k w k Nonlinear first-principles models used by the NOVA- y ¼ Cxk þ Du k þ n : ð10bÞ NLC algorithm are derived from mass and energy k k balances, and take exactly the form shown above in 8. The SMOC and PFC algorithms can use this model Unknown model parameters such as heat transfer form. An equivalent discrete-time transfer function coefficients and reaction kinetic constants are either model can be written in the form of a matrix fraction estimated off-line from test data or on-line using an description (Kailath, 1980): extended Kalman filter (EKF). In a typical application À 1 À 1 À 1 À 1 y ¼½I À Uy ðq Þ ½Uuðq Þu k þ Uvðq Þvk the process model has between 10 and 100 differential k À 1 algebraic equations. þ Uwðq Þwkþnk; ð11Þ MPC in industry M. Bauer, I.K. Craig / Journal of Process Control 18 (2008) 2–18 5 (Bauer, Craig, 2008) • EconomicM. assessment Bauer, I.K. Craig / Journal of ofAdvanced Process Control 18 (2008) Process 2–18 Control (APC) 5 Petrochemical North America Petrochemical North America Chemicals Africa Chemicals Africa Petroleum refining Europe Petroleum refining Europe Mineralsprocessing Mineralsprocessing Asia Pacific Asia Pacific Oil & Gas Oil & Gas South America South America Power & utilities Power & utilities 0% 10% 20% 30% Pulp & paper 0% 10% 20% 30% Pulp & paper M. Bauer,M. Bauer, I.K. Craig I.K. Craig / Journal / Journal of Process of Process Control Control 18 18 (2008) (2008) 2–18 2–18 7 7 Industrial gasesIndustrial gases Coal products Model predictive control Coal products Suppliers Suppliers Model predictive control StandardStandard Water & wastewaterWater & wastewater Users Users ConstraintConstraint control control FrequentlyFrequently Other Other Split-rangeSplit-range control control RarelyRarely Linear programming (LP) NeverNever 0% 20%0% 40% 20% 60% 40% 80% 60% 100% 80% 100% Linear programming (LP) Don't know Nonlinear control algorithms or models Don't know Fig. 2. Participants of APC survey by industry and by continent (total: 66 participants).Nonlinear Several answers control werealgorithms allowed or formodels industry sectors. Fig. 2. Participants of APC survey by industry and by continent (total: 66 participants). Several answers were allowedDead-time for industry compensation sectors. participants of APC survey by industry (worldwide) Dead-time compensation sections identify the respondent and capture the size of the the estimation of qualitative,Statistical non-monetary process control benefits sections identify the respondent and capture the size of the the estimation of qualitative,Statistical non-monetary process control benefits plant as well as the use of control technology within the (15%) as well as rules of thumb [47], such as that a new con- plant as well as the use of control technology within the (15%) as well as rules of thumb [47]Neural, suchnetworks as based that control a new con- organization. The third section queries the use of APC trol scheme will increaseNeural the networks throughput based control by a fixed percent- organization. The third section queries the use of APC trol scheme will increase the throughputExpert system by based a fixed control percent- technology and tools. Sections four to six are key to the age (7%). For the waterExpert and system waste-water based control industries, Lant technologyeconomic and tools. assessment Sections four process. to six While are Section key tofour the coversage (7%).and For Steff theens water[48] provide and waste-water a qualitativeFuzzy self-assessmentindustries,logic control Lant form Fuzzy logic control economic assessmentthe managerial process. process While by asking Section about four reporting, covers decisionand Stetoffens establish[48] provide how good a qualitative the processInternal model self-assessment control control is. (IMC) The disadvan- form Internal model control (IMC) the managerialmaking process and by budgeting, asking about Section reporting, five is concerned decision withto the establishtage ofhow such good a table the processis thatAdaptive it control is industry / selftuning is. The related control disadvan- and has to making andengineering budgeting, process Section and five the is estimation concerned of withcost and the benefits.tage ofbe such updated a table with is that the developmentAdaptive it is industry / selftuningDirectsynthesis of related newcontrol technologies. and(DS) has to engineeringSection process sixand requests the estimation the respondents of cost and opinion benefits. aboutbe the updatedThe with questionnaire the development resultsDirectsynthesis ofsupport new technologies. the (DS) industry–univer- Section sixaccuracy, requests importance the respondents and satisfaction opinion with about the current the esti-Thesity questionnaire study by Marlin results and support co-workers the industry–univer-[49], which0% high- 20% 40% 60% 80% 100% accuracy, importancemation process and satisfaction in place as well with as the about current future esti- trends.sity studylighted by theMarlin importance and co-workers of determiningFig.[49] 4., Industrial which the0% base high-use 20% of case APC 40% methods: 60% survey 80% results. 100% operation through in-depth analysis of historical plant mation process in place as well as about future trends. lighted the importance of determiningFig. 4. Industrial the base use of case APC methods: survey results. 4. Framework – the engineering process operationdata. through An importantIndustrial in-depth consideration use analysis of APC methods: forof thehistorical survey base results case plant identi- pointfication closer is to then a limit thexL selectionby Dl, see ofFig. the 5, appropriate[43]. The reduc- process mance that is expected from the new control strategy 4. Framework – the engineering process data.tion An in important variance is consideration often assumed for to bethe a base fixed case percentage. identi- P after. This improvement can be expressed as a profit index A good estimate of the performance that will resultpoint from closervariables to a limit fromx by whichDl, seetheFig. variance 5, [43] is. Theestimated. reduc- Latourmanceav that is expected from the new control strategy "Model Predictive Control" - © A. Bemporad.fication AllSeveral rights is reserved. then industrial theL selectionand academic of the authors appropriate assume a process reduc- computed from the variance reduction8/132 achieved by the new improved process control requires process insight astion wellin as variance[19] recommends is often assumed the analysis to be of a the fixed following percentage. variables:P after. This improvement can be expressed as a profit index A good estimate of the performance that will result from variablestion of from the standard which the deviation variance by is 50% estimated.[19,61–63] Latouror by avsystemb and a minimum achievable variance estimate [68]. an understanding of the control technology. Seven stepsSeveral of industrial and academic authors assume a reduc- computed from the variance reduction achieved by the new improved process control requires process insight as well as [19] 35%recommends•[64]Critical. Moreproduct the optimistic analysis qualities. estimates of the following even propose variables: a reduc- Several other profit and cost factors are not always easy assessing the performance before and after implementationtion of the standard deviation by 50% [19,61–63] or by systemb and a minimum achievable variance estimate [68]. an understanding of the control technology. Seven steps of tion• byImportant 85–90% [20,27,65] manipulated. variables. to express in monetary terms but nevertheless make a con- are described in the left-hand column of Fig. 1. In this35% sec-[64]. More optimistic estimates even propose a reduc- Several other profit and cost factors are not always easy assessing thetion, performance these steps before will be and discussed after implementation in detail. • CriticalThe• Material productassumption and qualities. ofenergy a fixed balance rate of parameters. reduction as applied siderable contribution to the assessment. In the following, tion by 85–90% [20,27,65]. to express in monetary terms but nevertheless make a con- are described inThe the focus left-hand of the column survey as of wellFig. as 1 of. In most this APC sec- projects• Importantby many• Constraint industrial manipulated variables. authors variables. is practical but in many cases several profit and cost factors are discussed. tion, these stepsis on will continuous be discussed processes. in detail. In the survey, 90% of respon-The• Materialsomewhat assumption and arbitrary. ofenergy a fixed To balance achieverate of parameters. a reduction more objective as applied measure, siderable contribution to the assessment. In the following, The focusdents of the work survey on continuousas well as of processes. most APC Cost/benefit projects by assess- many• Constraintan industrial upperIn-depth bound variables. authors process for the is insight economic practical such benefit but as knowledge in was many proposedcases of the by inter-several4.3.1. profit Profit and factors cost factors are discussed. is on continuousments processes. of batch processes In the survey, are conducted 90% of and respon- reportedsomewhat less Muskerelationship arbitrary.[44]. The To between minimum achieve variablesa variance more or objective of the the steady-state process measure, variable economic Both APC users and suppliers were asked in the ques- can be estimated from historical data and process knowl- tionnaire to name the three most important contributors dents workfrequently on continuous since processes.there are fewer Cost/benefit batch processes assess- an in upper theIn-depth boundmodel process for of the the insightprocess economic such is indispensable benefit as knowledge was proposedinformation of the by inter- for the4.3.1. Profit factors edge [66]. to the benefits that result from improved process control. chemical, petrochemical and refining industries. However,Muskerelationship[44]base. The case between minimum identification variables variance[50] or. of theA careful the steady-state process preliminary variable economic data-dri- Both APC users and suppliers were asked in the ques- ments of batch processes are conducted and reported less If an accurate process model along with the designed Several authors [1,23,69,70] discuss the benefits of APC frequently sincemost ofthere the are methods fewer described batch processes in the following in the can model be estimated ofven the analysis process from of historical theis indispensable plant data can provide and information process further knowl- insight for the priortionnaire to name the three most important contributors applied to both batch and continuous processes foredge which[66]controller. to any controlis available, upgrade then[51] data. for the new system can tosystems the benefits and their that contribution, result from improved which formed process the basis control. of chemical, petrochemical and refining industries. However, basebe case simulated identification and used[50]. to A careful estimate preliminary the average data-dri- perfor- the list in the questionnaire. Fig. 6 lists the most frequently the time trend is monitored. If an accurate process model along with the designed Several authors [1,23,69,70] discuss the benefits of APC most of the methods described in the following can be venmance. analysis This of the approach plant canwas provide pursued furtherby Zhou insight and Forbes prior named factors. Despite the small sample size a clear trend controller4.1.1. is available, Performance then functions data for the new system can systems and their contribution, which formed the basis of applied to both batch and continuous processes for which to any[42] controlas well upgrade as by others[51].[56,67] to get a realistic picture can be observed. In particular, throughput and quality, 4.1. Base case identification be simulatedIn and many used cases, to the estimate variance the of a average process variable perfor- can bethe list in the questionnaire. Fig. 6 lists the most frequently the time trend is monitored. of thelinked performance to a monetary of a new value. APC Although system. this This is approach, not the only which are directly related, were two frequently named mance.4.1.1.however, This Performance approach might functions notwas always pursued be by the Zhou best and choice Forbes since it namedprofit factors. factors. Despite A higher the quality small can sample be achieved size a by clear limiting trend The initial step when considering a new APC solution[42] as is wellinstance as by where others economic[56,67] to benefit get a can realistic be captured, picture it is thecan be observed. In particular, throughput and quality, 4.1. Base caseto analyze identification the performance of the current system or the so-Inrequires manymost cases, a commonly detailed the process variance used. model A ofperformance a andprocess significant variable function computa- can[22,52] be the throughput as shown in Fig. 7. APC shifts the curve up, of the performance of a new APC system. This approach, which are directly related, were two frequently named called base case. For the questionnaire, APC experts werelinkedtional todefines a eff monetaryort. either profit value. or loss Although as the process this isvariable not thex onlymoves thus allowing higher throughput at the same quality or The initialasked step how when they considering determine a the new base APC case solution performance is however, andinstanceThe mightaway where improvement from not economic an always optimum resulting benefit be operation the can from best be point. choicethe captured, Thecontrol since derivation it system is it the ofprofit a improved factors. quality A higher for the quality same can throughput be achieved or a by combina- limiting to analyze thebenefits. performance The majority of the of current APC suppliers system or (53%) the so-andrequires a largemostupgrade a commonly detailedperformance can process beused. expressed function model A inperformance requires andquantitative significant process termsfunction insight computa- by compar- and[22,52] expertthetion throughput of the two. as shown in Fig. 7. APC shifts the curve up, tional eingffort. the average performance before the new system, that is, thus allowingSome factors higher contributing throughput to the at benefit the same of APC quality are dif- or called base case.number For of the APC questionnaire, users (42%) estimate APC experts the variance were beforedefinesknowledge. either profit Generalbefore or loss guidelines as the process are di variablefficult to establish,x moves as control using historical data. Other estimation categoriesThethe improvementthe base economic case P resultingav impact, and depends from the estimated on the the control nature average of system the perfor- produc-improvedficult either quality to estimate for theor same to express throughput in monetary or a combina- terms. asked how they determine the base case performance and away from an optimum operation point. The derivation of a Nevertheless, they contribute to the initial assessment and benefits. Theinclude majority past of experience APC suppliers with a (53%) similar and project a large (25%)upgrade andperformance cantion be facility expressed function and on in requires thequantitative industry process[53] terms. insight Performance by compar- and expert functionstion of the two. f(x) ∆µ budget proposal, as shown in Fig. 1. An interesting result number of APC users (42%) estimate the variance before ingknowledge. the average Generalperformance guidelines before are the di newfficult system, to establish, that is, as Some factors contributing to the benefit of APC are dif- the base case P before, and the estimatedDistribution average of perfor- ficultof the either questionnaire to estimate is that or tothe express reduction in of monetary operating man-terms. control using historical data. Other estimation categories the economicBaseav impact case distribution depends on the naturenew system of the produc- power is named as a profit by 14% of the APC suppliers include past experience with a similar project (25%) and tion facility and on the industry [53]. Performance functions Nevertheless, they contribute to the initial assessment and ∆µ while it is not regarded to have an impact on the profit f(x) budgetimprovement proposal, by as the shown manufacturing in Fig. 1. companies. An interesting A reason- result Distribution of of theing questionnaire that manpower is reduction that the reduction results from of operating advanced pro-man- x Base case distribution new system powercess is control named is as that a profit operator by 14% attention of the APCnecessary suppliers for x L whileproblem it is not loops regarded or alarm to conditions have an impactcan be reduced on the profit[71]. Fig. 5. Performance estimation through shifting of a process variable improvementA reduction by of the operating manufacturing manpower companies. as a result A of reason- APC

closer to limit xL. ingwas that reported manpower for an reduction Austrian resultsreﬁnery from[72]. advanced pro- x cess control is that operator attention necessary for x L problem loops or alarm conditions can be reduced [71]. Fig. 5. Performance estimation through shifting of a process variable A reduction of operating manpower as a result of APC

closer to limit xL. was reported for an Austrian reﬁnery [72]. MPC in industry (Samad, IEEE CS Magazine, 2017)

• Impact of advanced control technologies in industry

70%

60%

50%

40%

30%

20%

10%

0% 1961-1970 1971-1980 1981-1990 1991-2000 2001-2010 2011-2020

(a)

18 16 14 12 10 8 6 4 2 0 1961-1970 1971-1980 1981-1990 1991-2000 2001-2010 2011-2020

(b) Fig. 3. Leaders of the IEEE Control Systems Society whose affiliation is with industry: (a)MPC percentage in by decade, industry (b) absolute numbers by decade. See text for explanation. 4.1. Advanced control technologies vary significantly in their impact Table 2 and perceptions thereof The percentage of survey respondents indicating whether a control technology had demonstrated (“Current Impact”) or was likely to demonstrate over the next five (Samad et al., 2020) years (“Future Impact”) high impact in practice. Table 2 shows the results of a survey conducted by the Industry

Committee in March, 2018. The results of an earlier, similar survey, Current Impact Future Impact with similar results, are reported in ( Samad, 2017 ). Respondents— Control Technology %High %High members of the Industry Committee—were asked whether each PID control 91% 78% System Identification 65% 72% technology in the list had demonstrated “high impact in multiple Estimation and filtering 64% 63% sectors,” “high impact in a single sector,” “medium impact,” “low Model-predictive control 62% 85% impact,” or “no impact.” Assessments of both “the present level of Process data analytics 51% 70% impact” and “the potential for future impact … over the next 10 Fault detection and identification 48% 78% years” were prompted for. Of the 77 members of the committee Decentralized and/or coordinated control 29% 54% Robust control 26% 42% then, 66 responded. Intelligent control 24% 59% The intent of the survey was to determine Industry Committee Discrete-event systems 24% 39% members’ opinions regarding the real-world impact of advanced Nonlinear control 21% 42% control technologies, such as model predictive control (MPC), ro- Adaptive control 18% 44% bust control, adaptive control, etc. The survey also included cross- Repetitive control 12% 17% Hybrid dynamical systems 11% 33% cutting ancillary topics such as system identification, data analyt- Other advanced control technology 11% 25% ics, and estimation. PID control was also included—not as an ad- Game theory 5% 17% vanced control technology but for calibration purposes. The survey as distributed included a glossary for the terms used (for exam- As can be observed, MPC is clearly considered more impactful, ple, the glossary noted that nonlinear control included feedback and likely to be more impactful, vis-à-vis other control technolo- linearization, dynamic inversion, sliding-mode control, etc.). gies, especially"As can those be observed, that can be MPC considered is clearly the “crown considered jewels” more of impactful, and likely to be more impactful, vis-à-vis other control technologies, especially those that can be considered the "crown jewels" of control theory - robust control, adaptive control, and nonlinear control."

static optimizer Steady-state •(economic) set-point optimization optimization •static I/O model

set-points

dynamic optimizer • coordinate multiple inputs MPC • performance optimization • constraint handling

measurements actuator set-points

regulators •fast-sampling low-level controllers • single-loop

"Model Predictive Control" - © A. Bemporad. All rights reserved. 11/132 MPC of automotive systems (Bemporad, Bernardini, Borrelli, Cimini, Di Cairano, Esen, Giorgetti, Graf-Plessen, Hrovat, Kolmanovsky Levijoki, Livshiz, Long, Pattipati, Ripaccioli, Trimboli, Tseng, Verdejo, Yanakiev, ..., 2001-present)

Powertrain engine control, magnetic actuators, robotized gearbox, Ford Motor Company power MGT in HEVs, cabin heat control, electrical motors Jaguar DENSO Automotive Vehicle dynamics Fiat traction control, active steering, semiactive suspensions, General Motors autonomous driving

Model predictive control toolset 1 / 8

suspension deflection

tire deflectionsuspension deflection

tire deflection

  4

Most automotive OEMs are looking into MPC4 solutions today

MPC• Coordinate for autonomous torque request and driving steering to achieve safe and comfortable autonomous driving in the presence of other • Coordinate torque request and steering to achieve safe and vehicles and obstacles comfortable autonomous driving with no collisions

• •MPCCombine combines path pathplanning, planning path ,trackingpath tracking and obstacle, and obstacle avoidance avoidance

• •StochasticEmploy stochastic prediction MPC models to takeare into used account to account uncertainty for uncertaintyand anddriver’s driver’s behavior behavior

"Model Predictive Control" - © A. Bemporad. All rights reserved. 13/132 MPC of gasoline turbocharged engines • Control throttle, wastegate, intake & exhaust cams to make engine torque track set-points, with max efficiency and satisfying constraints MPC Engine

Desired Actuators Achieved torque commands Torque

Measurements

numerical optimization problem solved in real-time on ECU

(Bemporad, Bernardini, Long, Verdejo, 2018)

• Coordinate engine torque request and continuously variable transmission (CVT) ratio to improve fuel economy and drivability

• Real-time MPC is able to take into account coupled dynamics and constraints, optimizing performance also during transients Engine torque request

Desired axle torque Engine Control

nowcar.com MPC CVT US06 Double Hill driving cycle ratio request CVT Control (Bemporad, Bernardini, Livshiz, Pattipati, 2018)

The MPC developed by General Motors and ODYS for torque tracking in turbocharged gasoline engines is in high-volume production since 2018

• Multivariable system, 4 inputs, 4 outputs. QP solved in real time on ECU (Bemporad, Bernardini, Long, Verdejo, 2018)

• Supervisory MPC for powertrain control also in production since 2018 (Bemporad, Bernardini, Livshiz, Pattipati, 2018)

First known mass production of MPC in the automotive industry

http://www.odys.it/odys-and-gm-bring-online-mpc-to-production

• MPC capabilities explored in new space applications

cooperating UAVs powered descent planetary rover

   (Bemporad, Rocchi, 2011) (Pascucci, Bennani, Bemporad, 2016) (Krenn et. al., 2012)

http://www.pw.utc.com/Press/Story/20100527-0100/2010

coal 1 ? ? photovoltaic coal 2

? ? ? hydro-storage ? ? ? demand natural gas

? ? ? wind farm

Dispatch power in smart distribution grids, trade energy on energy markets

Challenges: account for dynamics, network topology, physical constraints, and stochasticity (of renewable energy, demand, electricity prices)

FP7-ICT project “E-PRICE - Price-based Control of Electrical Power Systems” (2010-2013)

Drinking water network of Barcelona:

63 tanks 114 controlled flows 17 mixing nodes

• ≈5% savings on energy costs w.r.t. current practice • Demand and minimum pressure requirements met, smooth control actions • Computation time: ≈20 s on NVIDIA Tesla 2075 CUDA (sample time = 1 hr)

FP7-ICT project “EFFINET - Efficient Integrated Real-time Monitoring and Control of Drinking Water Networks”

(2012-2015)

• Goal: find a dynamic hedging policy of a portfolio replicating a synthetic option, so to minimize risk that payoff≠ portfolio wealth at expiration date

• A simple linear stochastic model describes the dynamics of portfolio wealth

• Stochastic MPC results:

Portfolio wealth vs. payoff at expiration Portfolio wealth vs. payoff at expiration 100 0.2

80 0.15

60 0.1

0.05 20 Final wealth in portfolio

0 0

−20 −0.05 0 50 100 150 200 −0.05 0 0.05 0.1 0.15 0.2 Stock price at expiration Payoff ( ) − x(ti) x(ti−1) p(T ) = max{w(T ) − K, 0} p(T ) = max 0,C + min ∈{1 } i ,...,Nfix x(ti−1) European call Napoleon cliquet

• Process control → linear MPC (some nonlinear too) 1970-2000

• Automotive control → explicit, hybrid MPC 2001-2010

• Aerospace systems and UAVs → linear time-varying MPC >2005

• Information and Communication Technologies (ICT) (wireless nets, cloud) → distributed/decentralized MPC >2005

• Energy, finance, automotive, water → stochastic MPC >2010

• Industrial production → embedded optimization solvers for MPC >2010

• Machine learning → data-driven MPC today

High-fidelity simulation model (simplified) control-oriented performance index prediction model & constraints u y G(z)

MPC closed-loop simulation design physical modeling + parameter estimation real-time system code identification /* z=A*x0; */ for (i=0;i

• MPC Toolbox (The Mathworks, Inc.): (Bemporad, Ricker, Morari, 1998-today) – Part of Mathworks’ official toolbox distribution – All written in MATLAB code – Great for education and research

• Hybrid Toolbox: (Bemporad, 2003-today) 9200+ downloads – Free download: http://cse.lab.imtlucca.it/~bemporad/hybrid/toolbox 1.5 downloads/day – Great for research and education

odys.it/embedded-mpc • ODYS Embedded MPC Toolset: (ODYS, 2013-today) – Very flexible MPC design and seamless integration in production – Real-time MPC code and QP solver written in plain C – Support for nonlinear models and deep learning – Designed and adopted for industrial production

Model predictive control toolset 1 / 8

• MPC is a universal control methodology:

– to coordinate multiple inputs/outputs, arbitrary models (linear, nonlinear, ...) – to optimize performance under constraints – intuitive to design, easy to calibrate and reconfigure = short development time

• MPC is a mature technology also in fast-sampling applications (e.g. automotive)

– modern ECUs can solve MPC problems in real-time – advanced MPC software tools are available for design/calibration/deployment

Ready to learn how MPC works ?

See more in “Numerical Optimization” course http://cse.lab.imtlucca.it/~bemporad/optimization_course.html Mathematical programming

min f(x) x f(x) s.t. g(x) ≤ 0

x ∈ Rn, f : Rn → R, g : Rn → Rm g(x) 0  x

    x g (x , x , . . . , x )  1   1 1 2 n   .   .  x =  .  , f(x) = f(x1, x2, . . . , xn), g(x) =  . 

xn gm(x1, x2, . . . , xn)

In general, the problem is difficult to solve use software tools

• Comparison on benchmark problems: http://plato.la.asu.edu/bench.html

• Taxonomy of many solvers for different classes of optimization problems: http://www.neos-guide.org

• NEOS server for remotely solving optimization problems:

http://www.neos-server.org

• Good open-source optimization software:

http://www.coin-or.org/

• GitHub , MATLAB Central , Google , ...

Definition n A set S ⊆ R is convex if for all x1, x2 ∈ S

λx1 + (1 − λ)x2 ∈ S, ∀λ ∈ [0, 1]

convex set nonconvex set S x1 x2 x1 x2

f(λx1 + (1 − λ)x2) ≤ λf(x1) + (1 − λ)f(x2) Jensen’s inequality ∀x1, x2 ∈ S, λ ∈ [0, 1]

The optimization problem min f(x) s.t. x ∈ S

is a convex optimization problem if S is a convex set S x x and f : S → R is a convex function 1 2

• Often S is defined by linear equality constraints Ax = b and convex inequality constraints g(x) ≤ 0, g : Rn → Rm convex

• Every local solution is also a global one (we will see this later)

• Efficient solution algorithms exist

• Often occurring in many problems in engineering, economics, and science

Excellent textbook: “Convex Optimization” (Boyd, Vandenberghe, 2002)

"Model Predictive Control" - © A. Bemporad. All rights reserved. 30/132 Polyhedra Definition Convex polyhedron = intersection of a finite set of half-spaces of Rn Convex polytope = bounded convex polyhedron

• Hyperplane (H-)representation: b 2 x= A 2 v2 A A 2 1 P = {x ∈ Rn : Ax ≤ b} v3

v1 A 1 x= b x=b3 A3 1 • Vertex (V-)representation: A3 Convex hull = transformation Xq Xp n from V- to H-representation P = {x ∈ R : x = αivi + βjrj} i=1 j=1 Vertex enumeration =

∑q transformation from H- to n αi, βj ≥ 0, αi = 1, vi, rj ∈ R V-representation i=1 when q = 0 the polyhedron is a cone vi = vertex, rj = extreme ray "Model Predictive Control" - © A. Bemporad. All rights reserved. 31/132 Linear programming

• Linear programming (LP) problem: -c ′ min c x x* s.t. Ax ≤ b, x ∈ Rn Ex = f George Dantzig (1914–2005)

′ • LP in standard form: min c x s.t. Ax = b x ≥ 0, x ∈ Rn • Conversion to standard form:

1. introduce slack variables Xn Xn aij xj ≤ bi ⇒ aij xj + si = bi, si ≥ 0 j=1 j=1 2. split positive and negative part of x   Xn Xn   + − a x + s = b a (x − x ) + s = b ij j i i ⇒ ij j j i i  j=1  j=1 ≥ + − ≥ xj free, si 0 xj , xj , si 0

• Converting maximization to minimization

max c′x = −(min −c′x)(more generally: max f(x) = − min{−f(x)}) x x x x • Equalities to double inequalities (not recommended) ( P Xn n a x ≤ b a x = b ⇒ Pj=1 ij j i ij j i n a x ≥ b j=1 j=1 ij j i

• Change direction of an inequality Xn Xn aij xj ≥ bi ⇒ −aij xj ≤ −bi j=1 j=1

An LP can be always formulated using “min” and “≤”

• Quadratic programming (QP) problem:

Ax b 1  min x′Qx + c′x 2 x* ≤ ∈ Rn s.t. Ax b, x 1 2x0Qx + c0x = constant Ex = f

• Convex optimization problem if Q ≽ 0 (Q = positive semidefinite matrix) 1 • Without loss of generality, we can assume Q = Q′:

′ ′ ′ 1 x′Qx = 1 x′( Q+Q + Q−Q )x = 1 x′( Q+Q )x + 1 x′Qx − 1 (x′Q′x)′ 2 2 2 ′ 2 2 2 4 4 1 ′ Q+Q = 2 x ( 2 )x • Hard problem if Q ̸≽ 0 (Q = indefinite matrix) 1A matrix P ∈ Rn×n is positive semidefinite (P ≽ 0) if x′P x ≥ 0 for all x. It is positive definite (P ≻ 0) if in addition x′P x > 0 for all x ≠ 0. It is negative (semi)definite (P ≺ 0, P ≼ 0) if −P is positive (semi)definite. It is indefinite otherwise.

1 min c′x min x′Qx + c′x xc 2 ≤ xc s.t. Ax b, x = [ xb ] ≤ s.t. Ax b, x = [ xb ] nc nb x ∈ R , x ∈ {0, 1} nc nb c b xc ∈ R , xb ∈ {0, 1}

mixed-integer linear program (MILP) mixed-integer quadratic program (MIQP)

• Some variables are real, some are binary (0/1)

• MILP and MIQP are NP-hard problems, in general

• Many good solvers are available (CPLEX, Gurobi, GLPK, FICO Xpress, CBC, ...) For comparisons see http://plato.la.asu.edu/bench.html

• AMPL (A Modeling Language for Mathematical Programming) most used modeling language, supports several solvers

• GAMS (General Algebraic Modeling System) is one of the first modeling languages

• GNU MathProg a subset of AMPL associated with the free package GLPK (GNU Linear Programming Kit)

• YALMIP MATLAB-based modeling language

• CVX (CVXPY) Modeling language for convex problems in MATLAB ( )

• CASADI + IPOPT Nonlinear modeling + automatic differentiation, nonlinear programming solver (MATLAB, , C++)

• Optimization Toolbox’ modeling language (part of MATLAB since R2017b)

• PYOMO -based modeling language

• GEKKO -based mixed-integer nonlinear modeling language

• PYTHON-MIP -based modeling language for mixed-integer linear programming

• PuLP A linear programming modeler for

• JuMP A modeling language for linear, quadratic, and nonlinear constrained optimization problems embedded in

• Linear prediction model

( Notation: x ∈ Rn x = Ax + Bu x = x(t) k+1 k k u ∈ Rm 0 y = Cx x = x(t + k|t) k k y ∈ Rp k uk = u(t + k|t)

kX−1 k j • Relation between input and states: xk = A x0 + A Buk−1−j j=0 • Performance index   u − ′≻ 0 NX1 R = R 0 u1 ′ ′ ′ ′≽  .  J(z, x0) = xN P xN + xkQxk+ukRuk Q = Q 0 z = . ′≽ . k=0 P = P 0 uN−1

∗ • Goal: find the sequence z that minimizes J(z, x0), i.e., that steers the state x to the origin optimally

"Model Predictive Control" - © A. Bemporad. All rights reserved. 38/132 Computation of cost function ¯ z }|Q {  ′     x1 Q 0 0 ... 0 x1        x2   0 Q 0 ... 0   x2        ′  .   . . . . .   .  J(z, x0) = x Qx0 +  .   . . . . .   .  0  .   . . . . .   .        xN−1 0 ... 0 Q 0 xN−1 xN 0 0 ... 0 P xN  ′     u0 R 0 ... 0 u0        u1   0 R... 0   u1        +  .   . . .   .  ......  .   . . . .   .  u − 0 ... 0 R u − N 1 | {z } N 1 R¯         x1 B 0 ... 0 u0 A        2   x2   AB B... 0   u1   A           .  =  . . .   .  +  .  x0 ......  .   . . . .   .   .  N−1 N−2 N x A BA B...B u − A N | {z } | N{z 1 } | {z } S¯ z T¯ ¯ ¯ ′ ¯ ¯ ¯ ′ ¯ ′ J(z, x0) = (Sz + T x0) Q(Sz + T x0) + z Rz + x0Qx0 1 ′ ′ ′ ′ 1 ′ ′ = z 2(R¯ + S¯ Q¯S¯) z + x 2T¯ Q¯S¯ z + x 2(Q + T¯ Q¯T¯) x0 2 | {z } 0 | {z } 2 0 | {z } H F ′ Y "Model Predictive Control" - © A. Bemporad. All rights reserved. 39/132 Linear MPC - Unconstrained case   u0 u1 1 ′ ′ ′ 1 ′ condensed  .  J(z, x0) = z Hz+x0F z+ x0Y x0 z = . 2 2 . form of MPC uN−1

• The optimum is obtained by zeroing the gradient

∇zJ(z, x0) = Hz + F x0 = 0

 ∗  u0 ∗  u1  and hence z∗ =   = −H−1F x (“batch” solution) . 0 ∗. uN−1

• Alternative #1: find z∗ via dynamic programming (Riccati iterations)

• Alternative #2: keep also x1, . . . , xN as optimization variables and the equality constraints xk+1 = Axk + Buk (non-condensed form, which is very sparse)

@ each sampling step t: past future r(t) y k predicted outputs

uk manipulated inputs

t t+k t+N • Minimize quadratic function (no constraints)   u0 1 u1 min f(z) = z′Hz + x′(t)F ′z z =  .  z 2 . uN−1

• solution: ∇f(z) = Hz + F x(t) = 0 ⇒ z∗ = −H−1F x(t) h i u(t) = − I 0 ... 0 H−1F x(t) = Kx(t)

unconstrained linear MPC = linear state-feedback!

"Model Predictive Control" - © A. Bemporad. All rights reserved. 41/132 Constrained Linear MPC ( x ∈ Rn x = Ax + Bu • Linear prediction model: k+1 k k u ∈ Rm y = Cx k k y ∈ Rp • Constraints to enforce: ( m umin ≤ u(t) ≤ umax umin, umax ∈ R p ymin ≤ y(t) ≤ ymax ymin, ymax ∈ R

• Constrained optimal control problem (quadratic performance index):

NX−1 ′ ′ ′   min xN P xN + xkQxk + ukRuk u z ′≻ 0 k=0 R = R 0 u1 ′   Q = Q ≽0 z = . ′ . P = P ≽0 u − s.t. umin ≤ uk ≤ umax, k = 0,...,N − 1 N 1 ymin ≤ yk ≤ ymax, k = 1,...,N

kX−1 k i • Linear prediction model: xk = A x0 + A Buk−1−i i=0 • Optimization problem (condensed form):

1 ′ 1 ′ ′ ′ V (x0) = x0Y x0+ min z Hz + x0F z (quadratic objective) 2 z 2

s.t. Gz ≤ W + Sx0 (linear constraints)

convex Quadratic Program (QP)   u0 u1   Nm • z = . ∈ R is the optimization vector z* . uN−1 • H = H′ ≻ 0, and H, F, Y, G, W, S depend on weights Q, R, P upper and lower

bounds umin, umax, ymin, ymax and model matrices A, B, C.

• Input constraints umin ≤ uk ≤ umax, k = 0,...,N − 1     1 0 ... 0 umax      0 1 ... 0   u     max   . .   .     . .. .   .  u (  . . .   .  0       u ≤ u  0 ... 0 1   u   u1  k max   z ≤  max  z =   − ≤ −  −   −   .  uk umin  1 0 ... 0   umin   .      .  0 −1 ... 0   −umin      uN−1 . . .  . .. .   .   . . .   .  0 ... 0 −1 −umin kX−1 k i • Output constraints yk = CA x0 + CA Buk−1−i ≤ ymax, k = 1,...,N i=0       CB 0 ... 0 ymax CA        CAB CB . . . 0   y   CA2     max     . .  z ≤  .  −  .  x0 . . . .  . .   .   .  N−1 N CA B . . . CAB CB ymax CA

past future r(t)

yk @ each sampling step t: predicted outputs uk manipulated inputs

t t+k t+N • Measure (or estimate) the current state x(t)    ∗  feedback u0  z }| { ∗  u  1 ′ ′ ′ ∗  1   min z Hz + x (t)F z • Get the solution z =  .  of the QP z 2 . ∗  uN−1   s.t. Gz ≤ W + S x(t)  |{z} feedback ∗ ∗ ∗ • Apply only u(t) = u0, discarding the remaining optimal inputs u1, . . . , uN−1

• Sample time: Ts = 1 s ( x(t + 1) = [ 1 1 ] x(t) + [ 0 ] u(t) • State-space realization: 0 1 1 y(t) = [ 1 0 ] x(t)

• Constraints: −1 ≤ u(t) ≤ 1 P 1 2 1 2 ′ 1 0 • Performance index: min k=0 yk + 10 uk + x2 [ 0 1 ] x2 (N = 2)

• QP matrices: 4.2 2 2 6 H = [ 2 2.2 ] ,F = [ 0 2 ] cost: 1 z′Hz + x′(t)F ′z + 1 x′(t)Y x(t) 1 0 2 2 4 6 −1 0 Y = [ 6 12 ] ,G = 0 1 0 −1 ≤ 1 0 0 constraints: Gz W + Sx(t) 1 0 0 W = 1 ,S = 0 0 1 0 0

x1 x2

u x2

go to demo linear/doubleint.m (Hybrid Toolbox) (see also mpcdoubleint.m in MPC Toolbox)

• Add constraint on second state at prediction time t + 1:

x2,k ≥ −1, k = 1

• New QP matrices:

4.2 2 2 6 4 6 H = [ 2 2.2 ] ,F = [ 0 2 ] ,Y = [ 6 12 ] " # " # " # −1 0 1 0 1 1 0 1 0 0 G = −1 0 ,W = 1 ,S = 0 0 0 1 1 0 0 0 −1 1 0 0

x1 x2

u x2

• State constraint x2(t) ≥ −1 is satisfied

• Objective: make the output y(t) track a reference signal r(t) • Let us parameterize the problem using the input increments

∆u(t) = u(t) − u(t − 1)

• As u(t) = u(t − 1) + ∆u(t) we need to extend the system with a new state

xu(t) = u(t − 1) ( x(t + 1) = Ax(t) + Bu(t − 1) + B∆u(t)

xu(t + 1) = xu(t) + ∆u(t)

 h i h i  x(t+1) = [ AB ] x(t) + [ B ] ∆u(t) xu(t+1) 0 I h xu(t)i I  x(t) y(t) = [ C 0 ] xu(t)

• Again a linear system with states x(t), xu(t) and input ∆u(t)

"Model Predictive Control" - © A. Bemporad. All rights reserved. 50/132 Linear MPC - Tracking • Optimal control problem (quadratic performance index): NX−1 ∥ y − ∥2 ∥ ∆u ∥2     min W (yk+1 r(t)) 2 + W ∆uk 2 z ∆u0 u0 k=0     , − −  ∆u   u  [∆uk uk uk−1], u−1 = u(t 1)  1   1  z =  .  or z =  .  . .  .   .  s.t. umin ≤ uk ≤ umax, k = 0,...,N − 1 ∆u − u − ymin ≤ yk ≤ ymax, k = 1,...,N N 1 N 1 ∆umin ≤ ∆uk ≤ ∆umax, k = 0,...,N − 1

weight W [x] = diagonal matrix (or Cholesky factor of Q[x] = (W [x])′W [x])

min J(z, x(t)) = 1 z′Hz + [x′(t) r′(t) u′(t − 1)]F ′z convex z 2  x(t) Quadratic   s.t. Gz ≤ W + S  r(t)  Program u(t − 1)

∥ u − ∥2 • Add the extra penalty W (uk uref (t)) 2 to track input references • Constraints may depend on r(t), such as emin ≤ yk − r(t) ≤ emax

1.6

1.4

1 1.2 • System: y(t) = 2 u(t) s +0.4s+1 1

0.8 d2y dy (or equivalently dt2 + 0.4 dt + y = u) 0.6

0.4

0.2

• Sampling with period Ts = 0.5 s: 0 0 10 20 30 ( 1.597 −0.8187 0.5 x(t + 1) = 1 0 x(t) + [ 0 ] u(t) y(t) = [ 0.2294 0.2145 ] x(t)

go to demo linear/example3.m (Hybrid Toolbox)

• Performance index:

X9 input 2 2 − tf(1,[1 .4 1]) min (yk+1 r(t)) + 0.04∆uk k=0 LTI System output

Step Linear Constrained Controller • Closed-loop MPC results: u(t) y(t)

u(t) y(t)

• Impose instead constraint −0.2 ≤ ∆u(t) ≤ 0.2 (slew-rate)

u(t) y(t)

• Measured disturbance v(t) = input that is measured but not manipulated ( manipulated xk+1 = Axk + Buk + Bvv(t) u(t) variables outputs yk = Cxk + Dvv(t) process y(t) measured v(t) k∑−1 disturbances k j j xk = A x0 + A Buk−1−j + A Bvv(t) state j=0 x(t)

• Same performance index, same constraints. We still have a QP: min 1 z′Hz + [x′(t) r′(t) u′(t − 1) v′(t)]F ′z z 2   x(t)    r(t)  s.t. Gz ≤ W + S    u(t − 1)  v(t)

• Note that MPC naturally provides feedforward action on v(t) and r(t)

• Reference not known in advance • Future refs (partially) known in (causal): advance (anticipative action):

rk ≡ r(t), ∀k = 0,...,N − 1 rk = r(t + k), ∀k = 0,...,N − 1 output / reference output / reference use r(t) use r(t+k)

input input

go to demo mpcpreview.m (MPC Toolbox)

• Same idea also applies for preview of measured disturbances v(t + k)

• We can use preview also to coordinate multiple MPC controllers

• Example: cascaded MPC

speed process #1 torque process #2 throttle

y(t) actual u1(t) u2(t)

r(t) MPC #1 MPC #2 u (t), u (t+1), …, u (t+N-1) reference ref,1 ref,1 ref,1 speed future desired u1(t)

• MPC #1 commands set-points to MPC #2

• Example: decentralized MPC

process #1 process #2 y (t) u (t) y1(t) 2 2

u1(t)

r (t) r1(t) 2 MPC #1 MPC #2 u1 (t), u1(t+1), …, u1(t+N-1)

u2 (t), u2(t+1), …, u2(t+N-1)

• Commands generated by MPC #1 are measured dist. in MPC#2, and vice versa

"Model Predictive Control" - © A. Bemporad. All rights reserved. 58/132 Soft constraints • Relax output constraints to prevent QP infeasibility N∑−1 y 2 ∆u 2 2 min ∥W (yk+1 − r(t))∥2 + ∥W ∆uk∥2 + ρϵϵ z k=0   ∆u(0)  .  subj. to xk+1 = Axk + Buk, k = 0,...,N − 1 z = . ∆u(N−1) umin ≤ uk ≤ umax, k = 0,...,N − 1 ϵ

∆umin ≤ ∆uk ≤ ∆umax, k = 0,...,N − 1

ymin − ϵVmin ≤ yk ≤ ymax + ϵVmax, k = 1,...,N

y ∆u • ϵ = “panic” variable, with weight ρϵ ≫ W ,W • Vmin,Vmax = vectors with entries ≥ 0. The larger the i-th entry of vector V , the relatively softer the corresponding i-th constraint

• Infeasibility can be due to: – modeling errors – disturbances – wrong MPC setup (e.g., prediction horizon is too short)

• Linear model with delays x(t + 1) = Ax(t) + Bu(t − τ) y(t) = Cx(t)

u(t 1) ... u(t ⌧) u(t) x(t) y(t) A, B C

• Map delays to poles in z = 0:

x (t) , u(t − k) ⇒ x (t + 1) = x − (t), k = 1, . . . , τ  k   k   k 1   x AB 0 0 ... 0 x 0 xτ 0 0 Im 0 ... 0 xτ 0  xτ−1     xτ−1   0    (t + 1) =  0 0 0 Im ... 0    (t) +   u(t) ...... x1 0 0 0 0 ... 0 x1 Im

• Apply MPC to the extended system

• Note: we must have the prediction horizon N ≥ τ, otherwise no input

u0, . . . , uN−1 has an effect on the output! "Model Predictive Control" - © A. Bemporad. All rights reserved. 60/132 Delays - Method #2 ( x(t + 1) = Ax(t) + Bu(t − τ) • Linear model with delays: y(t) = Cx(t) ( x¯(t + 1) = Ax¯(t) + Bu(t) • Delay-free model: x¯(t) , x(t + τ) y¯(t) = Cx¯(t)

• Design MPC for delay-free model, u(t) = fMPC(¯x(t)) • Compute the predicted state

τX−1 τ j − − x¯(t) =x ˆ(t + τ) = A x(t) + A B u|(t {z1 j}) j=0 past inputs!

• Compute the MPC control move u(t) = fMPC(ˆx(t + τ))

• For better closed-loop performance and improved robustness, xˆ(t + τ) can be computed by a more complex model than (A, B, C)

• Computation complexity depends on chosen prediction model

• Good models for MPC must be

– Descriptive enough to capture the most significant dynamics of the system

– Simple enough for solving the optimization problem

“Things should be made as simple as possible, but not any simpler.”

Albert Einstein (1879–1955)

• After the industrial success of MPC, a lot of research done:

– linear MPC ⇒ linear prediction model – nonlinear MPC ⇒ nonlinear prediction model – robust MPC ⇒ uncertain (linear) prediction model – stochastic MPC ⇒ stochastic prediction model – distributed/decentralized MPC⇒ multiple MPCs cooperating together – economic MPC ⇒ MPC based on arbitrary (economic) performance indices – hybrid MPC ⇒ prediction model integrating logic and dynamics – explicit MPC ⇒ off-line (exact/approximate) computation of MPC – solvers for MPC ⇒ on-line numerical algorithms for solving MPC problems – data-driven MPC ⇒ machine learning methods tailored to MPC design

• Main theoretical issues: feasibility, stability, solution algorithms (Mayne, 2014)

NX−1 y 2 ∆u 2 min ∥W (yk+1 − r(t))∥ + ∥W ∆uk∥ z k=0 subj. to umin ≤ uk ≤ umax, k = 0,...,N − 1 QP problem ymin ≤ yk ≤ ymax, k = 1,...,N ∆umin ≤ ∆uk ≤ ∆umax, k = 0,...,N − 1

• Feasibility: Will the QP problem be feasible at all sampling instants t?

• Input constraints only: always feasible if u/∆u constraints are consistent

• Hard output constraints: – When N < ∞ there is no guarantee that the QP problem will remain feasible at all t, even in the nominal case – N = ∞ ok in the nominal case, but we have an infinite number of constraints! – Maximum output admissible set theory: N < ∞ is enough (Gutman, Ckwikel, 1987) (Gilbert, Tan, 1991) (Chmielewski, Manousiouthakis, 1996) (Kerrigan, Maciejowski, 2000)

• Consider predicted and actual trajectories

x(t) x(t)

x*(t+2|t) x*(t+2|t)

x*(t+1|t)=x(t+1) x*(t+1|t)=x(t+1) 0 t t+N 0 t t+N

* * x(t) x (t+2|t+1)=x(t+2) x(t) x (t+2|t+1)=x(t+2)

x*(t+2|t) x*(t+2|t)

x*(t+1|t)=x(t+1) x*(t+1|t)=x(t+1) 0 t t+N 0 t t+N

• Even assuming perfect model and no disturbances, predicted open-loop trajectories and actual closed-loop trajectories can be different

optimal state x(k) • Special case: when the horizon is infinite, open-loop trajectories and closed-loop trajectories coincide.

time x(t) 0 k N optimal input x*(t+2|t)= x*(t+2|t+1)=x(t+2) u(k)

x*(t+1|t)=x(t+1) 0 t + 1 time • This follows by Bellman’s principle of optimality: 0 k N

“Given the optimal sequence u∗(0), . . . , u∗(N − 1) and the corresponding optimal trajectory x∗(0), . . . , x∗(N), the subsequence u∗(k), . . . , u∗(N − 1) is optimal for the subproblem on the horizon [k, N], starting from the optimal state x∗(k).”

"An optimal policy has the property that, regardless of the decisions taken to enter a particular state, the remaining decisions made for leaving that stage must constitute an optimal policy." Richard Bellman (Bellman, 1957) (1920–1984)

NX−1 ′ ′ ′ min xN P xN + xkQxk + ukRuk z k=0 QP problem

s.t. umin ≤ uk ≤ umax, k = 0,...,N − 1 ymin ≤ Cxk ≤ ymax, k = 1,...,N

Q = Q′ ≽ 0, R = R′ ≻ 0, P = P ′ ≽ 0

• Stability is a complex function of the model (A, B, C) and the MPC parameters

N, Q, R, P, umin, umax, ymin, ymax

• Stability constraints and weights on the terminal state can be imposed over the prediction horizon to ensure stability of MPC

• Theorem: Let the MPC law be based on N∑−1 ∗ ′ ′ V (x(t)) = min xkQxk + ukRuk k=0 s.t. xk+1 = Axk + Buk

umin ≤ uk ≤ umax

ymin ≤ Cxk ≤ ymax

xN = 0 ← “terminal constraint”

with R,Q ≻ 0, umin < 0 < umax, ymin < 0 < ymax. If the optimization problem is feasible at time t = 0 then

lim x(t) = 0, lim u(t) = 0 t→∞ t→∞

and the constraints are satisfied at all time t ≥ 0, for all R,Q ≻ 0.

• Many more convergence and stability results exist (Mayne, 2014)

Proof: Main idea: use the value function V ∗(x(t)) as a Lyapunov-like function

t t ′ • Let zt = optimal control sequence at time t, zt = [u0 . . . uN−1] t t ′ • By construction z¯t+1 = [u1 . . . uN−1 0] is a feasible sequence at time t + 1 ∗ ′ ′ ∗ • The cost of z¯t+1 is V (x(t)) − x (t)Qx(t) − u (t)Ru(t) ≥ V (x(t + 1)) ∗ ∗ • V (x(t)) is monotonically decreasing and ≥ 0, so ∃ limt→∞ V (x(t)) , V∞

• Hence 0 ≤ x′(t)Qx(t) + u′(t)Ru(t) ≤ V ∗(x(t)) − V ∗(x(t + 1)) → 0 for t → ∞

• Since R,Q ≻ 0, limt→∞ x(t) = 0, limt→∞ u(t) = 0

Reaching the global optimum exactly is not needed to prove convergence

• Theorem: Consider the MPC control law based on optimizing N∑−1 ∗ ′ ′ V (x(t)) = min ykQyyk + ukRuk k=0 s.t. xk+1 = Axk + Buk

umin ≤ uk ≤ umax

ymin ≤ yk ≤ ymax

either N = ∞ or xN = 0

If the optimization problem is feasible at time t = 0 then for all R = R′ ≻ 0, ′ ≻ Qy = Qy 0 lim y(t) = 0, lim u(t) = 0 t→∞ t→∞ and the constraints are satisfied at all time t ≥ 0. Moreover, if (C,A) is a

detectable pair then limt→∞ x(t) = 0.

t t ′ t t ′ • The shifted optimal sequence z¯t+1 = [u1 . . . uN−1 0] (or z¯t+1 = [u1 u2 ...] in case N = ∞) is feasible sequence at time t + 1 and has cost ∗ ′ ′ ∗ V (x(t)) − y (t)Qyy(t) − u (t)Ru(t) ≥ V (x(t + 1)) • Therefore, by convergence of V ∗(x(t)), we have that ′ ′ ∗ ∗ 0 ≤ y (t)Qyy(t) + u (t)Ru(t) ≤ V (x(t)) − V (x(t + 1)) → 0 for t → ∞

• Since R,Qy ≻ 0, also limt→∞ y(t) = 0, limt→∞ u(t) = 0 • For all k = 0, . . . , n − 1 we have that kX−1 ′ ∥ k j − − ∥2 0 = lim y (t+k)Qyy(t+k) = lim LC(A x(t)+ A Bu(t+k 1 j)) 2 t→∞ t→∞ j=0 ′ where Qy = L L (Cholesky factorization) and L is nonsingular

• As u(t) → 0, also LCAkx(t) → 0, and since L is nonsingular CAkx(t) → 0 too, for all k = 0, . . . , n − 1

• Hence Θx(t) → 0, where Θ is the observability matrix of (C,A)

• If (C,A) is observable then Θ is nonsingular and hence limt→∞ x(t) = 0

• If (C,A) is only detectable, we can make a canonical observability decomposition and show that the observable states converge to zero

• As also u(t) → 0 and the unobservable subsystem is asymptotically stable, the unobservable states must converge to zero asymptotically too

• We want to track a constant reference r. Assume xr and ur exist such that x = Ax + Bu r r r (equilibrium state/input) r = Cxr

• Formulate the MPC problem (assume umin < ur < umax, ymin < r < ymax)

N∑−1 ′ ′ min (yk − r) Qy(yk − r) + (uk − ur) R(uk − ur) k=0 s.t. xk+1 = Axk + Buk

umin ≤ uk ≤ umax, ymin ≤ Cxk ≤ ymax

xN = xr

• We can repeat the convergence proofs in the shifted-coordinates

xk+1 − xr = A(xk − xr) + B(uk − ur), yk − r = C(xk − xr)

• Drawback: the approach only works well in nominal conditions, as the input

reference ur depends on A, B, C (inverse static model)

1. No stability constraint, infinite prediction horizon N → ∞ (Keerthi, Gilbert, 1988) (Rawlings, Muske, 1993) (Bemporad, Chisci, Mosca, 1994)

2. End-point constraint xN = 0 (Kwon, Pearson, 1977) (Keerthi, Gilbert, 1988) (Bemporad, Chisci, Mosca, 1994)

3. Relaxed terminal constraint xN ∈ Ω (Scokaert, Rawlings, 1996)

4. Contraction constraint ∥xk+1∥ ≤ α∥x(t)∥, α < 1 (Polak, Yang, 1993) (Bemporad, 1998)

∗ All the proofs in (1,2,3) use the value function V (x(t)) = minz J(z, x(t)) as a Lyapunov function.

NX−1 y 2 ∆u 2 2 min ∥W (yk − r(t))∥ + ∥W ∆uk∥ + ρϵϵ z 2 2 k=0

subj. to umin ≤ uk ≤ umax, k = 0,...,N − 1 ∆umin ≤ ∆uk ≤ ∆umax, k = 0,...,N − 1 ∆uk = 0, k = Nu,...,N − 1 ymin − ϵVmin ≤ yk ≤ ymax + ϵVmax, k = 1,..., Nc

ymax • The input horizon Nu limits the number

of free variables yk – Reduced performance – Reduced computation time uk

typically Nu = 1 ÷ 5 t t+Nu t+Nc t+N

• The constraint horizon Nc limits the number of constraints – Higher chance of violating output constraints but reduced computation time

• Other variable-reduction methods exist (Bemporad, Cimini, 2020)

P ′ N−1 ′ ′ • Special case: J(z, x0) = xN P xN + k=0 xkQxk + ukRuk, Nu = N, with matrix P solving the Algebraic Riccati Equation

P = A′PA − A′PB(B′PB + R)−1B′PA + Q

Jacopo Francesco Riccati (1676–1754)

(unconstrained) MPC = LQR for any choice of the prediction horizon N

Proof: : Easily follows from Bellman’s principle of optimality (dynamic ′ ∞ programming): xN P xN = optimal “cost-to-go” from time N to .

"Model Predictive Control" - © A. Bemporad. All rights reserved. 75/132 MPC and Constrained LQR • Consider again the constrained MPC law based on minimizing NX−1 ′ ′ ′ min xN P xN + xkQxk + ukRuk z k=0 s.t. umin ≤ uk ≤ umax, k = 0,...,N − 1 ymin ≤ yk ≤ ymax, k = 1,...,N uk = Kxk, k = Nu,...,N − 1 • Choose matrix P and terminal gain K by solving the LQR problem

K = −(R + B′PB)−1B′PA P = (A + BK)′P (A + BK) + K′RK + Q

• In a polyhedral region around the origin, constrained MPC = constrained LQR

for any choice of the prediction and control horizons N, Nu (Sznaier, Damborg, 1987) (Chmielewski, Manousiouthakis, 1996) (Scokaert, Rawlings, 1998) (Bemporad, Morari, Dua Pistikopoulos, 2002) • The larger the horizon N, the larger the region where MPC ≡ constrained LQR "Model Predictive Control" - © A. Bemporad. All rights reserved. 76/132 MPC and Constrained LQR

• Some MPC formulations also include the terminal constraint xN ∈ X∞ X umin ≤ K k ≤ umax ∀ ≥ ∞ = x :[ ymin ] [ C ](A + BK) x [ ymax ] , k 0

that is the maximum output admissible set for the closed-loop system

(A + BK,B,C) and constraints umin ≤ Kx ≤ umax, ymin ≤ Cx ≤ ymax

• This makes MPC ≡ constrained LQR where the MPC problem is feasible

• Recursive feasibility in ensured by the terminal constraint for all x(0) such that the MPC problem is feasible @t = 0

• The domain of feasibility may be reduced because of the additional constraint

"Model Predictive Control" - © A. Bemporad. All rights reserved. 77/132 Double integrator example( 1 1 0 x(t + 1) = [ 0 1 ] x(t) + [ 1 ] u(t) • Model and constraints: y(t) = [ 1 0 ] x(t) −1 ≤ u(t) ≤ 1 • Objective:

X∞ ∀ ≥ 1 uk = Kxk, k Nu,K = LQR gain min y2+ u2 k 10 k k=0 Nu = N = 2 P 1 y2 + 1 u2 + x′ [ 2.1429 1.2246 ] x k=0 k 10 k 2 |1.2246{z 1.3996} 2 Riccati matrix • QP matrices (cost function normalized by max singular value of H) 10 0.8365 0.3603 0.4624 1.2852 H = 0.3603 0.2059 ,F = 0.1682 0.5285 " # 5 1 0 1 0 0 −1 0 1 0 0 G = 0 1 ,W = 1 ,S = 0 0 0 0 −1 1 0 0 velocity position,

Linearized model:  " # " #  −0.0151 −60.5651 0 −32.174 −2.516 −13.136  −0.0001 −1.3411 0.9929 0 −0.1689 −0.2514 x˙ = 0.00018 43.2541 −0.86939 0 x + −17.251 −1.5766 u 0 0 1 0 0 0   0 1 0 0 y = 0 0 0 1 x (Kapasouris, Athans, Stein, 1988)

:); 80*79 • Inputs: elevator and flaperon (=flap+aileron) angles /0'$%+1 !"##$% • Outputs: attack and pitch angles -')., &'$()*+%, • Sampling time: Ts = 0.05 sec (+ ZOH) !+'' ◦ /0'$%+1 • Constraints: ±25 on both angles 2+130*"#01)'4)50,

• Open-loop unstable, poles are 6$%*07)'4)50, −7.6636, −0.0075 ± 0.0556j, 5.4530 2)*$%)'4)50, go to demo linear/afti16.m (Hybrid Toolbox) see also afti16.m (MPC Toolbox)

Wednesday, June 2, 2010 Example: AFTI-F16 aircraft

• Prediction horizon N = 10, control horizon Nu = 2

∆u 0.1 0 u • Input weights W = [ 0 0.1 ], W = 0 − 25◦ • Input constraints umin = umax = 25◦

30 20 20 15 10 y1(t) 10 u(t) 0 W y = [ 1 0 ] -10 0 1 y2(t) 5 -20 0 -30 0 0.5 1 1.5 2 2.5 3 3.5 4 0 0.5 1 1.5 2 2.5 3 3.5 4 30 20 20 15 10 y1(t) 10 0 y 100 0 u(t) W = [ 0 1 ] y2(t) 5 -10 -20 0

0 0.5 1 1.5 2 2.5 3 3.5 4 0 0.5 1 1.5 2 2.5 3 3.5 4

◦ • Add output constraints y1,min = −y1,max = 0.5

30 20 20 15 y1(t) 10 10 y 1 0 u(t) 0 W = [ 0 1 ] y2(t) 5 -10 0 -20 0 0.5 1 1.5 2 2.5 3 3.5 4 0 0.5 1 1.5 2 2.5 3 3.5 4

• Soft output constraint = convex penalty with dead zone

min ρϵ2

s.t. y1,min − ϵV1,min ≤ y1 ≤ y1,max − ϵV1,max

• Linear control (=unconstrained MPC) + input clipping

104 30

20 3 10 y1(t) 2 u(t) 0 W y = [ 1 0 ] -10 0 1 y2(t) 1 -20 0 -30 0 0.5 1 1.5 2 2.5 3 3.5 4 0 0.5 1 1.5 2 2.5 3 3.5 4

unstable!

Saturation needs to be considered in the control design!

• Saturation is dangerous because it can break the control loop

r(t) saturation linear ua(t) x(t) controller process

• MPC takes saturation into account and handles it automatically (and optimally)

r(t) saturation MPC ua(t) x(t) controller process

subj. to ∆umin ≤ ∆uk ≤ ∆umax, k = 0,...,Nu − 1 ∆uk = 0, k = Nu,...,N − 1 umin ≤ uk ≤ umax, k = 0,...,Nu − 1 ymin − ϵVmin ≤ yk ≤ ymax + ϵVmax, k = 1,..., Nc

• weights: the larger the ratio W y/W ∆u the more aggressive the controller

• input horizon: the larger Nu, the more “optimal” but more complex the controller • prediction horizon: the smaller N, the more aggressive the controller

• constraints horizon: the smaller Nc, the simpler the controller

• limits: controller less aggressive if ∆umin, ∆umax are small

• penalty ρϵ: pick up smallest ρϵ that keeps soft constraints reasonably satisfied

Always try to set Nu as small as possible!

• Let Ts = sampling time used in MPC predictions

• For predicting Tp time units in the future we must set 4 Tp N = 3 Ts 2

0 0 1 2 3 4 5 6 time (s)

≤ du ≤ • Slew-rate constraints u˙ min dt u˙ max on actuators are related to Ts by

z∆u}|min{ z∆u}|max{ du uk − uk−1 u˙ , ≈ Tsu˙ min ≤ ∆uk ≤ Tsu˙ max dt Ts

• The MPC cost to minimize can be thought in continuous-time as Z Tp ∥ y − ∥2 ∥ u − ∥2 ∥ u˙ ∥2 2 J = W (y(τ) r) 2 + W (u(τ) ur) 2 + W u˙ (τ) 2 dτ + ρϵϵ 0  − NX1  ≈  ∥ y − ∥2 ∥ u − ∥2 ∥ u˙ −1 ∥2 −1 2 Ts W (yk+1 r) 2 + W (uk ur) 2 + |W {zTs } ∆uk 2 + ρϵTs ϵ k=0 W ∆u

• Hence, when changing the sampling time from T1 to T2, can can keep the same MPC cost function by leaving W y, W u unchanged and simply rescaling

u˙ u˙ ∆u W T1 W T1 ∆u ρϵ T1 ρϵ T1 W2 = = = W1 ρϵ2 = = = ρϵ1 T2 T2 T1 T2 T2 T2 T1 T2

• Note: Ts used for controller execution can be ≠ than Ts used in prediction

• Small controller Ts ⇒ fast reaction to set-point changes / disturbances

• Humans think infinite precision …computers do not!

• Numerical difficulties may arise if variables assume very small/large values

• Example:

y1 ∈ [−1e − 4, 1e − 4] [V] y1 ∈ [−0.1, 0.1] [mV] y2 ∈ [−1e4, 1e4] [Pa] y2 ∈ [−10, 10] [kPa]

• Ideally all variables should be in [−1, 1]. For example, one can replace y with

y/ymax

• Scaling also possible after formulating the QP problem (=preconditioning)

• Numerical issues may arise with open-loop unstable models kX−1 k j • When condensing the QP we substitute xk = A x0 + A Buk−1−j that j=0 may lead to a badly conditioned Hessian matrix H    ′ Q 0 0 ... 0   B 0 ... 0 B 0 ... 0 " # 0 Q 0 ... 0 R 0 ... 0 AB B ... 0   AB B ... 0 0 R ... 0  . . .   . . . . .   . . .  . . . H = ...... + ...... − − 0 ... 0 Q 0 − − AN 1BAN 2B ... B AN 1BAN 2B ... B 0 ... 0 R 0 0 ... 0 P

• Pre-stabilizing the system with uk = Kxk + vk leads to xk+1 = AK xk + Bvk, kX−1 , k j AK A + BK, and therefore xk = AK x0 + AK Bvk−1−j j=0 • Input constraints become mixed constraints umin ≤ Kxk + vk ≤ umax

• K can be chosen for example by pole-placement or LQR

r(t) u(t) y(t) Optimizer Process measured reference x(t) outputs

x(t) Observer state estimate

• Full state x(t) of process may not be available, only outputs y(t) • Even if x(t) is available, noise should be filtered out • Prediction and process models may be quite different • The state x(t) may not have any physical meaning (e.g., in case of model reduction or subspace identification)

We need to use a state observer

• Example: Luenberger observer xˆ(t + 1) = Axˆ(t) + Bu(t) + L(y(t) − Cxˆ(t))

u(t) manipulated variables yu(t) controlled variables unmeasured measured disturbances v(t) plant outputs

nd(t) unmeasured d(t) model disturbance unmeasured white noise model disturbances

zm(t) n (t) + m measurement m(t) + ym(t) white noise noise model measured outputs

unmeasured disturbance model measurement noise model ( ( ¯ ¯ ˜ ˜ xd(t + 1) = Axd(t) + Bnd(t) xm(t + 1) = Axm(t) + Bnm(t) ¯ ¯ ˜ ˜ d(t) = Cxd(t) + Dnd(t) m(t) = Cxm(t) + Dnm(t)

• Note: the measurement noise model is not needed during optimization

• Plant model { x(t + 1) = Ax(t) + Buu(t) + Bvv(t) + Bdd(t)

y(t) = Cx(t) + Dvv(t) + Ddd(t)

• Full model for designing Kalman filter Rudolf Emil Kalman (1930–2016) [ ] [ ][ ] x(t+1) ¯ x(t) [ ] [ ] ABdC 0 Bu Bv xd(t+1) = 0 A¯ 0 xd(t) + 0 u(t) + 0 v(t)+ xm(t+1) [ 0] 0 A˜ xm[ (t)] 0 [ ] 0 ¯ BdD 0 Bu B¯ nd(t) + 0 nm(t) + 0 nu(t) 0 [ B˜ ] 0 x(t) ¯ ˜ ¯ ˜ ym(t) = [ Cm DdmC C ] xd(t) + Dvmv(k) + Dmnd(t) + Dmnm(t) xm(t)

• nd(k) = source of modeling errors

• nm(k) = source of measurement noise

• nu(k) = white noise on input u (added to compute the Kalman gain)

yˆm(t|t − 1) = Cmxˆ(t|t − 1)

xˆ(t|t) =x ˆ(t|t − 1) + M(ym(t) − yˆm(t|t − 1))

• Time update

xˆ(t + 1|t) = Axˆ(t|t − 1) + Bu(t) + L(ym(t) − yˆm(t|t − 1))

• Note that if L = AM then xˆ(t + 1|t) = Axˆ(t|t) + Bu(t)

• The separation principle holds (under certain assumptions) (Muske, Meadows, Rawlings, 1994) "Model Predictive Control" - © A. Bemporad. All rights reserved. 92/132 I/O feedthrough • We always assumed no feedthrough from u to measured y

yk = Cxk+Duk + Dvvk + Dddk,Dm = 0 • This avoids static loops between state observer, as xˆ(t|t) depends on u(t) via

yˆm(t|t − 1), and MPC (u(t) depends on xˆ(t|t))

• Often D = 0 is not a limiting assumption as – often actuator dynamics must be considered (u is the set-point to a low-level controller of the actuators) – most physical models described by ordinary differential equations are strictly causal, and so is the discrete-time version of the model

• In case D ≠ 0, we can assume a delay in executing the commanded u

yk yk = Cmxk + Duk−1

yk-1 and treat u(t − 1) as an extra state uk uk-1 • Not an issue for unmeasured outputs t (k-1)Ts kTs (k+1)Ts "Model Predictive Control" - © A. Bemporad. All rights reserved. 93/132 Integral action in MPC Output integrators u(t) manipulated variables yu(t) unmeasured v(t) measured disturbances plant controlled variables outputs

nd(t) unmeasured d(t) model + disturbance zm(t) unmeasured + white noise model disturbances

ni(t) output white noise integrators

nm( t) m(t) + measurement + y (t) noise model m white noise measured outputs

• Introduce output integrators as additional disturbance models • Under certain conditions, observer + controller provide zero offset in steady-state

• Add constant unknown disturbances on measured outputs:    xk+1 = Axk + Buk d = d  k+1 k yk = Cxk + dk

• Use the extended model to design a state observer (e.g., Kalman filter) that estimates both the state xˆ(t) and disturbance dˆ(t) from y(t)

• Why we get offset-free tracking in steady-state (intuitively): – the observer makes Cxˆ(t) + dˆ(t) → y(t) (estimation error) – the MPC controller makes Cxˆ(t) + dˆ(t) → r(t) (predicted tracking error) – the combination of the two makes y(t) → r(t) (actual tracking error)

• In steady state, the term dˆ(t) compensates for model mismatch

• See more on survey paper (Pannocchia, Gabiccini, Artoni, 2015)

• More general result: consider the disturbance model    xk+1 = Axk + Buk + Bddk special case: output integrators  dk+1 = dk  Bd = 0, Dd = I yk = Cxk + Dddk

Theorem Let the number nd of disturbances be equal to the number ny of measured outputs. Then all pairs (Bd,Dd) satisfying " # I − A −Bd rank = nx + ny CDd

guarantee zero offset in steady-state (y = r), provided that constraints are not active in steady-state and the closed-loop system is asymptotically stable. "Model Predictive Control" - © A. Bemporad. All rights reserved. 96/132 Disturbance model example

1 −1 −2 1 1 0 −2 3 2 0 • Open-loop system: A = 0 2 −1 −1 , B = 1 , Cm = [ 1 1 0 0 ] 0 2 0 −2 1

A 0 • We cannot add an output integrator since the pair ([ 0 1 ] , [ C 1 ]) is not observable

• Add an input integrator: B = B, D = 0, rank ABd = 5 = n + n d d Cm Dd x y   x(t + 1) = Ax(t) + B(u(t) + d(t)) d(t + 1) = d(t)  y(t) = Cx(t) + 0 · d(t)

(Kwakernaak, Sivan, 1972)

• Idea: add the integral of the tracking error as an additional state (original idea developed for integral action in state-feedback control)

• Extended prediction model:   · ←  x(t + 1) = Ax(t) + Buu(t) + 0 r(t) r(t) is seen as a meas. disturbance  − ← q(t + 1) = q(t) + |Cx(t){z r(t}) integral action   tracking error y(t) = Cx(t)

∥ i ∥2 i • W q 2 is penalized in the cost function, otherwise it is useless. W is a new tuning knob

• Intuitively, if the MPC closed-loop is asymptotically stable then q(t) converges to a constant, and hence y(t) − r(t) converges to zero.

"Model Predictive Control" - © A. Bemporad. All rights reserved. 98/132 Reference / Command Governor (Bemporad, Mosca, 1994) (Gilbert, Kolmanovsky, Tan, 1994) (Garone, Di Cairano, Kolmanovsky, 2016) actual control reference input constrained r(t) w(t) local u(t) variables c(t) desired feedback y(t) r(t) reference reference process ¼ governor controlled outputs

primal system x(t)

estimated plant + controller states

• MPC manipulates set-points to a linear feedback loop to enforce constraints • Separation of problems:

– Local feedback guarantees offset-free tracking y(t) − w(t) → 0 in steady-state in the absence of constraints – Actual reference w(t) generated by MPC to take care of constraints • Advantages: small-signal properties preserved, fewer variables to optimize

∥ − ∥2 w(t) = arg minw w r(t) 2 s.t. ct+k ∈ C

• In control systems, integral action occurs if the controller has a transfer-function from the output to the input of the form

B(z) u(t) = y(t),B(1) ≠ 0 (z − 1)A(z)

• One may think that the ∆u-formulation of MPC provides integral action ...

... is it true ?

• Example: we want to regulate the output y(t) to zero of the scalar system

x(t + 1) = αx(t) + βu(t) y(t) = x(t)

• Design an unconstrained MPC controller with horizon N = 1

2 2 ∆u(t) = arg min∆u0 ∆u0 + ρy1 s.t. u0 = u(t − 1) + ∆u0 y1 = x1 = αx(t) + β(∆u0 + u(t − 1))

• By substitution, we get

2 − 2 ∆u(t) = arg min∆u0 ∆u0 + ρ(αx(t) + βu(t 1) + β∆u0) 2 2 − = arg min∆u0 (1 + ρβ )∆u0 + 2βρ(αx(t) + βu(t 1))∆u0 − βρα − ρβ2 − = 1+ρβ2 x(t) 1+ρβ2 u(t 1) • Since x(t) = y(t) and u(t) = u(t − 1) + ∆u(t) we get the linear controller

ρβα 1+ρβ2 z u(t) = − y(t) z = 1 − 1 No pole in z 1+ρβ2 • Reason: MPC gives a feedback gain on both x(t) and u(t − 1), not just on x(t)

"Model Predictive Control" - © A. Bemporad. All rights reserved. 101/132 Integral action and ∆u-formulation • Numerical test (with MPC Toolbox) alpha=.5; beta=3; sys=ss(alpha,beta,1,0);sys.ts=1; rho=1;p=1;m=1; weights=struct('OV',rho,'MVRate',1); mpc1=mpc(sys,1,p,m,weights); setoutdist(mpc1,'remove',1); % no output disturbance model mpc1tf=tf(mpc1); sum(mpc1tf.den{1})

ans = 0.9000

• Now add an output integrator as disturbance model setoutdist(mpc1,'integrators'); % add output disturbance model mpc1tf=tf(mpc1); sum(mpc1tf.den{1})

ans = -6.4185e-16

• Assume we have an internal model of the reference signal r(t)

r r x (t + 1) = Arx (t) r r(t) = Crx (t)

• Consider the augmented prediction model " # " #" # " # xk+1 A 0 xk B r = r + uk xk+1 0 Ar xk 0

r • Designh i a state observer h i (e.g., via pole-placement) to estimate x(t), x (t) from y(t) C 0 x(t) = r r(t) 0 Cr x (t) xk • Design an MPC controller with output e = y − r = [ C −Cr ] r k k k xk

N X output 0.5 • Penalize ek (+small penalty on u or ∆u)

k=1 0 • Set control horizon Nu = N -0.5 0 5 10 15 20 25 30 35 40 45 50

input 2 • Example: r(t) = r0 sin(ωt + ϕ0), ω = 1 rad/s 1 0

-1

h i -2 1.6416 −0.7668 0.2416 -3 A = 1 0 0 0 5 10 15 20 25 30 35 40 45 50 h 0i 0.125 0 0.5 B = 0 0 observer poles placed in C = [ 0.1349 0.0159 −0.1933 ] {0.3, 0.4, 0.5, 0.6, 0.7} 1.7552 −1 sample time Ts = 0.5 s Ar = 1 0 Cr = [ .2448 .2448 ] input saturation −2 ≤ u(t) ≤ 2

• Several MPC design features available: – explicit MPC

– time-varying/adaptive models, nonlinear models, weights, constraints

– stability/frequency analysis of closed-loop (inactive constraints)

– … • Prediction models can be generated by the Identification Toolbox or automatically linearized from Simulink

• Fast command-line MPC functions (compiled EML-code)

• Graphical User Interface

• Simulink library (compiled EML-code)

>> mpclib

>> mpcDesigner (old version: >> mpctool)

See video on Mathworks’ web site (link)

Symbol Value (MKS) Meaning >> mpcmotor LS 1.0 shaft length dS 0.02 shaft diameter JS negligible shaft inertia JM 0.5 motor inertia βM 0.1 motor viscous friction coefficient R 20 resistance of armature see also kT 10 motor constant ρ 20 gear ratio linear/dcmotor.m kθ 1280.2 torsional rigidity (Hybrid Toolbox) JL 50JM nominal load inertia βL 25 load viscous friction coefficient Ts 0.1 sampling time

  0 1 0 0     0  − kθ − βL kθ 0    [ ]  JL JL ρJL   0  θL ˙ x˙ =  0 0 0 1  x +   V θL    0  x =   θM ˙ 2 kT θM k k βM +k /R θ − θ − T RJM [ ρJ 0 ] ρ2J J M M M [ ] θL = 1 0 0 0 x y = θL [ ] T − kθ T = kθ 0 ρ 0 x

>> [plant, tau] = mpcmotormodel; >> plant = setmpcsignals(plant,'MV',1,'MO',1,'UO',2);

• The input DC voltage V is bounded withing the range

|V | ≤ 220 V

2 • Finite shear strength τadm = 50N/mm requires that the torsional torque T satisfies the constraint |T | ≤ 78.5398 Nm

• Sampling time of model/controller: Ts = 0.1s

>> MV = struct('Min',-220,'Max',220); >> OV = struct('Min',{-Inf,-78.5398},'Max',{Inf,78.5398}); >> Ts = 0.1;

∑p−1 y 2 ∆u 2 2 min ∥W (yk+1 − r(t))∥ + ∥W ∆uk∥ + ρϵϵ ∆U k=0

subj. to ∆umin ≤ ∆uk ≤ ∆umax, k = 0, . . . , m − 1

∆uk = 0, k = m, . . . , p − 1

umin ≤ uk ≤ umax, k = 0, . . . , m − 1

ymin − ϵVmin ≤ yk ≤ ymax + ϵVmax, k = 1, . . . , p

>> Weights = struct('MV',0,'MVRate',0.1,'OV',[0.1 0]); >> p = 10; >> m = 2; >> mpcobj = mpc(plant,Ts,p,m,Weights,MV,OV);

R µ M Tr V e µ s T µ JM L b r J M L b L

Closed-loop simulation using the sim command

>> Tstop = 8; % seconds >> Tf = round(Tstop/Ts); % simulation iterations >> r = [pi*ones(Tf,1) zeros(Tf,1)]; >> [y1,t1,u1] = sim(mpcobj,Tf,r);

closed-loop simulation in Simulink

>> mdl = 'mpc_motor'; >> open_system(mdl) >> sim(mdl)

θL(t) T (t) V (t)

4 100 400

3 300

2 50 200

1 100

0 0 0

-1 -100

-2 -50 -200

-3 -300

-4 -100 -400 0 5 10 15 20 0 5 10 15 20 0 5 10 15 20

• same MPC tuning parameters

• setpoint r(t) = π sin(0.4t) deg

θ Lf Lr • Take simulation model sldemo_suspn in MATLAB z u u f r h

˙ Ffront = 2Kf (Lf θ − (z + h)) + 2Cf (Lf θ − z˙) + uf

Ffront,Frear = upward force on body from front/rear suspension

Kf ,Kr = front and rear suspension spring constant

Cf ,Cr = front and rear suspension damping rate

Lf ,Lr = horizontal distance from c.g. to front/rear suspension θ, θ˙ = pitch (rotational) angle and its rate of change z, z˙ = bounce (vertical) distance and its rate of change

• For control purposes we add external forces uf , ur as manipulated variables

• The system has 4 states: θ, θ˙, z, z˙

• Measured disturbances: road height h, pitch moment from vehicle acceleration

• Step #1: get a linear discrete-time model (4 inputs, 3 outputs, 4 states)

>> plant_mdl = 'suspension_model'; >> op = operspec(plant_mdl); >> [op_point, op_report] = findop(plant_mdl,op); >> sys = linearize(plant_mdl, op_point); >> Ts = 0.025; % sample time (s) >> plant = c2d(sys,Ts);

>> plant = setmpcsignals(plant,'MV',[1 2],'MD',... [3 4],'MO',[1 2],'UO',3);

• Step #2: design the MPC controller

>> dfmax = .1/.1; % [kN/s] >> MV = struct('RateMin',-Ts*dfmax, Ts*dfmax,... 'RateMax',Ts*dfmax,Ts*dfmax); >> OV = []; >> Weights = struct('MV',[0 0], 'MVRate',[.01 .01],... 'OV',[.01 0 10]);

>> p = 50; % Prediction horizon >> m = 5; % Control horizon

>> mpcobj = mpc(plant,Ts,p,m,Weights,MV,OV);

Vehicle Suspension Model ?

1/Iyy 1 1 Theta My s Thetadot s Theta 1/(Body Inertia) Pitch moment THETAdot THETA induced by vehicle acceleration Thetadot

signal1 signal1 -Front Pitch Moment Rear Pitch Moment

FrontForce RearForce signal2 signal2

Front Suspension Rear Suspension h Road Height

1/Mb 1 1 Z+h s Zdot s Z Z+h 1/(Body Mass) Zdot Z

-9.81 Zdot acceleration Zdot due to gravity

cm 1000 signal1 100 mo Theta Gain2 Gain4 uf 1/1000 ur mv MPC ref y0 1000 reference Gain5 Gain3 signal1 md signal2 100

-3 _ #10 3 (rad/s) z_ (cm/s) • Closed-loop MPC results 6 2 Open-loop Closed-loop 4 0

2 -2

0 -4 Open-loop Closed-loop -2 -6 0 2 4 6 8 10 0 2 4 6 8 10

MPC Front Force uf (N) MPC Rear Force ur (N) 150 20

0 100

-20 50 -40

0 -60

-50 -80 0 2 4 6 8 10 0 2 4 6 8 10

My (Nm) h, Z+h (cm) 100 5 road height h 80 relative displacement Z+h 0

60 -5 40

-10 20

0 -15 0 2 4 6 8 10 0 2 4 6 8 10

• Unconstrained MPC gain + linear observer = linear dynamical system

• Closed-loop MPC analysis can be performed using standard frequency-domain tools (e.g., Bode plots for sensitivity analysis)

process ym(t) x(t)

u(t)

MPC x^(t) r(t)

>> sys=ss(mpcobj) returns the LTI object of the MPC controller >> sys=tf(mpcobj) (when constraints are inactive)

• Given a desired linear controller u = Kdx, find a set of weights Q, R, P defining an MPC problem such that h i −1 − I 0 ... 0 H F = Kd

i.e., the MPC law coincides with Kd when the constraints are inactive

• Recall that the QP matrices are H = 2(R¯ + S¯′Q¯S¯), F = 2S¯′Q¯T¯, where   Q 0 0 ... 0       B 0 ... 0 A 0 Q 0 ... 0 R 0 ... 0   0 R ... 0 AB B ... 0 A2 ¯ . . . . . ¯   ¯   ¯   Q =  . . . . .  , R = . . . . , S = . . .. . , T = ...... 0 ... 0 Q 0 − − 0 ... 0 R AN 1BAN 2B ... B N 0 0 ... 0 P A • The above inverse optimality problem can be cast to a convex problem (Di Cairano, Bemporad, 2010)

• More general result: see (Zanon, Bemporad, 2020)

(Di Cairano, Bemporad, 2010)

• Open-loop process: y(t) = 1.8y(t − 1) + 1.2y(t − 2) + u(t − 1)

• Constraints: −24 ≤ u(t) ≤ 24, y(t) ≥ −5

• Desired controller = PID with gains KI = 0.248, KP = 0.752, KD = 2.237 " # y(t−1) KD u(t) = − KI I(t) + KP y(t) + (y(t) − y(t − 1)) y(t−2) Ts x(t) = I I − I(t−1) (t) = (t 1) + Tsy(t) u(t−1)

• Matching result (using inverse LQR): 6.401 0.064 −0.001 0.020 422.7 241.7 50.39 201.4 ∗ 0.064 6.605 0.006 0.080 ∗ ∗ 241.7 151.0 32.13 120.4 Q = −0.001 0.006 6.647 −0.020 ,R = 1,P = 50.39 32.13 19.85 26.75 0.019 0.080 −0.020 6.378 201.4 120.4 26.75 106.6

y 0

−10

−20 0 2 4 6 8 10 12 14 16 18 20 t

50 matched MPC

u1 0

−25

−50 0 2 4 6 8 10 12 14 16 18 20 t

20 what PID would apply 0 −20

y2 −40 −60 −80 −100 • Note: This is not trivially a 0 2 4 6 8 10 12 14 16 18 20 t

saturation of a PID controller. 50 In this case saturating the PID 25

u2 0

output leads to closed-loop −25

−50 instability! 0 2 4 6 8 10 12 14 16 18 20 t

"Model Predictive Control" - © A. Bemporad. All rights reserved. 126/132 Linear MPC based on linear programming Linear MPC based on LP (Propoi, 1963) (Bemporad, Borrelli, Morari, 2003) ( x ∈ Rn x = Ax + Bu • Linear prediction model: k+1 k k u ∈ Rm y = Cx k k y ∈ Rp • Constraints to enforce: ( umin ≤ u(t) ≤ umax ymin ≤ y(t) ≤ ymax

• Constrained optimal control problem (∞-norms): ∥v∥∞ , max |vi| i=1,...,n

NX−1 min ∥P xN ∥∞ + ∥Qxk∥∞ + ∥Ruk∥∞   z u0 k=0 R, Q, P u1 z =  .  full rank . u − s.t. umin ≤ uk ≤ umax, k = 0,...,N − 1 N 1 ymin ≤ yk ≤ ymax, k = 1,...,N

• Basic trick: introduce slack variables (Qi = i-th row of Q)

x ≥ i ϵk Q xk i = 1, . . . , n x ≥ − i − x ϵk Q xk k = 0,...,N 1 ϵ ≥ ∥Qx ∥∞ k k u ≥ i u ϵk R uk i = 1, . . . , m ϵ ≥ ∥Ru ∥∞ k k u ≥ − i − x ϵk R uk k = 0,...,N 1 ϵ ≥ ∥P x ∥∞ N k x ≥ i ϵN P xN i = 1, . . . , n x ≥ − i ϵN P xN

• Example: minx |x| → minx,ϵ ϵ s.t. ϵ ≥ x ϵ ≥ −x

kX−1 k i • Linear prediction model: xk = A x0 + A Buk−1−i i=0 • Optimization problem:

V (x0) = min [1 ... 1 0 ... 0]z (linear objective) z

s.t. Gz ≤ W + Sx0 (linear constraints)

Linear Program (LP) ′ ′ ′ , u u x x ∈ RN(nu+2) • optimization vector: z [ϵ0 . . . ϵN−1 ϵ1 . . . ϵN u0, . . . , uN−1]

• G, W , S are obtained from weights Q, R, P , and model matrices A, B, C • Q, R, P can be selected to guarantee closed-loop stability (Bemporad, Borrelli, Morari, 2003)

a0 x + b4 Result `(x) 4 a10 x + b1 Every convex piecewise affine function n ℓ : R → R can be represented as the a30 x + b3 max of affine functions, and vice versa (Schechter, 1987) x a20 x + b2 Example: ℓ(x) = |x| = max{x, −x} { ′ ′ } ℓ(x) = max a1x + b1, . . . , a4x + b4 • Constrained optimal control problem

ℓk, ℓN , gk, gN are arbitrary NX−1 convex piecewise affine (PWA) min ℓN (xN ) + ℓk(xk, uk) U k=0 functions

s.t. gk(xk, uk) ≤ 0, k = 0,...,N − 1 gN (xN ) ≤ 0

• Minimization of a convex PWA function ℓ(x):

ϵ minϵ,x ϵ ′  ϵ ≥ a x + b1  1 ϵ ≥ a′ x + b s.t. 2 2  ϵ ≥ a′ x + b  3 3 x ≥ ′ ϵ a4x + b4 ≥ { ′ ′ ′ ′ } • By construction ϵ max a1x + b1, a2x + b2, a3x + b3, a4x + b4 • By contradiction it is easy to show that at the optimum we have that

{ ′ ′ ′ ′ } ϵ = max a1x + b1, a2x + b2, a3x + b3, a4x + b4

• Convex PWA constraints ℓ(x) ≤ 0 can be handled similarly by imposing ′ ≤ ∀ aix + bi 0, i = 1, 2, 3, 4

• QP- and LP-based share the same set of feasible inputs (Gz ≤ W + Sx)

• When constraints dominate over performance there is little difference between them (e.g., during transients)

• Small-signal response, however, is usually less smooth with LP than with QP, because in LP an optimal point is always on a vertex

LP optimizer QP optimizer always on a same feasible set may lie vertex anywhere ( ) Gz ≤ W + Sx z : {z : Gz ≤ W + Sx} Gz¯ + Eϵ¯ ≤ W¯ + Sx¯

ϵ = additional slack variables introduced to represent convex PWA costs