Robustness Margin Computation for Large Scale Systems

Computers and Chemical Engineering 23 (1999) 1021–1030 www.elsevier.com/locate/compchemeng

Robustness margin computation for large scale systems

Richard D. Braatz *, Evan L. Russell

Department of Chemical Engineering, Large Scale Systems Research Laboratory, Uni6ersity of Illinois at Urbana-Champaign, 600 South Mathews A6enue, Box C-3, Urbana, IL 61801-3782, USA

Received 6 January 1997; accepted 23 March 1999

Abstract

Large scale systems have large numbers of inputs and outputs, and include whole chemical plants as well as some unit operations, such as paper machines, polymer ﬁlm extruders, and adhesive coaters. The importance of ensuring robustness of the closed loop system to model uncertainties increases as the process dimensionality increases; hence developing algorithms for computing robustness margins for large scale systems is of immense practical importance. Computational complexity is a tool of computer scientists which has had impact in understanding large scale optimization problems, both theoretically and in terms of ﬁnding computational solutions. Computational complexity theory is used to determine the level of accuracy and computational speed that are obtainable by algorithms for computing robustness margins, and as to which algorithms are likely for providing practical robustness margin computation for large scale systems. © 1999 Elsevier Science Ltd. All rights reserved.

Keywords: Large scale systems; Robustness; Closed loop system; Algorithms; Computational complexity

1. Introduction Braatz, 1997; Featherstone & Braatz, 1998a,b; Russell & Braatz, 1998a,b; VanAntwerp & Braatz, 1998). The Robust control for single loop and 2×2 chemical importance of ensuring robustness of the closed loop processes has been studied extensively over the past 15 system to model uncertainties increases as the process years (Palmor & Shinnar, 1981; Song, Fisher & Shah, dimensionality increases (Featherstone & Braatz, 1997; 1984; Laughlin, Jordan & Morari, 1986; Yousefpor, Russell & Braatz, 1998a; Featherstone & Braatz, Palazoglu & Hess, 1988; Doyle, Packard & Morari, 1998b); hence developing algorithms for computing ro- 1989; Kozub, MacGregor & Harris, 1989; Morari & bustness margins for large scale systems is of immense Zafiriou, 1989; Campo & Morari, 1990; Zafiriou & practical importance. This explains why researchers Marchal, 1991; Arkun & Calvet, 1992; Schaper, Seborg have spent many man-centuries working to derive effi- & Mellichamp, 1992; Amann & Allgower, 1994; Horn, cient numerical algorithms for computing robustness Arulandu, Gombas, VanAntwerp & Braatz, 1996; margins (a man-century refers to one man working 40 h Skogestad & Postlethwaite, 1996; Stryczek & Brosilow, per week for one century). In this paper computational 1996). On the other hand, whole chemical plants as well complexity theory (described below) is used to deter- as some unit operations, such as paper machines, poly- mine the level of accuracy and computational speed mer film extruders, and adhesive coaters have large that are obtainable by algorithms for computing ro- numbers of inputs and outputs. More recently chemical bustness margins, and as to which algorithms are likely engineers have become interested in applying rigorous control systems techniques to these large scale systems to provide practical robustness margin computation for (Ricker & Lee, 1995; Dave, Willig, Kudva, Pekny & large scale systems. Doyle, 1997; Rao, Campbell, Rawlings & Wright, 1997; The purpose of computational complexity theory is Rigopoulos, Arkun & Kayihan, 1997; Russell, Power & to characterize the inherent difficulty of calculating the solution for a problem under study. This theory can be used to characterize computational problems as being * Corresponding author. Tel.: +1-217-3335073; fax: +1-217- 3335052. in one of two classes: P and NP-hard. The class P refers E-mail address: [email protected] (R.D. Braatz) to problems in which the exact time needed to solve the

0098-1354/99/$ - see front matter © 1999 Elsevier Science Ltd. All rights reserved. PII: S0098-1354(99)00268-9 1022 R.D. Braatz, E.L. Russell / Computers and Chemical Engineering 23 (1999) 1021–1030 problem can be bounded by a single function which is though such algorithms are of interest, the main result polynomial in the amount of data needed to define the implies that there exist large scale uncertain systems for problem. Such computational problems are said to be which such algorithms will grossly underestimate the solvable in polynomial-time, and include linear and size of the robust performance margin. Practical al- convex quadratic programs. gorithms for computing robustness margins should be Although the exact consequences of a problem being based on rigorous upper and lower bounds that are NP-hard is still a fundamental open question in the tight for typical systems, although the main result im- theory of computational complexity, it is generally ac- plies that such bounds will not be tight for some cepted that a problem being NP-hard means that it problems. cannot be computed in polynomial time in the worst All proofs require only basic linear algebra and fol- case. It is important to understand that being NP-hard low from relating robustness margin computation with is a property of the problem itself, not of any particular some specialized nonconvex programs. Given that com- algorithm. Whether a problem is polynomial-time or putational complexity theory is not usually studied by NP-hard determines the highest computational effi- chemical engineers, we provide a brief introduction to ciency which can be expected by any algorithm, and as its basic principles. This is followed by the results, and to which classes of algorithms can provide practical a discussion of how the results relate to other results in solutions to the problem. For example, an engineer the literature. may look to interior point algorithms (Nesterov & Nemirovskii, 1994) for computing the solution to a 1.1. Notation polynomial-time optimization problem; whereas would consider using approximations, heuristics, or branch- In what follows, matrices are upper case and vectors and-bound techniques for computing the solution for and scalars are lower case. The set of real numbers is R; an NP-hard optimization problem (Papadimitriou & the set of complex numbers is C, and the set of ratio- Steiglitz, 1982; McCormick, 1983; Ryoo & Sahinidis, nale is Q. The Euclidean 2-norm of vector x is defined T 1996; VanAntwerp, Braatz & Sahinidis, 1998). The by x 2 = x x, whereas the vector -norm of x is solution to an NP-hard optimization problem takes defined by x =max xi . The maximum singular s much longer to compute, and for some large scale value of matrix A is represented by (A), and Ir is the problems even a supercomputer may be unable to find r×r identity matrix. An ‘0’ will be used to represent the global solution to high accuracy within a reasonable either zero or a matrix of zeros. amount of time (Garey & Johnson, 1983). It is known that the exact computation of the robustness margin is NP-hard (Braatz, Young, Doyle & 2. Introduction to computational complexity theory Morari, 1994). Although the general v recognition problem is NP-hard, special cases (that is, with restric- Here we provide a description of computational com- tions on the structure or field of M or D) may be plexity theory that is concise and useful for practi- simpler to compute. For example, when the M matrix is tioners. This includes a discussion of how this theory restricted to be rank one, the calculation of v has can be applied to -approximation problems, as such sublinear growth in problem size, irrespective of the descriptions are not provided in the standard textbooks perturbation structure (Chen, Fan & Nett, 1991). In (Papadimitriou & Steiglitz, 1982; Garey & Johnson, this manuscript we provide the simplest known proof 1983). that robustness margin computation is NP-hard even Computational complexity theory allows a character- when all of the uncertainties are represented as inde- ization of the inherent difficulty of calculating the solu- pendent real parameter variations. This is followed by a tion for a problem under study. Problems (or related result that robustness margin computation is equivalent versions of the same problem) are generally NP-hard for the case of one complex scalar uncertainty characterized as being in one of two classes: P and and independent real parameter variations. These re- NP-hard. The class P refers to problems in which the sults indicate that computing the robustness margin exact time needed to solve the problem can always be even for systems with very simple uncertainty structures bounded by a single function which is polynomial in is NP-hard. the amount of data needed to define the problem. Such The main result of this manuscript is that even problems are said to be solvable in polynomial time. approximating the robust performance margin is a hard Although the exact consequences of a problem being computational problem. This result is especially impor- NP-hard is still a fundamental open question in the tant in light of recently-proposed techniques for robust- theory of computational complexity, it is generally ac- ness margin computations using Monte Carlo methods cepted that a problem being NP-hard means that it and other methods of testing only a subset of the plants cannot be computed in polynomial time in the worst in the uncertainty set (Barmish & Lagoa, 1997). Al- case. It is important to understand that being NP-hard R.D. Braatz, E.L. Russell / Computers and Chemical Engineering 23 (1999) 1021–1030 1023 is a property of the problem itself, not of any particular 2.2. The classes P and NP algorithm. It is also important to understand that having a problem be NP-hard does not imply that practical The classes P and NP refer to the recognition 6ersions algorithms are not possible. Practical algorithms for only. The class P refers to the class of recognition NP-hard problems exist and typically involve approxi- problems that can be answered by a polynomial-time mation, heuristics, branch-and-bound, or local search algorithm. An example would be the linear program- (Papadimitriou & Steiglitz, 1982; Garey & Johnson, ming recognition problem. 1983; McCormick, 1983; Ryoo & Sahinidis, 1996). De- Researchers believe that class NP is richer than the termining whether a problem is polynomial-time or class P (P"NP). For a problem to be in NP, it is not NP-hard tells the large scale systems engineer what kind required that it can be answered in polynomial time by of computational efficiency can be expected by any an algorithm. It is only required that, if x is a yes algorithm, and what kinds of algorithms to investigate instance of the problem, then this x can be checked in for providing practical solutions to the problem. polynomial time for validity. In other words, a problem is in NP if we can calculate fw(x) and check that x in 2.1. Three 6ersions of an optimization problem polynomial time. This condition is usually referred to as polynomial-time 6erifiability. This definition automati- Consider the optimization problem cally implies that P is a subset of NP. If a recognition problem is in the class NP, then the sup fw(x), (1) x w where w as the vector of data required to uniquely complement is in the class co-NP. The co-NP problem related to our optimization problem would be: ‘Given specify an instance of an optimization problem, and the B k,isfw(xˆ ) k?’ It is unclear how to verify if this feasible region w and objective function fw(x) are functions of the problem data. For the optimization problem is answered by yes, except by solving the problems considered in this manuscript, the feasible evaluation version of the optimization problem. This region will be non-empty, and compact (closed and certainly appears to be more difficult than verifying if w the recognition version is in NP. For this reason re- bounded). Then the supremum is achieved by at least " one x in the feasible region: searchers believe that NP co-NP. ( fw = sup fw(x)=max fw(x) (2) x w x w 2.3. NP-complete and NP-hard problems An optimization problem can be written in three closely-related but different versions (Papadimitriou & NP-complete problems are the hardest problems in Steiglitz, 1982): NP. Examples of problems that are NP-complete are 6 P1. The optimization ersion: find the optimal feasible the traveling salesman problem, the max-cut problem, solution x, that is, such that fw(x) is maximized. and the indefinite quadratic programming problem. 6 6 ( P2. The e aluation ersion: Find the value fw =fw(xˆ ) Almost all researchers believe that NP-complete prob- of the optimal solution. lems are harder than P problems, but there is no proof. 6 P3. The recognition ersion: Given k, is there a feasi- Any problem that is at least as hard as an NP-com- ] ble solution x such that fw(x) k? plete problem is said to be NP-hard.AnNP-hard The recognition version can also be written as: problem can refer to much broader classes of problems ] ‘Given k,isfw(xˆ ) k?’ The recognition version is im- than recognition problems (Garey & Johnson, 1983). In portant for studying the complexity of the problem, particular, if the recognition version of an optimization because it is the type of problem traditionally studied is NP-complete, then the corresponding evaluation and by the theory of computation. Unlike the first two optimization versions are NP-hard. versions, the recognition version is a question, which can be answered by yes or no. The recognition 6ersion is no harder than the e6alua- 2.4. Computational complexity of -approximation tion 6ersion (P35P2), since we can immediately deduce problems 6 the answer to the recognition ersion from the answer to the e6aluation 6ersion. In other words, the evaluation Because we have assumed that the feasible region w, problem is at least as difficult to solve as the recogni- in Eq. (1) is non-empty, and compact, the infimum is tion problem, so if a recognition problem is NP-hard, achieved by at least one x in the feasible region: then the corresponding evaluation problem must also fw = inf fw(x)=min fw(x) (3) be difficult. x w x w If fw(x) is easy to compute, then the evaluation Then the -approximation problem for Eq. (1) is to version cannot be significantly harder than the opti- 0 compute a value fw R for which mization version (P25P1). Thus the ordering from 5 5 ( 0 5e ( least difficult to most difficult is P3 P2 P1. fw −fw fw −fw . (4) 1024 R.D. Braatz, E.L. Russell / Computers and Chemical Engineering 23 (1999) 1021–1030

This definition is preferable to others in the literature Lemma 1. (Polynomial programming with box con- in that it is invariant to translation and dilation (that is, straints) There is a constant k\0 such that the follow- multiplying by and adding a constant) of the objective ing is true. If there exists a polynomial-time algorithm function (Vavasis, 1992; Bellare & Rogaway, 1993; that can compute an -approximation for Eq. (6), Vavasis, 1993). Quantifying the optimization objective where (n)=1−n−g, then P=NP. in different units does not affect the quality of the approximation as measured by Eq. (4). Proof 1. Follows directly from the proof of Theorem A 0-approximation algorithm provides the exact op- 3.1 in (Bellare & Rogaway, 1993). QED. timal solution, while a 1-approximation algorithm need only find any feasible point and compute its objective. The general consensus in the computational commu- We will study the existence of polynomial-time al- nity that P"NP implies that the -approximation gorithms for computing an -approximation. Let n be problem for polynomial programming with box con- a measure of the quantity of data needed to describe an straints is also hard. Note that the particular form of instance of an optimization problem (for example, this Eq. (6) in Lemma 1 considers very weak forms of could be the number of elements in the data vector w, approximation, as it allows the quality of the approxi- or the number of rows in a matrix which contains most mation to degrade as a function of problem size. In of the data). To provide the strongest results, will be particular, for fixed k and large n, (n) in Lemma 1 selected to be a function of n—this allows the consider- approaches one, which only requires that the approxi- ation of algorithms for which the accuracy of the mation algorithm be able to find a feasible point and approximation degrades as the size of the problem evaluate the corresponding objective function. Thus (measured by n) increases. Lemma 1 indicates that the existence of even a weak Even when the exact optimization problem is NP- polynomial-time approximation algorithm for polyno- hard, the -approximation problem can be either easy mial programs with box constraints is highly unlikely. or hard (Ausiello, D’Atri & Protasi, 1980). For an example of a hard exact optimization problem that has an easy -approximation algorithm, consider the class of concave quadratic programming problems 3. Robust stability and robust performance margins min x THx+p Tx (5) Ax5b Measuring the robustness of uncertain systems is of where H is of rank one. The exact optimization prob- immense practical importance. The standard procedure lem is NP-hard (Pardalos & Vavasis, 1991), whereas an for computing robust stability margins is to first con- algorithm exists that computes an -approximation in vert the description of the closed loop system into a polynomial-time as a function of problem size (Vavasis, v-computation problem (procedures for doing this are 1992). As another example, while the exact knapsack described in (Morari & Zafiriou, 1989; Russell & problem is NP-hard, its solution can be approximated Braatz, 1996; Skogestad & Postlethwaite, 1996; Russell in polynomial-time within e6ery constant factor of the et al., 1997)). optimum (Barland, Kolaitis & Thakur, 1996). Define the set D of block diagonal perturbations by The well-known traveling salesman problem is NP- ! hard (Johnson & Papadimitriou, 1989), and its -ap- D= diag {l rI ,…,l r I , l c I ,…,l c I , D 1 r 1 k rk k+1 rk+1 m rm rm+1 proximation problem is also hard (Sahni & Gonzales, l " 1976). More specifically, finding a polynomial-time al- ,…, D } l r R; l c C; D C ri×ri ; % r =n . (7) rl i i ri i gorithm that approximates the optimum within a given i=1 n×n constant is equivalent to establishing that P=NP Let MC Then the structured singular value vD(M) (Sahni & Gonzales, 1976; Kolaitis & Thakur, 1993; is defined as (Fan, Tits & Doyle, 1991) Reinelt, 1994; Yannakakis, 1994). As another example, vD(M) consider the class of polynomial programs with box constraints {0 if there does not exist D t ! " D such that det(I−MD)=0, max % 5 x 5 (1−x ) (6) 05x51 i j k=1 iA jB ! n k k vD(M) min{|¯ (D) det(I−MD)=0} otherwise where t is the number of terms in the polynomial DD objective, n is the number of optimization variables, (8) and for each k, A and B are disjoint subsets of {1, …, k k Without loss of generality we have taken M and each n}. The exact optimization problem is NP-hard. The subblock of D to be square. -approximation problem is also hard, as stated in the following lemma. R.D. Braatz, E.L. Russell / Computers and Chemical Engineering 23 (1999) 1021–1030 1025

In a similar manner, robust performance margins are where CQ m×m is a ﬁxed constant invertible matrix computed by converting the system description into a with rational elements. The task of determining skewed-m computation problem, where skewed-v has whether all matrices in M are nonsingular is NP-hard. the same deﬁnition as v, except with rows and/or Poljak and Rohn (Poljak and Rohn, 1993) had ear- columns of M (Eq. (8)) re-scaled (Smith, 1990; Braatz lier proved a closely related result, but had a much & Morari, 1991; Fan & Tits, 1992): more complicated approach. v s We will use Lemma 3 to show that exact computa- v D(M) tion is NP-hard for the class of v problems with DD | D v {0 if there does not exist such that ¯ ( 1) independent real parameter variations. Consider with MQ n×n, kQ, and mixed real/complex uncertainty 51 and det(I−MD)=0, blocks. ! ! D n s 1 0 v D(M) min |¯ (D ) |¯ (D )51; det I−M DD 2 1 D Theorem 1 (NP-hardness of robust stability margin com- 0 2 "n putation with independent real perturbations). v-compu- =0 −1 otherwise (9) tation with nonrepeated real perturbations is NP-hard.

D D D Proof 2. Consider the class of matrices M deﬁned in Eq. where =diag{ 1, 2} The reader is directed to the citations for more details; here we would just like to (11). All matrices in M are nonsingular if and only if note that any algorithm for approximately computing v can be used to directly approximately calculate skewed- C zn det= "0, Ö y 5h; Ö z 5h (12) v and vice versa (Packard, 1988; Skogestad & Morari, y T 1 1988; Smith, 1990; Braatz & Morari, 1991; Fan & Tits, 1992; Braatz, Morari & Skogestad, 1996). I zn C 0n Udet m det "0, y TC−1 1 0 1

4. Computational complexity of exact robustness Ö y 5h, Ö z 5h (13) margin computation 0 −zn Udet I − "0, Braatz et al. (1994) and Coxson and DeMarco (1992) m+1 −y TC−1 0 proved that exact v-computation is NP-hard. The Ö 5h Ö 5h Braatz et al. proof did not prove NP-hardness for y , z (14) classes of v problems with independent real parameter Dr Dr a 6 variations. The Coxson and DeMarco proof did ad- Deﬁne z=− zw and y=− yw, where w=− k1 Dr Dr dress independent real parameter variations, but was and z and y are real diagonal perturbation matrices lengthy. The following approach addresses independent with independent real scalar uncertainties. Then Eq. real perturbations, while providing a much shorter path (14) is equivalent to between a basic computation problem (Lemma 2) and robust margin computation than provided by Coxson Dr n 0 zw r Udet I− "0, Ö D 51/k, and DeMarco. w TDrC−1 0 z First we will recall a well-known NP-complete prob- y Ö Dr 5 lem (Garey & Johnson, 1983): y 1/k (15)

Lemma 2. Given an integer m\0 and aQ m, a 5 0 In Dr 0 n C−1 0 n 2 Udet I− y "0, 0.1, with positive entries, the task of determining T Dr w 0 0 z 0 w whether the equation Ö Dr 5 Ö Dr 5 z 1/k, y 1/k (16) m T % a t= aiti =0 (10) i=1 C−1 0 n 0 In Dr 0 n Udet I− y "0, Ö T Dr has a solution t with ti {−1, 1}, i=1,…, m,isNP- 0 w w 0 0 z complete. Nemirovskii (Nemirovskii, 1993) used this r r Ö D 51/k, Ö D 51/k (17) result to prove that the following problem is NP-hard. z y 0 C−1n Dr 0 n Lemma 3. Consider the set of matrices Udet I− y "0, ww T 0 0 Dr ! n) " z C z m M= y, zQ ; y 5h; z 5h (11) T Ö Dr 5 Ö Dr 5 y 1 z 1/k, y 1/k (18) 1026 R.D. Braatz, E.L. Russell / Computers and Chemical Engineering 23 (1999) 1021–1030

s DD(M)]kUvD(M0)]k, (22) Uv B Dr(M) k, (19) (Fan & Tits, 1992), thus the complexity of the skewed-m where problem is equivalent to the complexity of the v-computation problem. The v-computation problem for this 0 C−1n M= (20) perturbation class is NP-hard by corollary 1. QED. ww T 0 and

Dr 0 n 5. Computational complexity of ·-approximate Dr = y . (21) Dr robustness margin computation 0 z Thus there exists a singular matrix in M if and only Here we show that it is hard to compute robust v ] if Dr(M) k. Determining whether there exists a singu- performance margins within a given . lar matrix in M is NP-hard by Lemma 3, and the above The ﬁrst step in our development is to show that the equivalences show that the nonsingularity task polyno- polynomial program with box constraints (Eq. (6)) can mially reduces to a v problem with real M and indepen- be represented as a skewed-v problem. dent real scalars. QED. Lemma 4 (Polynomial program with box constraints Models for real systems always have unmodeled dy- reduces to robustness margin problems). The polynomial namics associated with them. Unmodeled dynamics program with box constraints polynomially reduces to correspond to having at least one complex uncertainty real v, mixed v, real skewed-v, and mixed skewed-v in the v problem. The next result states that v-compu- problems. tation is NP-hard for this practically-motivated class of problems. Proof 5. The proof is trivial for k=0, so consider the case where k\0. Treat the constraints as uncertainty Corollary 1. (NP-hardness of robust stability margin and the objective function as the performance objective computation with mixed perturbations) v-computation of a robust performance problem. The constraint set is problems with one complex scalar perturbation with the {x 05x51} remainder being nonrepeated real scalar perturbations is NP-hard. ={x x=x¯ +Dxw; Dx =diag[l r,…, l r]; l r [−1/k,1/k]}, (23) Proof 3. Because the v problem in the proof of Theo- 1 n i rem 1 has real M and all perturbations are 1×1, where application of Schur’s determinant formulae (Zhou, Doyle & Glover, 1995) implies that any one of the real x¯ =(1/2)1 (24) perturbations can be replaced by a complex scalar and perturbation without affecting the value of the robustness margin. QED. w=(k/2)1 (25) The realization algorithm of Russell et al. (1997) That the computation of robust performance margins computes in polynomial time a matrix M that satisﬁes is also a hard problem immediately follows. t ! " % 5 5 Dr fw(x):= xi (1−xj) =Fu(M, ) Corollary 2. (NP-hardness of robust performance mar- k=1 i Ak j Bk gin computation with independent real perturbations). Dr = Fu(M, ) , (26) Robust performance margin computation with nonrepeated real scalar perturbations is NP-hard. where MQ (m+1)×(m+1), Dr ={diag [l r,…, l r ] l r Proof 4. The robust performance margin for a system 1 m j l r with nonrepeated real scalar perturbations can be writ- R; each j possibly repeated}, (27) ten directly in terms of a skewed-v problem with the t % same number of nonrepeated real scalar perturbations m= Ak + Bk , (28) plus one additional complex perturbation (Smith, 1990; k=1

Fan & Tits, 1992). The main loop theorem (Theorem Ak refers to the number of elements in the vector of

5.2 of Packard (1988)) allows the polynomial-time con- indices Ak, and the linear fractional transformation 0 Dr struction of M such that Fu(M, ) is deﬁned by R.D. Braatz, E.L. Russell / Computers and Chemical Engineering 23 (1999) 1021–1030 1027

Dr Dr Dr −1 Fu(M, )=M22 +M21 (I−M11 ) M12. (29) box constraints. From Lemma 1 this implies that P= NP. QED. Then the polynomial program with box constraints can be written as For all practical purposes, computing approxima- Dr tions of v is just as hard as computing approximations 5max5 fw(x)= max Fu(M, ) (30) 0 x 1 Dr 51/k of skewed-v, since either function can be approximated Now apply the robust performance theorem (Doyle, by the other using bisection (Packard, 1988; Skogestad 1982) to give & Morari, 1988; Fan & Tits, 1992), which is a polyno- t ! " mial-time operation (Khachiyan & Todd, 1993; Boyd, v ] U % 5 5 ] D(M) k 5max5 xi (1−xj) k, 0 x 1 El Ghaoui, Feron & Balakrishnan, 1994). k=1 i Ak j Bk (31) where D l r l r l c l r l c 6. Comparison with other results ={diag [ 1,…, m, m+1] j R; m+1 l r C; each j possibly repeated}. (32) In a rather lengthy proof, Coxson and DeMarco Similarly, apply the skewed-v main loop theorem (1992) used the results of Poljak and Rohn to prove (Smith, 1990; Fan & Tits, 1992) to give that exact v-computation with independent real 0Dr v s 0 parameter variations is NP-hard. Braatz et al. (1994) 5max5 fw(x)= max Fu(M, ) = D(M) (33) 0 x 1 Dr 51/k proved NP-hardness of v for the following classes of where M0is equal to M but with some of its rows scaled systems: by k. Using the fact that M is real, the Schur determi- 1. systems with pure real perturbations; nant formula can be used to show that in both cases the 2. systems with mixed perturbations in which the com- complex scalar perturbation l c can be replaced by a plex block enters nontrivially in the robustness mar- real scalar perturbation. gin computation; and Since 3. systems for which the robustness margin computa- t t tion is a continuous function of the problem data. % 5 % m= Ak + Bk n=tn, (34) The result for classes 2 and 3 implied that the NP- k=1 k=1 hardness of v is rather generic. Theorem 1 and corollary the quantity of data needed to describe the v and 1 are stronger results than those of Braatz et al. (1994), skewed-v problems is bounded by a polynomial func- because they show NP hardness with nonrepeated real tion of the quantity of data needed to describe the perturbations. Toker and Ozbay (1995) proved that v polynomial program with box constraints. QED. computation is NP-hard for systems with pure complex scalar uncertainties. Taken together, these results indi- The main result applies Lemma 4 to show that the cate that the is NP-hard for nearly any practical class of -approximation of robust performance margins is perturbations, that is, NP-hardness is an inherent prop- hard. erty of worst-case robust stability analysis. Corollary 2 indicates that the same is true for robust performance Theorem 2 (-Approximation of robust performance analysis. margins is hard). Consider the skewed-v problem with: The NP-hardness of robustness margin computation (i) all real scalar uncertainties; (ii) one complex and the is not a property of the problem formulation—the same rest real scalar uncertainties; or (iii) any superset that result holds when the robustness margin is written in contains these uncertainty descriptions. There is a con- terms of km (Safonov, 1982; Chiang & Safonov, 1992) stant k\0 such that the following is true. If there exists or other formalisms. Subclasses for which robustness a polynomial-time algorithm that can compute an -ap- margin computation is polynomial time are problems proximation for skewed-v where (n)=1−n−g, then where M has rank 1 (Kharitonov, 1978; Bartlett, Hollot P=NP. & Lin, 1989; Chen et al., 1991; Barmish, 1993; Braatz & Crisalle, 1998), and for processes with special types of Proof 6. Suppose there exists a polynomial-time -ap- transfer function matrices and uncertainty structures proximation algorithm for skewed-v. Apply the al- (Braatz & VanAntwerp, 1996; Hovd, Braatz & Skoges- gorithm to the skewed-v problem deﬁned in the proof tad, 1996). Any realistic large scale chemical process will of Lemma 4. The one-to-one correspondences between ha6e a corresponding M matrix of much higher rank than Dr and x and their objective functions (Eq. (26)) imply one. The class of systems considered by (Braatz & that the polynomial-time -approximation algorithm VanAntwerp, 1996; Hovd et al. 1996) is of industrial for skewed-v could be used to provide a polynomial- relevance, but the results cannot be extended to pro- time -approximation for the polynomial program with cesses with general transfer function matrices. 1028 R.D. Braatz, E.L. Russell / Computers and Chemical Engineering 23 (1999) 1021–1030

The result that computing approximate robust per- Computational complexity theory was applied to the formance margins is hard (Theorem 2) is truly a power- problems of robust stability and performance margin ful result. The previous results concerning the computation. The main results imply that it is highly computational complexity of robustness margin compu- unlikely that polynomial-time algorithms exist that can tation (Demmel, 1992; Nemirovskii, 1993; Poljak & calculate robustness margins either exactly or within a Rohn, 1993; Braatz et al., 1994; Toker & Ozbay, 1995) useful desired level of accuracy for all problems. These do not imply this. This is because the requirements for results are especially important in light of recently-pro- relating the NP-hardness of one approximation prob- posed techniques for robustness margin computations lem to another are much stricter than for relating the using Monte Carlo methods and other methods of NP-hardness of their exact problems (see, for example, testing only a subset of the plants in the uncertainty set the discussion on page 134 of Garey and Johnson (Barmish & Lagoa, 1997). Although such algorithms (1983) for a basic discussion, or the papers (Feige, are of interest, the main result implies that there exist Goldwasser, Lovasz, Safra & Szegedy, 1991; Arora & large scale uncertain systems for which such algorithms Safra, 1992; Arora, Lund, Motwani, Sudan & Szegedy, will grossly underestimate the size of the robust perfor- 1992; Feige & Lovasz, 1992; Bellare & Rogaway, 1993; mance margin. Kolaitis & Thakur, 1993; Zuckerman, 1993; Khanna, Practical algorithms for computing robustness mar- Motwani, Sudan & Vazirani, 1994; Tardos, 1994; Yan- gins should be based on rigorous upper and lower nakakis, 1994; Agarwal & Condon, 1995; Arora, 1995; bounds that are tight for typical systems, although the Bellare, Goldreich & Sudan, 1995; Feige & Verbitsky, main result implies that such bounds will not be tight 1996) for more details. for some problems. The feasibility of this approach for The exact and approximate computational complex- systems with up to a few hundred uncertain parameters ity results for v directly generalize to related robustness has been demonstrated in computational studies analysis and synthesis problems, using techniques de- (Young & Doyle, 1990; Young, Newlin & Doyle, 1991; scribed by Toker and Ozbay (1995). In particular, both Newlin, 1996). Current efforts are underway to develop the exact and approximate computation of polynomial-time model reduction algorithms that can the supremum of v of a transfer function evaluated substantially reduce the computational expense associ- as a function of frequency, and ated with robustness margin computation for typical large scale systems. Preliminary results indicate that v-optimal controllers such an approach can enable robustness margins to be are hard problems. Also, any such robustness problem computed for systems with up to a thousand uncertain which contains a superset of the uncertainty structures parameters (Russell & Braatz, 1998b). considered here are also hard. For example, exact or approximate v-computation for systems with two complex scalar perturbations, one full complex block, and References repeated real scalar perturbations are hard problems. Any easy proof of this is to define a class of matrices M Agarwal, S., & Condon, A. (1995). On approximation algorithms for which have zero rows and columns corresponding to hierarchical MAX-SAT. In Proceedings of the 10th annual struc- the extra complex and repeated real perturbations, so ture in complexity theory conference (pp. 214–222). Minneapolis, that v for the overall matrix M is equal to v for a lower MN. v dimension matrix with independent real scalar pertur- Amann, N., & Allgower, F. (1994). -Suboptimal design of a ro- bustly performing controller for a chemical reactor. International bations and one complex perturbation. Then apply Journal of Control, 59, 665–687. Corollary 1 and Theorem 2. Arkun, Y., & Calvet, J.-P. (1992). Robust stabilization of input/output linearizable systems under uncertainty and disturbances. AIChE Journal, 38, 1145–1156. Arora, S. (1995). Reductions, codes, PCPs, and inapproximability. In 7. Conclusions Proceedings of the 36th annual symposium on foundations of computer science (pp. 404–413). Milwaukee, WI: IEEE Computer Society Press. Computational complexity theory provides a power- Arora, S., Lund, C., Motwani, R., Sudan, M., & Szegedy, M. (1992). ful guide for solving the computational problems that Proof verification and hardness of approximation problems. In arise when applying rigorous optimization-based ap- Proceedings of the 33rd annual symposium on foundations of computer science (pp. 14–23). Pittsburgh, PA: IEEE Computer Soci- proaches to large scale systems. When a problem is ety Press. determined to be in the class P, then computationally- Arora, S., & Safra, S. (1992). Probabilistic checking of proofs. In efficient polynomial-time algorithms can be developed. Proceedings of the 33rd annual symposium on foundations of com- When a problem is determined to be NP-hard, then its puter science (pp. 2–13). Pittsburgh, PA: IEEE Computer Society computation is more difficult and practical algorithms Press. Ausiello, G., D’Atri, A., & Protasi, M. (1980). Structure preserving must be based on heuristics, branch-and-bound, and/or reductions among convex optimization problems. Journal of Com- local search. puter and Systems Sciences, 21, 136–153. R.D. Braatz, E.L. Russell / Computers and Chemical Engineering 23 (1999) 1021–1030 1029

Barland, I., Kolaitis, P. G., & Thakur, M. N. (1996). Integer pro- Featherstone, A. P., & Braatz, R. D. (1997). Control-oriented model- gramming as a framework for optimization and approximability. ing of sheet and film processes. AIChE Journal, 43, 1989–2001. In Proceedings of the ele6enth annual IEEE conference on compu- Featherstone, A. P., & Braatz, R. D. (1998a). Input design for large tational complexity (pp. 249–259). Philadelphia, PA. scale sheet and film processes. Industrial Engineering and Chemical Barmish, B. R. (1993). New tools for robustness of linear systems. New Research, 37, 449–454. York: MacMillan. Featherstone, A. P., & Braatz, R. D. (1998b). Integrated robust Barmish, B. R., & Lagoa, C. M. (1997). The uniform distribution: a identification and control of large scale processes. Industrial Engi- rigorous justification of its use in robustness analysis. Mathemat- neering and Chemical Research, 37, 97–106. ics of Control, Signals, and Systems, 10, 203–222. Feige, U., Goldwasser, S., Lovasz, L., Safra, S., & Szegedy, M. Bartlett, A. C., Hollot, C. V., & Lin, H. (1989). Root locations of an (1991). Approximating clique is almost NP-complete. In Proceed- entire polytope of polynomials: it suffices to check the edges. ings 32nd annual symposium on foundations of computer science Mathematics of Control, Signals, and Systems, 1, 61–71. (pp. 2–12). San Juan, Puerto Rico. Bellare, M., Goldreich, O., & Sudan, M. (1995). Free bits, PCPs, and Feige, U., & Lovasz, L. (1992). Two-prover one-round proof systems: non-approximability—towards tight results. In Proceeding of the their power and their problems. In Proceedings of the 24th annual 36th annual symposium on foundations of computer science (pp. ACM symposium on the theory of computing (pp. 733–744). 422–431). Milwaukee, WI: IEEE Computer Society Press. Victoria, BC, Canada. Bellare, M., & Rogaway, P. (1993). The complexity of approximating Feige, U., & Verbitsky, O. (1996). Error reduction by parallel repeti- 6 a nonlinear program. In P. M. Pardalos, Complexity in numerical tion-a negative result. In Proceedings of the ele enth annual IEEE optimization (pp. 16–32). Singapore: World Scientific. conference on computational complexity (pp. 70–76). Philadelphia, Boyd, S., El Ghaoui, L., Feron, E., & Balakrishnan, V. (1994). Linear PA. matrix inequalities in system and control theory. In Studies in Garey, M. R., & Johnson, D. S. (1983). Computers and Intractibility: applied mathematics, vol. 15. Philadelphia, PA: SIAM. a guide to NP-completeness. New York: W.H. Freeman. Braatz, R. D., & Crisalle, O. D. (1998). Robustness analysis for Horn, I. G., Arulandu, J. R., Gombas, C. J., VanAntwerp, J. G., & systems with ellipsoidal uncertainty. International Journal of Ro- Braatz, R. D. (1996). Improved filter design in internal model bust and Nonlinear Control, 8, 1113–1117. control. Industrial Engineering and Chemical Research, 35, 3437– Braatz, R. D., & Morari, M. (1991). v-Sensitivities as an aid for 3441. robust identification. In Proceedings of the American control con- Hovd, M., Braatz, R. D., & Skogestad, S. (1996). SVD controllers for H , H , and v-optimal control. Automatica, 33, 433–439. ference (pp. 231–236). Piscataway, NJ: IEEE Press. 2− − Johnson, D. S., & Papadimitriou, C. H. (1989). Computational Braatz, R. D., Morari, M., & Skogestad, S. (1996). Loopshaping for complexity. In E. L. Lawler, J. K. Lenstra, A. H. G. Rinnooy robust performance. International Journal of Robust and Nonlinear Kan, & D. B. Shmoys, The tra6eling salesman problem (pp. Control, 6, 805–823. 37–85). Chichester: Wiley. Braatz, R. D., & VanAntwerp, J. G. (1996). Robust cross-directional Khachiyan, L. G., & Todd, M. J. (1993). On the complexity of control of large scale paper machines. In Proceedings of the IEEE approximating the maximal inscribed ellipsoid for a polytope. international conference on control applications (pp. 155–160). Mathematical Programming, 61, 137–159. Piscataway, NJ: IEEE Press. Khanna, S., Motwani, R., Sudan, M., & Vazirani, U. (1994). On Braatz, R. D., Young, P. M., Doyle, J. C., & Morari, M. (1994). syntactic versus computational views of approximability. In Pro- Computational complexity of v calculation. IEEE Transactions on ceedings 35th annual symposium on foundations of computer science Automatic Control, 39, 1000–1002. (pp. 819–830). Santa Fe, NM. Campo, P. J., & Morari, M. (1990). Robust control of processes Kharitonov, V. L. (1978). Asymptotic stability of an equilibrium subject to saturation nonlinearities. Computers and Chemical En- position of a family of systems of linear differential equations. gineering, 14, 343–358. Differential’nye Ura6eniya, 14, 1483–1485. Chen, J., Fan, M. K. H., & Nett, C. N. (1991). The structured Kolaitis, P. G., & Thakur, M. N. (1993). Polynomial-time optimiza- singular value and stability of uncertain polynomials: a missing tion, parallel approximation, and fixpoint logic. In Proceedings of link. In: ASME Annual Winter Meeting, Atlanta, GA. the 8th annual structure in complexity theory conference (pp. Chiang, R. Y., & Safonov, M. G. (1992). Robust control toolbox: for 31–41). San Diego, CA. use with MATLAB. Natick, MA: The Math Works. Kozub, D. J., MacGregor, J. F., & Harris, T. J. (1989). Optimal IMC Coxson, G., & DeMarco, C. (1992). Computing the real structured inverses: design and robustness considerations. Chemical Engi- 6 singular alue is NP-hard. Report ECE-92-4, Department of Elec- neering Science, 44, 2121–2136. trical and Computer Engineering, University of Wisconsin- Laughlin, D. L., Jordan, K. G., & Morari, M. (1986). Internal model Madison. control and process uncertainty-mapping uncertainty regions for Dave, P., Willig, D. A., Kudva, G. K., Pekny, J. F., & Doyle, F. J. SISO controller-design. International Journal of Control, 44, (1997). LP methods in MPC of large scale systems: application to 1675–1698. paper-machine CD control. AIChE Journal, 43, 1016–1031. McCormick, G. P. (1983). Nonlinear programming. theory, algorithms, Demmel, J. W. (1992). The component-wise distance to the nearest and applications. New York: Wiley Interscience. singular matrix. SIAM Journal of Matrix Analysis and Applica- Morari, M., & Zafiriou, E. (1989). Robust process control. Englewood tions, 13, 10–19. Cliffs, NJ: Prentice-Hall. Doyle, J. C. (1982). Analysis of feedback systems with structured Nemirovskii, A. (1993). Several NP-hard problems arising in robust uncertainties. IEE Proc. Pt. D., 129, 242–250. stability analysis. Mathematics of Control, Signals, Systems, 6, Doyle III, F. J., Packard, A. K., & Morari, M. (1989). Robust 99–105. controller design for a nonlinear CSTR. Chemical Engineering Nesterov, Y., & Nemirovskii, A. (1994). Interior point polynomial Science, 44, 1929–1947. algorithms in convex programming. In Studies in applied mathe- Fan, M. K. H., & Tits, A. L. (1992). A measure of worst-case H matics, vol. 13. Philadelphia, PA: SIAM. performance and of largest acceptable uncertainty. Systems and Newlin, M. P. (1996). Model 6alidation, computation, and control. Control Letters, 18, 409–421. PhD thesis, Caltech, Pasadena. Fan, M. K. H., Tits, A. L., & Doyle, J. C. (1991). Robustness in the Packard, A. K. (1988). What’s new with v: structured uncertainty in presence of mixed parametric uncertainty and unmodeled dynam- multi6ariable control. PhD thesis, University of California, ics. IEEE Transactions on Automatic Control, 36, 25–38. Berkeley. 1030 R.D. Braatz, E.L. Russell / Computers and Chemical Engineering 23 (1999) 1021–1030

Palmor, Z. J., & Shinnar, R. (1981). Design of advanced process Smith, R. S. R. (1990). Model 6alidation for uncertain systems. PhD controllers. AIChE Journal, 27, 793–805. thesis, California Institute of Technology, Pasadena. Papadimitriou, C. H., & Steiglitz, K. (1982). Combinatorial opti- Song, H. K., Fisher, D. G., & Shah, S. L. (1984). Experimental mization: algorithms and complexity. Englewood Cliffs, NJ: Pren- evaluation of a robust self-tuning PID controller. Canadian Jour- tice-Hall. nal of Chemical Engineering, 62, 755–763. Pardalos, P. M., & Vavasis, S. A. (1991). Quadratic programming Stryczek, K., & Brosilow, C. B. (1996). Mp tuning of multivariable with one negative eigenvalue is NP-hard. Journal of Global Opti- uncertain processes. In Proceedings of the fifth international con- mization, 1, 15–22. ference on chemical process control (CPC-V). Tahoe City, CA. Poljak, S., & Rohn, J. (1993). Checking robust non-singularity is Tardos, G. (1994). Multi-prover encoding schemes & three-prover NP-hard. Mathematics of Control, Signals, and Systems, 6, 1–9. proof systems. In Proceedings of the ninth annual structure in Rao, C. V., Campbell, J. C., Rawlings, J. B., & Wright, S. J. (1997). complexity theory conference (pp. 308–317). Amsterdam, The Efficient implementation of model predictive control for sheet Netherlands. and film forming processes. In Proceedings of the American con- Toker, O., & Ozbay, H. (1995). On the NP-hardness of the purely trol conference (pp. 2940–2944). Piscataway, NJ: IEEE Press. complex m computation, analysis/synthesis, and some related Reinelt, G. (1994). The traveling salesman problem: computational problems in multidimensional systems. In Proceedings of the solutions for TSP applications. In Lectures in computer science, American control conference (pp. 447–451). Piscataway, NJ: vol. 840. Berlin: Springer-Verlag. IEEE Press. Ricker, N. L., & Lee, J. H. (1995). Nonlinear model predictive VanAntwerp, J. G., & Braatz, R. D. (1998). Fast model predictive control of the Tennessee–Eastman challenge process. Computers control of sheet and film processes. IEEE Transactions on Con- and Chemical Engineering, 19, 961–981. trol Systems Technology, in press. Rigopoulos, A., Arkun, Y., & Kayihan, F. (1997). Identification of VanAntwerp, J. G., Braatz, R. D., & Sahinidis, N. V. (1998). full profile disturbance models for sheet forming processes. Globally optimal robust process control. Journal of Process Con- AIChE Journal, 43, 727–740. trol, in press. Russell, E. L., & Braatz, R. D. (1996). Analysis of large scale Vavasis, S. A. (1992). On approximation algorithms for concave systems with model uncertainty, actuator and state constraints, programming. In Recent ad6ances in global optimization (pp. and time delays. In AIChE Annual Meeting, Chicago, IL. Paper 3–18). Princeton, NJ: Princeton University Press. 45a. Vavasis, S. A. (1993). Polynomial time weak approximation al- Russell, E. L., & Braatz, R. D. (1998a). The average-case iden- gorithms for quadratic programming. In Complexity in numerical tifiability and controllability of large scale systems. Journal of optimization (pp. 490–500). Singapore: World Scientific. Process Control, in press. Yannakakis, M. (1994). Some open problems in approximation. In Russell, E. L., & Braatz, R. D. (1998b). Model reduction for the M. Bonuccelli, P. Crescenzi, & R. Petreschi, Algorithms and robustness margin computation of large scale uncertain systems. complexity. In: Lectures in computer science, vol. 778 (pp. 33– Computers and Chemical Engineering, 22, 913–926. 53). Berlin: Springer-Verlag. Russell, E. L., Power, C. P. H., & Braatz, R. D. (1997). Multidi- Young, P. M., & Doyle, J. C. (1990). Computation of the structured mensional realizations of large scale uncertain systems for multi- singular value with real and complex uncertainties. In Proceed- variable stability margin computation. International Journal of ings of the IEEE conference on decision and control (pp. 1230– Robust and Nonlinear Control, 7, 113–125. 1235). Piscataway, NJ: IEEE Press. Ryoo, H. S., & Sahinidis, N. V. (1996). A branch-and-reduce ap- Young, P. M., Newlin, M. P., & Doyle, J. C. (1991). v analysis with proach to global optimization. Journal of Global Optimization, 8, real parametric uncertainty. In Proceedings of the IEEE confer- 107–139. ence on decision and control (pp. 1251–1256). Piscataway, NJ: Safonov, M. G. (1982). Stability margins of diagonally perturbed IEEE Press. multivariable feedback systems. IEE Proc. Pt. D, 129, 251–256. Yousefpor, M., Palazoglu, A., & Hess, R. (1988). Robust control Sahni, S., & Gonzales, T. (1976). P-complete approximation prob- design for SISO systems based on constrained optimization. In- lems. Journal of the ACM, 23, 555–565. ternational Journal of Control, 62, 477. Schaper, C. D., Seborg, D. E., & Mellichamp, D. A. (1992). Proba- Zafiriou, E., & Marchal, A. L. (1991). Stability of SISO quadratic bilistic approach to robust process control. Industrial Engineering dynamic matrix control with hard output constraints. AIChE and Chemical Research, 31, 1694–1704. Journal, 37, 1550–1560. Skogestad, S., & Morari, M. (1988). Some new properties of the Zhou, K., Doyle, J. C., & Glover, K. (1995). Robust and optimal structured singular value. IEEE Transactions on Automatic Con- control. New Jersey: Prentice-Hall. trol, 33, 1151–1154. Zuckerman, D. (1993). NP-complete problems have a version that’s Skogestad, S., & Postlethwaite, I. (1996). Multi6ariable feedback hard to approximate. In Proceedings of the eight annual structure control: analysis and design. New York: Wiley. in complexity theory conference (pp. 305–312). San Diego, CA.