Information-Based Complexity of Uncertainty Sets in Feedback Control Le Yi Wang and Lin Lin, Member, IEEE

IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 46, NO. 4, APRIL 2001 519 Information-Based Complexity of Uncertainty Sets in Feedback Control Le Yi Wang and Lin Lin, Member, IEEE

Abstract—In this paper, a notion of information-based in this paper to characterize complexity of an uncertainty set can complexity is introduced to characterize complexities of plant be illustrated in the following example. uncertainty sets in feedback control settings, and to understand relationships between identification and feedback control in dealing with uncertainty. This new complexity measure extends A. Illustrative Examples the Kolmogorov entropy to problems involving information Example 1: A device’s dynamic behavior varies signifi- acquisition (identification) and processing (control), and provides cantly with the altitude where it is installed. The installer of a tangible measure of “difficulty” of an uncertainty set of plants. In the special cases of robust stabilization for systems with either the device is supposed to input the value , and then a pre- gain uncertainty or unstructured additive uncertainty, the com- designed controller suitable for that is automatically selected plexity measures are explicitly derived. for implementation. Suppose that the total range of altitude is Index Terms—Complexity, feedback, identification, informa- . is equipped with a metric, namely, the tion, metric robustness, uncertainty. absolute value . Assume that the best controllers can provide robust stability and performance only for a range of uncertainty on of length . is a metric measure of optimal robustness of I. INTRODUCTION controllers. The question is: How many robust controllers must N THIS PAPER, a notion of information-based complexity, be predesigned and stored in the system computer? Let be I termed as control complexity, is introduced to characterize the minimum number of such controllers. complexities of uncertainty sets in feedback control settings, 1) Case 1— can be Accurately Measured: In this case, the and to understand complexity relationships between identifica- output of measuring the altitude will belong to tion and feedback control. Compared to tremendous progress in design methodologies of robust and adaptive control, the central issue of complexity in control systems remains largely unexplored. Essentially, study It can be shown that the minimum number of controllers is of system complexity intends to answer the question: What is a “difficult” plant to model, to identify, and to control? Tradi- (1) tionally, a common perception of complexity of a finite-dimensional linear time invariant (LTI) system is the order or dimen- and one example of robust ranges of these controllers is sion in its representation such as transfer functions or state space models. This complexity is closely related to many numerical issues involved in representing, identifying, designing, and simu- (2) lating the system. Many aspects of model reduction have been where, by (1), . explored based on this understanding. 2) Case 2— can Only be Measured with an Accuracy of Several new developments since the 1970s have provided an , : In this case, the output of measuring the alternative paradigm of measures of plant complexity, which is altitude will belong to related to feedback mechanism in a more fundamental manner. These include optimal robustness in the theories of , , , gap, etc. from control points of view, and the Kolmogorov entropy and -width, Gelfand -width and identification -width for system modeling and identification. To implement a robust controller based on the measured ,itis In this paper, we will provide a new perspective in character- necessary that . Under this condition, we can show that izing plant complexity. The basic ideas and approaches we take the minimum number of controllers is

(3) Manuscript received February 27, 1998; revised July 30, 1999 and March 16, 2000. Recommended by Associate Editor G. Gu. L. Y. Wang is with the Department of Electrical and Computer En- and one example of robust ranges of these controllers is gineering, Wayne State University, Detroit, MI 48202 USA (e-mail: [email protected]). L. Lin is with iBiquity Digital, Inc., Warren, NJ 07059 USA. Publisher Item Identifier S 0018-9286(01)03454-7. (4)

Note that the overlapping of the robust ranges is necessary to to when crosses the point in . For instance, if cover, e.g., the possible measuring output the controllers in (2) of Case 1 were used, the switching points, , where is a very small number. which will be shown shortly to be undesirable, would be From (3), we can easily see that the better the metric robustness (larger ), the smaller the complexity (smaller ); the better the identification in reducing metric uncertainty (smaller ), the smaller the complexity. In this example, the control scheme involves a finite set of controllers selected from a space of admissible controllers . Let be the set of all such schemes. Since the individual For simplicity, assume that the plant and robust controllers are controllers can be designed off-line, one relevant measure of linear, and when is in the robust range of the implemented con- complexity of the scheme is the number of controllers in the troller, the resultant closed-loop system is exponentially stable. set, rather than the complexity of individual controllers. This It is well known that under this exponential stability, there exists leads to a complexity measure on . Let be the set a finite staying time such that if always stays in the robust of schemes in with complexity bounded by . For a given range of the controller at least seconds, then the total adap- uncertainty set of plants and a performance index tive system will remain stable. It can be easily verified that if and the variation rate satisfies then the minimum staying time will be guaranteed. Controllers in (2) have becomes a complexity measure of the uncertainty set . This is , and, hence, not suitable for adaptation here. in the sense that is considered as “difficult” if more complex Next, we modify (2) to increase the number of controllers controllers must be used. Viewed from the point of hybrid sys- from to to provide the robustness ranges of as follows: tems, the number of controllers is directly related to the size of discrete-event state space, while system orders are related to the size of analog state space. (5) In a sense, we are encountering adaptation in this example, at least a primitive one: selecting different controllers based and on the outcome of an identification process. Apparently, the complexity measure in this example does not characterize time- varying aspects of the systems involved. As a result, it cannot be used directly to characterize complexity of adaptive or switched (6) systems in which time-varying behavior is dominant. However, there are applications in which the size of discrete-event state Unlike the controllers in (2), the robustness ranges in (5) have space is a relevant complexity measure and reduction of it is de- overlaps. Now, the up-switching points become sirable. Also, for slowly time-varying systems, the complexity measure will become relevant. In fact, the effect of time variation on complexity resembles identification errors, and hence it and down-switching points are increases complexity, as shown in the following example from [17]. Example 2: Continuing Example 1, we now study gain scheduling when the altitude varies with time, slowly with rate : , for all and . One may The complexity is higher than . By substituting in simply imagine that the device is now mounted on a mobile (6), we have station and an altitude sensor is also installed. For simplicity, assume that the sensor can read altitude accurately. Adaptation is now done by the standard gain scheduling. Given controllers , a set of up-switching points Compared to (3), this expression tells us that time variation increases complexity; and in this special case, the effect is similar to identification uncertainty . In particular, and a set of down-switching points Hence, the complexity measure in (3) becomes a limiting complexity when variation rates approach zero. Further discussions and simulation results will be presented in Section VI. However, rigorous analysis and development of are established with complexity measures in time-varying systems remain an open research problem and are beyond the scope of this paper. With intuitive understanding from these examples, we now From the current controller , switching of controllers from proceed to a more rigorous description of the complexity mea- to occurs when crosses the point in , or from sure introduced in this paper. WANG AND LIN: INFORMATION-BASED COMPLEXITY OF UNCERTAINTY SETS IN FEEDBACK CONTROL 521

B. Metric Robustness ness ranges can collectively cover the uncertainty set. In Classical control developed in the framework of Bode, this case, the smallest covering of the uncertainty set by Nyquist, and Nichols aims at designing feedback controllers using -level balls, which is also the size of control ta- under which the closed-loop systems are insensitive to mod- bles, is a measure of complexity of the plant uncertainty eling errors and disturbances, namely robust in today’s terms. set, and is called control complexity of the uncertainty set. It was observed that some plants are intrinsically difficult to This complexity measure is closely related to the Kol- control in terms of this sensitivity analysis, no matter how mogorov entropy [8]. For a given set of functions and a much effort was made in tuning controllers. This observation set of modeling spaces in a metric space, the Kolmogorov led to an intuitive perception of the “complexity” of controlling -complexity of is defined as the minimum number of such plants. functions from that can approximate the set within The seminal work of George Zames and great research the error bound . The main differences between our com- progress on robust control since the 1980s have ushered in a plexity measure and the Kolmogorov complexity are: 1) in new era in our understanding and characterization of feedback Kolmogorov entropies balls of uniform norm radii are used robustness. Optimal robustness problems in metric spaces, to model an uncertainty set, whereas we are using balls of formulated by using metrics such as the norm, norm, uniform performance levels; and 2) we take into consideration gap metric, metric, etc., have provided perhaps the most of identification errors. relevant measures of “difficulties” of a plant to be controlled In closing, we would like to comment that complexity mea- robustly. For example, it is now clearly understood that plants sures have been explored extensively in the problems of com- with near unstable pole/zero cancellations are difficult plants, munications, system modeling and identification. This is exem- in the sense that they allow very small robustness radii (in the plified by system orders for numerical computations and con- selected metric space) no matter how controllers are selected. trol design; Kolmogorov -width [8], [11], Kolmogorov -en- In several important special cases, such as gain uncertainty, tropy [8], [21], and identification -width [23] in deterministic additive and multiplicative unstructured uncertainty, gap un- identification problems; Akaike’s information criterion [1] and certainty and coprime-factor uncertainty, optimal robustness Rissanen’s shortest description [13] in stochastic identification problems are solved and optimal controllers can now be either problems; and Shannon’s information in communication theory. constructed in closed forms or obtained numerically. These results provide metric measures of complexity of a plant on the D. Organization of the Paper one hand, and characterize the ability of feedback to achieve The rest of the paper is organized as follows. The main as- robustness on the other. Hence, from this point of view, a plant sumptions on systems and uncertainty sets are discussed in Sec- which allows only small optimal robustness is more complex tion II. The basic concepts of control complexity are introduced to control than those with larger optimal robustness radii. This in Section III. Some generic properties of these complexity mea- idea of characterizing plant complexity is particularly useful in sures are provided (Proposition 1). To illustrate the potential understanding the issues of modeling, uncertainty, and control utility and computation of these measures, the concepts are ap- from an information point of view: A plant of high complexity plied to robust stabilization problems for systems subject to gain requires more effort in acquiring information during modeling or unstructured additive uncertainties in Sections IV and V. For and identification before a robust controller can be successfully systems with gain uncertainties, the exact control complexity is applied. obtained (Theorem 1). For systems with unstructured additive uncertainties, their control complexities are related to the Kol- C. Basic Ideas mogorov -entropies for which many results are available for The main ideas we employ in this paper can be briefly sum- various classes of uncertainty sets (Theorem 2). Examples and marized as follows. Suppose that a performance index is given simulation are provided in SectionVI. Complexity relationships to describe the desired behavior of a feedback system. between identification and feedback are further discussed in SectionVII. Finally, some conclusions are drawn in Section VIII 1) Metric robustness, or equivalently robustness ranges, of for several open issues along the direction of this paper. controllers is expressed as balls in a metric space, such as the length in Example 1. Such a ball in the metric space is characterized by its center, norm radius, and, more im- E. Related Literature portantly, by the robust performance level (say, ) achiev- Complexity issues in modeling and identification have able by applying a robust feedback. Such a ball will be been pursued by many researchers. The concepts of -net and called a -level ball. -dimension in the Kolmogorov sense [8] were first employed 2) The ability of identification in reducing uncertainty is ex- by Zames [21], [22] in studies of model complexity and pressed in terms of metric errors in the same metric space, system identification. For certain classes of continuous- and such as in Example 1. discrete-time systems, the -widths and -dimensions in the 3) For a required performance specification given by the kernel norm and the norm were obtained by Zames, level , if the uncertainty set can be covered by a single Lin, and Wang [21], [18]. The notion of identification -width -level ball, then robust control is sufficient. For an un- was introduced in [23] to characterize intrinsic complexities certainty set which cannot be covered by a single -level in worst-case identification problems. Complexity issues in ball, a set of controllers will be used so that their robust- system identification were rigorously studied by Tse, Dahleh, 522 IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 46, NO. 4, APRIL 2001 and Tsitsiklis [4]. Poolla and Tikku [12]. -widths of many other classes of functions and operators were summarized in the books by Pinkus [11] and Vitushkin [16]. A general framework of information based complexity was comprehensively developed by Traub, Wasilkowski, and Wozniakowski [15]. The essential features and generic characterizations of topological approaches in robust and adaptive control are summarized and elaborated by Pait in [10]. In stochastic systems, complexity measures such as Shannon information and Kolmogorov probability entropies have been widely used in communication and control systems. For control applications, the reader is referred to the book by Saridis Fig. 1. Feedback system. [14] for an exposure on entropy approaches, and to the book by Caines [2] for an encyclopedical treatment of linear sto- II. UNCERTAINTY SETS AND FEEDBACK chastic systems including entropy methods. Recently, Xie and Guo [20] obtained a complete characterization of feedback ro- A. Systems and Feedback Configurations bustness against uncertainties on growth rates of nonlinear time- denotes a normed algebra of SISO causal stable linear time varying first-order systems. invariant systems with norm . Examples of include the The concept of control complexity introduced in this paper following typical algebras of stable systems. is an extension of the Kolmogorov -entropy [8], [21] which 1) (Discrete-time stable systems) : The Hardy space characterizes the complexity of a set of functions by the of analytic functions on the unit disk with minimal number of functions which can approximate the set . to a precision level . While the Kolmogorov entropy has 2) (Continuous-time stable systems) : The Hardy been extensively used in approximation theory, computational space of analytic functions on the right-half plane with complexity theory, information theory, and operator theory, . as well as system modeling and identification, it often fails 3) : Discrete-time systems with impulse responses to provide a relevant measure of complexity of uncertainty which satisfy . sets in feedback control problems. The main reason is that the 4) : Continuous-time systems with impulse responses ability of feedback to achieve robust stability and performance which satisfy . depends on the nominal system structure, not merely the size of uncertainty. The control complexities introduced in this paper Finite dimensional subspaces of the above systems are also such incorporate the ability of feedback in achieving design spec- examples. Since we are dealing with SISO LTI systems, we will ifications robustly. The Kolmogorov entropy was introduced use to denote both the system and its transfer function, and in the 1950s [8]. The book by Carl and Stephani [3] contains its impulse response. many new developments on the theory and applications of the Unstable systems will belong to , the extended space of Kolmogorov entropy up to the late 1980s. Some terms used in (see, e.g., [5] and [19] for detailed discussions on extended this paper, such as covering numbers, are borrowed from [3]. spaces). contains as a subspace. A more general framework of uncertainty, information Consider the feedback system in Fig. 1. , , , are ref- and complexity in information acquisition and processing erence input, disturbance, actuator signal, and plant output, re- is reported in [17], in which complexity problems beyond spectively. The interconnection of a feedback and a plant single-input–single-output (SISO) systems, metric spaces, or in is well posed if all elements of the closed-loop system feedback are studied. Notation: denote the real numbers, complex numbers, and integers. The subscript denotes restriction to nonnegative values, as in and . The absolute value of is . The spaces of vectors or matrices with elements in will (7) be denoted by or . For a matrix , is its transpose. Suppose is a normed space with norm . are in , and stable if they are in . Suppose that the set of denotes the ball of center and radius in .1 For admissible controllers is . is said to robustly , is the union of the subsets. For stabilize a subset of plants if is stable for all , is the diameter of . We denote by the set of all feedback controllers which can robustly stabilize . may be empty, implying that is not robustly stabilizable by using controllers in . In the special case where contains only one plant with a coprime factorization representation and , is given explicitly by the Youla parameterization of all stabilizing 1It will be specified whether the ball is open or closed, on the basis of specific applications. controllers for . WANG AND LIN: INFORMATION-BASED COMPLEXITY OF UNCERTAINTY SETS IN FEEDBACK CONTROL 523

B. Uncertainty Sets which assigns a finite nonnegative value to a plant- Uncertainty sets considered in this paper are limited to those controller pair if the feedback interconnec- parameterized by an uncertain variable taking values in a tion is stable, and otherwise. subset of a normed space . Formally, let be an We will use balls in the metric space to characterize feed- uncertainty set of plants represented by back robustness. For a given ball , the corresponding plant uncertainty set is . (8) denotes the set of controllers in which robustly stabilize . We define the optimal robust performance by where is a mapping relating the plant to its uncertainty variable . As a result, descriptions of a plant uncertainty set will include the structure information and the uncertainty set . The structure information is assumed to be available from the outset and will be carried intact in the definitions if is not empty of complexity measures. Examples of uncertain systems with such parameterization otherwise. include most cases commonly studied in the robust control lit- Namely, is the best achievable robust performance for erature. by using controllers in . By including the Example 3: performance level, the ball is now characterized by its center 1) Gain uncertainty: with norm and , , norm radius , and the performance level . . Or phase uncertainty: For a given performance level , the set of all balls in with with norm and , performance levels bounded by is called the set of -level balls , where is a nominal plant. 2) Parametric uncertainty: with norm (9) and is a compact set in One may interpret the set from different angles. First, for a given and a desired performance level , one can define

In other words, is the optimal robustness radius for the nominal plant . This is a metric characterization of the complexity of . In this characterization, a nominal plant 3) Additive unstructured uncertainty: is considered as “hard” or “difficult” if the corresponding , or multiplicative unstructured uncertainty: is small, regardless its order or internal representation. . Where is a Alternatively, one may interpret this as a metric character- nominal plant and a weighting function, ization of the ability of feedback in achieving robust stability may be , , , , or other metric spaces and performance. In this view, the information on feedback ro- of stable systems. bustness is generically represented by the ball .2 4) Linear fraction transformation (LFT) type: As a result, a controller is considered as optimal if it achieves , where this robustness. This characterization implies inevitably that the , are nominal closed-loop systems. information about feedback robustness outside the ball will be may be , , , , or other metric spaces lost. Retrieving this lost information requires ad hoc analysis of stable systems. of specific systems, and, hence, is not suitable for theoretical C. Metric Characterization of Feedback Robustness development. This is similar to the situation of using norms to characterize sizes of matrices by ignoring ad hoc details. Generally, the set of plants which can be stabilized by a given We shall comment in passing that metric characterization of controller is expressed in the Youla parameterization. feedback robustness is nothing new. This is exemplified by op- This is a complete characterization of feedback robustness. Such timal robustness measures in norm, norm, metric, gap characterization is, however, far too complicated for informa- metric, etc. Tremendous progress in the modern robust control tion processing. theory clearly indicates the advantages of such characterization. In this paper, we employ the following metric characteriza- Also, metric measures of robustness is structure sen- tion of feedback robustness. Suppose that a plant has the sitive since the optimal size of the ball depends on the structure with , where is equipped with plant structure . This is expected in feedback control norm , and controllers are selected from a set . The performance of a feedback system is measured by a certain per- 2Depending on q and , optimal robustness may or may not extend to the boundary of the ball. As a result, the balls may be open, closed or partially formance index closed. This will be specified in our examples. To avoid notational complica- tions, we will simply use the symbol fll@ Yr A in our general discussions without specifying whether it is open or closed. 524 IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 46, NO. 4, APRIL 2001 problems. For example, for a plant with the structure , , , the balls can be robustly stabilized for any center and finite . On the other hand, for , the ball Fig. 2. Control table for the example. can be robustly stabilized only if , where is a finite constant depending on and . uncertainty set . Denote by the class of all possible posterior uncertainty sets , III. CONTROL COMPLEXITY OF UNCERTAINTY SETS is a possible posterior uncertainty Consider an uncertainty set , de- set (10) fined in (8). As in other applications, modeling of uncertainty sets aims at representing the uncertainty set by simple base uncertainty sets. The main approach we employ in this paper is The variable parameterizes all possible outcomes of an identi- based on the following two principles: 1) The structural infor- fication experiment. Since every in is a possible true vari- mation is kept intact in the modeling process. In other words, able, covers , namely, instead of modeling , we are modeling . In Example 3, are stable and bounded, and takes the simple forms of intervals (11) or convex sets, whereas often contains unstable systems and is complicated. As a result, it is much easier to model than . 2) The modeling space is defined in (9). In approxima- We will denote the set of all possible posterior information on tion theory, one is seeking a finite set of functions which can the plant by approximate the uncertainty set to a precision level . In other words, the balls of norm radii are used as a modeling space. (12) This leads to the concept of Kolmogorov -entropy and dimension. By using the set as our modeling space, we represent the uncertainty set by balls of performance level . B. Complexity Definition: Suppose that for a given , is an indexed If the prior uncertainty set is not contained in the robust- class of sets in ness range of any single controller , we are seeking multiple controllers whose robustness ranges will collectively contain or cover . The main question is: How many controllers are needed? The situation becomes further compounded by the fact that identification is is said to be a -covering of if usually imperfect. Feedback control which follows identification is implemented based on the posterior uncertainty sets . We note that since all is a possible true variable, the robustness ranges of the set of controllers must be at least a -covering of . However, we will demonstrate by an example that this Interchangeably, we say that is modeled by . covering may not be sufficient to accommodate identification errors. A. Identification Example 4: Consider a plant with gain uncertainty Here, the term “identification” is used in the broad sense of , where is the LTI information acquisition by some unspecified means. Possible nominal plant. Suppose is not robustly stabilizable by a single examples of identification include direct measurement of pa- LTI controller, but for a covering rameters (possibly with sensor noises), estimation of parame- of , and are robustly stabiliz- ters from corrupted input–output data, data transfer through a able separately by controllers and . A control table which network with limited channel capacity, or parameters obtained provides such robustness ranges for is given by Fig. 2. from model reduction, etc. Rather than dealing with specific de- However, if an identification scheme can only locate to, say, tails of characterizing and computing identification errors, in an accuracy of 0.2, that is, the posterior information takes the this paper we focus on the generic impact of infidelity of in- form of formation acquisition on control complexity. The treatment reported in this paper is limited to metric information. For more general discussions, see [17]. Consider a plant that belongs to a given prior uncertainty set then will not be an appropriate covering. For instance, . Identification experiments pro- when the posterior uncertainty set produced by the identification vide additional information about the plant and reduce the un- scheme is , one cannot de- certainty on the plant uncertain variable from to a posterior termine a robust controller from the control table in Fig. 2 since WANG AND LIN: INFORMATION-BASED COMPLEXITY OF UNCERTAINTY SETS IN FEEDBACK CONTROL 525

is not contained in any robustness ranges of the controllers A sequence of posterior information of in the table. is said to be monotone and convergent to if In fact, for an appropriate selection of controllers based on posterior information , -level coverings must satisfy the additional inclusion property: for any uncertainty set , and there exists such that

where is the diameter of the set in .In this case, we will write .If is control compact, namely, there exists a finite -covering of it, then for any Intuitively, this means that for any possible outcome of the identification scheme, there should be a controller in the table which can achieve robust performance level for . This motivates the following notion of control complexity. First, we recall the mathematical notion of refiners from set we have theory. Suppose and are two classes of subsets of when

3) Feedback Invariance is said to be a refiner of , denoted by , if for every a) Suppose is a controller such that the plant- , there exists such that . controller pair is well posed for all Let and be defined as in (10) and (12). Let be an indexed set of balls in , where

(13) Let

is said to be a covering of if is a refiner of

and and a -covering of if, in addition, for all , . Now, we introduce the notion of control complexity. Then Definition 1: (14) 1) The control complexity of is the minimum size of all -coverings of . If there exists no -covering b) If, in addition, the identification scheme has the fol- of , then .3 is said to be control lowing feedback invariance property: compact if . 2) When the underlying identification is accurate, namely, is reduced to a singleton, and , we will use the symbol to denote the corre- i.e., identification experiments on and on pro- sponding control complexity. duce the same posterior information on , then The following properties of can be derived from their definitions. (15) Proposition 1: has the following properties. 1) Monotonicity 4) Subadditivity a) . b) For , . c) If , then . Remarks: d) Make explicit the dependence of the complexity on 1) The monotonicity states that control based on perfect the underlying set (for admissible controllers) identification schemes can always be less complex than by the notation .If , then that based on imperfect identification [see a)]. It claims . also that the tighter the performance specifications, the 2) Convergence more complex the control must be [see b)]; the better the 3We point out in passing that to be compatible to some traditional frameworks identification schemes, the less complex the control can of complexity, one may elect to use the control entropy log x @ Y&A. be [see c)]; the stronger the ability of feedback to achieve 526 IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 46, NO. 4, APRIL 2001

performance specifications, the less complex the control precision of , i.e., , then the posterior gain is [see d)]. uncertain is 2) Under the stated conditions, is the limiting complexity when identification becomes accurate. 3) The feedback invariance (14) indicates that applying a fixed feedback controller after identification but before It is noted that feedback control cannot change the complexity of the uncertainty set. whenever (18) 4) Examples of identification problems with the feedback invariant property include those from a fixed-length Namely, better identification will produce smaller posterior un- observation in identification -width problems [23]. For certainty sets in the sense of refiners. those problems with the feedback invariant property, (15) Theorem 1: claims that applying a fixed feedback before identification will not change the complexity of uncertainty sets. 1) If , then Control complexities will be computed for some stabilization problems in the following sections. (19) IV. CONTROL COMPLEXITY:STABILIZATION OF PLANTS WITH UNCERTAIN GAINS In robust stabilization problems, we may simply postulate the 2) If , then performance index as

if if . Proof: In the Appendix. Let It is noted from (19) that

(16) with a finite-dimensional nominal plant in . The following results are valid for both discrete-time systems ( ) and continuous-time systems ( ). In this special In other words, the complexity approaches the limit when the case of interval uncertainty, balls of variable radii are reduced identification error approaches zero. to intervals of variable lengths. For stabilization problems, the This example demonstrates clearly that the Kolmogorov en- coverings of induce a tropy is not appropriate for characterizing control complexity. covering of in the form of with4 Indeed, for the compact interval its Kolmogorov -complexity (or -entropy ) is finite and . In contrast

Khargonekar and Tannenbaum [7] obtained a necessary and sufficient condition for to be robustly stabilizable by a fixed controller in , namely as

(17) Hence, the norm compactness and control compactness are fun- damentally different. This example reflects one of the main mo- where is the optimal gain margin which depends only tivations for introducing a new complexity measure for control on the nominal plant . and the corresponding optimal or systems. suboptimal controllers and can be readily obtained by means of Nevanlinna–Pick interpolation [7]. V. C ONTROL COMPLEXITY:STABILIZATION OF PLANTS WITH If , then can be robustly stabilized by a fixed con- ADDITIVE UNSTRUCTURED UNCERTAINTIES troller and for any . Assume, therefore, . Concentrate now on the case of unstructured additive uncer- 5 Suppose an identification scheme is applied to identify the tainty plant in (16). If the identification can only locate the gain to a (20) 4The reason for using semi-open intervals, instead of open or closed intervals, is that the robust stabilizability of systems with gain uncertainty is expressed in 5It is easy to extend the discussions here to the cases a @hAYl Yv , terms of semi-open intervals. etc. WANG AND LIN: INFORMATION-BASED COMPLEXITY OF UNCERTAINTY SETS IN FEEDBACK CONTROL 527 where is a nominal plant which has a coprime factorization with and for some

is a fixed weighting function, and the uncertainty set will be specified later on. A covering of by open balls induces a covering of by .6 Lemma 1 [6]: is robustly stabilizable if and only if

(21) Fig. 3. Switching logic for Example 5. where VI. ILLUSTRATIVE EXAMPLES In this section, we will illustrate the concepts and results of the previous sections by several numerical examples. In these It is interesting to observe that the condition (21) is independent examples, the resulting control tables will have the minimal of the uncertainty center . We are now ready to sizes (when balls are used to model the uncertainty sets). establish a relationship between the control complexity and the Example 5: Let with Kolmogorov entropy. Let be the Kolmogorov -entropy of , i.e., the minimum cardinality of coverings of by open balls of uniform radius [21], [8]. Formally

where , . By [7], and is the optimal achievable gain margin. Since , we cannot robustly stabilize the uncertainty set by a single LTI controller. for some Note that if can be accurately measured, then the control complexity of is

Suppose that an identification scheme is applied to the plant in (20). If the scheme can only identify the plant to a precision of in the norm, then posterior information on is given by Hence, we will need at least three controllers to cover the uncertainty range . Controllers that achieve optimal gain margins can be shown to take the form , We would like to establish an estimate of . . When is very close to two, a gain margin close Theorem 2: Suppose and is defined as in Lemma to is achieved. We will select the following three con- 1. For any trollers: 1) if , then

(22)

2) If , then with respective robustness ranges , , . Collectively, . For a demonstration of deriving a potential gain-scheduling Proof: In Appendix. scheme from the table, we now allow to be time varying, albeit It is known that for norm compact , slowly. When is in the robustness range of the controller, as . In this sense, Theorem 2 gives an asymp- the closed-loop system is exponentially stable with decaying totically accurate estimate of when is small. rates in as , , . 6The reason of using open balls here is that metric robustness in this case does This, together with the overlapping of the robustness ranges, not include the boundary. implies that these controllers can be used in gain scheduling 528 IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 46, NO. 4, APRIL 2001

Fig. 4. Simulation for systems with time varying gains. control when varies sufficiently slowly. The gain-scheduling The plant initial value , and the time-varying gain or switching logic is shown in Fig. 3. Fig. 4 demonstrates simulation results in which plant and controllers are implemented as follows. The plant is For and sampling interval second, the trajec- tories of control signals, plant output, and controller switching are shown. It should be emphasized that if varies faster, it may become necessary to increase the number of controllers, indicating that where denotes the application of an operator uncertain time variation will affect complexity of an uncertainty to the signal , whereas the left multiplication by is the set. Beyond a certain rate of variation, the number of controllers usual multiplication in the time-domain. is a random noise is no longer relevant in analyzing the behavior of the controlled uniformly distributed in . The controllers are defined system. In this case, one is forced to deal with the analysis of as follows: for a switched system. See, e.g., [9] for many challenging issues in switching control problems. Example 6: For a plant with additive uncertainty WANG AND LIN: INFORMATION-BASED COMPLEXITY OF UNCERTAINTY SETS IN FEEDBACK CONTROL 529 we can compute that , and Correspondingly, we can construct the two robust controllers as

For an uncertainty set , the controller Arguments similar to those in Example 5 can be made in devel- oping a gain-scheduled scheme by switching between and when and vary with time sufficiently slowly. When is the only time-varying component, a switching logic is given in Fig. 5. Fig. 6 demonstrates simulation results that show the trajecto- ries of control signals, plant output, and controller switching.

VII. RELATIONS BETWEEN IDENTIFICATION AND CONTROL COMPLEXITIES can robustly stabilize it. In particular, for The relationship between fidelity of information acquisition (identification) and complexity of information processing (control) established in the previous sections can be employed to establish a complexity relationship between identification and control. Now, suppose that contains both parametric and unstruc- The control complexity expresses the complexity of uncer- tured uncertainties tainty sets as a function of identification errors. The identification errors are, in turn, functions of identification complexity. As a result, metric complexities of identification and control are related. The better the identification is, the less complex the con- where is the ball of center 0 and radius 0.1 in trol can be, or equivalently, the more complex control is, the less . Suppose that is identified with error . Then, complex the identification needs to be. Let us illustrate this idea the output of identification will be the uncertainty set with the following examples. . Hence A. Effects of Identification Errors on Control Consider an uncertain discrete-time system with prior information

Since , we will try to cover by balls of radius . Since the diameter of , we cannot cover by a single ball of radius 0.333. Choose Suppose the control complexity is constrained by , . We will show that

(23) due to, say, communication channel capacity, computer time or Indeed, if , then space limitations. We would like to know how accurate identi- , with .If , then fication must be. Suppose is computed as in Theorem 1. By Theorem 1, the identification error must satisfy

Hence, .If , then

i.e.,

(24) Hence, . Since is arbitrary in , (23) is valid. where by Theorem 1. 530 IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 46, NO. 4, APRIL 2001

The worst-case error occurs when all measurements give the same center, and in this case the optimal worst-case identification error can be obtained

Therefore, the inequality (24) is satisfied if and only if

B. Identification -Width and Control Fig. 5. Switching logic for Example 6. Suppose a priori information of an uncertain discrete-time plant is given by a general , , and is Suppose certain identification experiments are applied to the the identification -width of introduced in [23], i.e., the system with additive noise at the times of ob- best achievable worst-case identification error based on servation consecutive observations. For a given level of performance specification and identification error , is the corresponding control complexity. Note that is a monotone decreasing function of (the length of observation), and a monotone decreasing function of .Now, the function where the probing input is bounded by , the noise is bounded by , and is the impulse response of the nominal discrete-time plant. Define which is monotone increasing is a monotone decreasing function of . For a given identifica- and unbounded since is unstable. We claim that the in- tion time complexity , gives the minimum complexity equality (24) can be satisfied if and only if of control required to achieve the specified performance level . Then, the inverse function of

(25) gives the minimal time complexity of identification which is needed to achieve the performance specifications when the com- The inequality (25) elaborates a relationship among identifica- plexity of control is limited by . tion observation length , signal-to-noise ratio (SNR) , ro- One possible application of such tradeoff between complexi- bust stabilizability of the system , and complexity of control ties of identification and control might be allocation of resources . to identification and control. Suppose and are cost To show (25), observe that consecutive input–output mea- functions of implementing identification experiments and con- surements are related by trol mechanism, respectively. Then, optimization of the com- bined cost function

will provide a candidate for optimal allocation of resources to achieve a given level of performance specifications. This implies that VIII. CONCLUDING REMARKS Despite tremendous progress in control design methodologies, complexity issues in feedback control remain largely unexplored. While intuitive perceptions of complex plants and controllers have been employed in practical systems, rigorous formulation and investigation of complexity prob- The identification error at each time instant is lems in an information-based framework turn out to be an extremely challenging task. One reason for this difficulty is that it requires a clarifying and rigorous characterization of the ability of feedback mechanism in achieving robust performance. Recent advance in metric characterization of feedback robustness provides first time a solid foundation WANG AND LIN: INFORMATION-BASED COMPLEXITY OF UNCERTAINTY SETS IN FEEDBACK CONTROL 531

Fig. 6. Simulation for systems with time varying additive uncertainty. on which complexity issues may be analyzed rigorously in APPENDIX an information framework. This paper is a preliminary effort in this pursuit. Proof of Theorem 1 The main approach employed in this paper is an extension 1) We will prove (19) by showing the optimality of the cov- of the classical mathematical notion of Kolmogorov entropy, ering by accommodating the inherent feedback robustness in the problem formulation. The key ideas employed in this paper follow the fundamental philosophy of George Zames in devel- oping a comprehensive information-based theory of feedback, for , where , ; and , identification, and adaptation. , , and There remain many open issues. For instance, it is intuitively apparent that for time-varying plants, their variation rates have significant bearing on identification and control complexities. At this moment, it is not clear how such intuition can be clar- ified into a rigorous argument. Finally, the information-based framework discussed in this paper is naturally related to communication problems. Control-communication interaction in this Since , , are robustly stabilizable. framework seems to be an extremely interesting area for fur- Moreover, for any , select ther investigation. . 532 IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 46, NO. 4, APRIL 2001

Then, we have . Therefore, .It which implies follows that

To prove the lower bound, we observe that by Lemma 1, for (26) , is a -covering of if and only if , , and covers . Or equivalently, is a -covering of in The optimality of is shown by contradiction. Suppose the sense of Kolmogorov. Therefore, we conclude, after taking is another -covering of minimum of cardinalities of all such coverings, with . Assume, without loss of generality, that . The lower bound follows since . . Since is a -covering of , and . 2) If , then by Lemma 1, is Also, . Consequently, for all not robustly stabilizable. Therefore, . , which implies ACKNOWLEDGMENT The basic ideas of complexity utilized in this paper originated Now, induction leads to in G. Zames’ persistent pursuit of an information-based theory of feedback systems. The development of this paper was greatly influenced by his philosophy, suggestions, and comments. The authors would like to dedicate this paper to the memory of G. Zames. Since , i.e., The authors would also like to express their sincere appre- ciation for the insightful comments, constructive suggestions, frank criticisms, and valuable information on related literature we have from the anonymous reviewers, as well as M. Y. Fu and G. X. Gu. They have played a significant role in helping us improve the technical quality and readability of this paper. i.e., , a contradiction. Therefore, . REFERENCES 2) Equality (19) implies that for and [1] H. Akaike, “A new look at the statistical model identification,” IEEE Trans. Automat. Contr., vol. AC-19, pp. 716–723, Apr. 1974. [2] P. E. Caines, Linear Stochastic Systems. New York: Wiley, 1988. [3] B. Carl and I. Stephani, Entropy, Compactness and the Approximation of Operators. Cambridge, U.K.: Cambridge Univ. Press, 1990. This, together with (18) and the monotonicity of , [4] M. A. Dahleh, T. Theodosopoulos, and J. N. Tsitsiklis, “The sample implies complexity of worst-case identification of FIR linear systems,” Sys. Control Lett., vol. 20, no. 3, 1993. [5] C. A. Desoer and M. Vidyasagar, Feedback Systems: Input–Output Prop- erties. New York: Academic, 1975. [6] J. C. Doyle, B. A. Francis, and A. Tannenbaum, Feedback Control for . Theory. New York: Macmillan, 1992. [7] P. P. Khargonekar and A. Tannenbaum, “Non-Euclidian metrics and the Proof of Theorem 2 robust stabilization of systems with parameter uncertainty,” IEEE Trans. Automat. Contr., vol. AC-30, pp. 1005–1013, June 1985. 1) Suppose , [8] A. N. Kolmogorov, “On some asymptotic characteristics of completely is an optimal -covering of in the bounded spaces,” Dokl. Akad. Nauk SSSR, vol. 108, no. 3, pp. 385–389, 1956. Kolmogorov sense, i.e., [9] D. Liberzon and A. S. Morse, “Basic problems in stability and design of switched systems,” IEEE Contr. Syst. Mag., vol. 19, no. 5, pp. 59–70, 1999. and [10] F. M. Pait, “On the topologies of spaces of linear dynamical systems commonly employed as models for adaptive and robust control design,” in Proc. 3rd SIAM Conf. Control Intelligent Transportation Systems Then for any , there exists Appl., 1995. such that [11] A. Pinkus, n-Widths in Approximation Theory. New York: Springer- Verlag, 1985. [12] K. Poolla and A. Tikku, “On the time complexity of worst-case system identification,” IEEE Trans. Automat. Contr., vol. 39, pp. 944–950, May which implies that 1994. [13] J. Rissanen, Estimation of structure by minimum description length, (27) Workshop Ration, in Approx. Syst., Catholic Univ., Louvain, France. [14] G. Saridis, Stochastic Processes, Estimation and Control: The Entropy Now, define .By Approach. New York: Wiley, 1995. [15] J. F. Traub, G. W. Wasilkowski, and H. Wozniakowski, Information- (27) Based Complexity. New York, NY: Academic, 1988. [16] A. G. Vitushkin, Theory of the Transmission and Processing of Informa- tion: Pergamon Press, 1961. WANG AND LIN: INFORMATION-BASED COMPLEXITY OF UNCERTAINTY SETS IN FEEDBACK CONTROL 533

[17] L. Y. Wang, “Uncertainty, information and complexity in identification Lin Lin (S’91–M’94) received the B.E. and M.E. degrees from Tsinghua Uni- and control,” Int. J. Robust Nonlinear Control, vol. 10, pp. 857–874, versity, Beijing, China, and the Ph.D. degree from McGill University, Montreal, 2000. QC, Canada, in 1984, 1987, and 1993, respectively, all in electrical engineering. [18] L. Y.Wang and L. Lin, “On metric complexity of discrete-time systems,” From 1985 to 1987, he participated in a research project on power system sta- Systems Contr. Lett., vol. 19, pp. 287–291, 1992. bilization at Je Jiang Hydro, China. From 1993 to 1998, he worked on several [19] J. C. Willems, The Analysis of Feedback Systems. Cambridge: MIT signal processing and control projects in Canada, including active noise cancel- Press, 1971. lation at Nortel, Montreal, QC, Canada, power system control and simulation at [20] L. L. Xie and L. Guo, “How much uncertainty can be dealt with by CAE Electronics, Montreal, QC, Canada, linear transmitter design at Glenayre, feedback?,” IEEE Trans. Automat. Contr., vol. 45, pp. 2203–2217, Dec. and control-loop monitoring and diagnosis at Paprican. From 1998 to 2000, he 2000. was an Engineering Manager at Ericsson (MPD Technologies), Hauppauge, NY, [21] G. Zames, “On the metric complexity of causal linear systems: 4-entropy responsible for the development of ultra-linear wideband RF power amplifiers. and 4-dimension for continuous time,” IEEE Trans. Automat. Contr., He recently joined iBiquity Digital (Lucent Digital Radio), Warren, NJ, to de- vol. AC-24, pp. 222–230, Feb. 1979. velop in-band-on-channel digital radio technologies. Since 1994, he has been [22] , “Information-based theory of identification and adaptation, Ple- an Adjunct Professor in the Department of Electrical Engineering, McGill Uni- nary Speech,” in 1996 Decision Control Conf., Kobe, Japan, December versity. His research interests are in the areas of wireless communications, DSP 1996. technologies, complexity theory, system identification, and adaptive system de- [23] G. Zames, L. Lin, and L. Y. Wang, “Fast identification n-widths and sign. uncertainty principles for LTI and slowly varying systems,” IEEE Trans. Automat. Contr., vol. 39, pp. 1827–1838, Oct. 1994.

Le Yi Wang received the M.E. degree in computer control from the Shanghai Institute of Mechanical Engineering, Shanghai, China, and the Ph.D. degree in electrical engineering from McGill Univer- sity, Montreal, QC, Canada, in 1982 and 1990, respectviely. Since 1990, he has been with Wayne State Univer- sity, Detroit, MI, where he is currently an Associate Professor in the Department of Electrical and Com- puter Engineering. His research interests are in the areas of H-infinity optimization, complexity and information, robust control, time-varying systems, system identification, adaptive systems, hybrid and nonlinear systems, as well as automotive and medical applications of control methodologies. He is an Editor of the Journal of System Sciences and Complexity. Dr. Wang was awarded the Research Initiation Award in 1992 from the Na- tional Science Foundation. He also received the Faculty Research Award from Wayne State University in 1992, and the College Outstanding Teaching Award from the College of Engineering, Wayne State University in 1995. He is an As- sociate Editor of the IEEE TRANSACTIONS ON AUTOMATIC CONTROL.