ISSUES on ROBUST ADAPTIVE FEEDBACK CONTROL Michael

ISSUES ON ROBUST ADAPTIVE FEEDBACK CONTROL

Michael Athans ∗,1 Sajjad Fekri ∗,2 Antonio Pascoal ∗

∗ Institute for Systems and Robotics, Instituto Superior T´ecnico, Lisbon, Portugal email: athans,sfekri,[email protected] { Tel: (+351) 218 418 054 }

Abstract: We discuss recent progress in the ﬁeld of robust adaptive control with special emphasis on methodologies that use multiple-model architectures. We emphasize that the selection of the number of models, estimators and compensators in such architectures must be based on precise deﬁnition of the robust performance requirements. We illustrate some of the concepts and outstanding issues for a new methodology that blends robust nonadaptive mixed µ-synthesis and stochastic hypothesis-testing concepts leading to the so-called RMMAC architecture. A numerical example is used to illustrate the RMMAC design methodology. Copyright c 2005 IFAC ° Keywords: multiple model adaptive control, adaptive control, robust control, robust adaptive control, robust µ-synthesis

1. INTRODUCTION available results seem to be very promising, but still require a great deal of theoretical and prag- The solution to the so-called “adaptive control” matic research to arrive at the “Holy Grail” of problem is akin to the elusive search for the “Holy adaptive control. Thus, the solution to the adap- Grail” in the context of feedback control system tive control problem is still not available. design. In spite of forty years of research, several books and hundreds of articles we still lack, in our We first discuss a general philosophy for designing view, a universally accepted design methodology “robust” adaptive multivariable feedback control for adaptive control which is based on sound the- systems for linear time-invariant (LTI) plants that oretical issues and suitable for engineering imple- include both unmodeled dynamics and uncertain mentations in real-life control systems. In this pa- real parameters in the plant state-space descrip- per we overview some recent progress in adaptive tion. The adjective “adaptive” refers to the fact designs that employ multiple-models. Currently that the real parameter uncertainty and performance requirements require the implementation of a feedback architecture with greater complex- 1 M. Athans is also Professor (emeritus), Dept. of EECS, ity than that of the best possible non-adaptive MIT, Cambridge, MA, USA. controller. The word “robust” refers to the desire 2 This work was supported in part by the Por- tuguese FCT POSI programme under framework QCA that the adaptive control system remains stable III. The second author benefited from PhD grant and also meets the posed performance specifica- Ref. SFRH/BD/6199/2001 of the FCT (Portuguese Foun- dation for Science and Technology). tions for all-possible “legal” parameter values and robust synthesis can not deal with slowly-varying unmodeled dynamics. parameters, from a theoretical perspective. In order to place our remarks in a proper perspec- It is highly desirable that the above attributes tive we pose a set of basic engineering questions of nonadaptive robust feedback systems be also that naturally arise when we deal with adaptive reflected in the design of robust adaptive con- control: trollers as well. Thus, in our view, an adaptive control design must explicitly yield stability- and (1). What do we gain by using adaptive control? performance-robustness guarantees, not just sta- (2). How do we fairly predict and compare per- bility (which has been the central focus of almost formance improvements (if any) of a proposed all adaptive control methodologies). adaptive design vis-à-vis the “best nonadaptive” one? 1.1 Brief Historical Perspective (3). How do we design adaptive controllers with guaranteed robust-stability and robust-performance Early approaches to adaptive control, such as the in the presence of unmodeled dynamics and un- model-reference adaptive control (MRAC) method measurable plant disturbances and sensor noises? and its variants, were concerned with real-time (4). Is the increased complexity of an adaptive parameter identification and simultaneous adjust- controller justified by the performance improvement of the loop-gain. Representative references ment? What should be the level of complexity for are (Landau, 1979; Narendra and Annaswamy, ˚ performance guarantees? 1988; Sastry and Bodson, 1989; Aström and Wit- tenmark, 1995; Ioannou and Sun, 1996). In the Even though there is no precise universally ac- MRAC method the emphasis was on proving cepted definition of “adaptive control” the above global convergence to the uncertain real param- questions are (or should be) at the heart of adap- eter and while using deterministic Lyapunov (hy- tive control research; they have motivated the perstability) arguments for inferring closed-loop discussion perspective adopted in this paper. stability. However, the assumptions required for It is important to stress at this point that the stability and convergence did not include the vast majority of approaches to adaptive control presence of unmodeled dynamics, unmeasurable deal with the case of constant uncertain uncertain disturbances and sensor noise. Moreover, no ex- real parameters. However, from an engineering plicit and quantifiable performance requirement perspective, the true value of an adaptive system was posed for the adaptive system; rather the can only be judged by its performance when the “goodness” of the MRAC design was judged uncertain real parameters change “slowly with by the nature of the command-following error time”, within predefined limits. based upon simulations. It turned out that classical MRAC systems can become unstable in the Thus, one designs and tests an adaptive system presence of plant disturbances, sensor noise and for constant real parameters, using whatever the- high-frequency unmodeled dynamics (Rohrs et al., oretical approaches developed, but it should be 1985). Moreover the MRAC methodology was lim- also tested for time-varying parameters as well. ited to single-input single-output (SISO) plants; The current accepted concept of a robust (non- attempts to extend the MRAC methodology to adaptive) feedback control system, for linear time- the multi-input multi-output (MIMO) case were invariant (LTI) plants, is that the designed com- extremely cumbersome. Because of these short- pensator must be such so as to guarantee (if pos- comings, we shall not further address the MRAC sible) closed-loop stability and to also meet posed methodology in the sequel. performance specifications, most often reflecting Later, and the more recent, approaches to the superior disturbance-rejection. This attribute is adaptive problem involved multiple-model tech- often referred to as “stability- and performance- niques which, in principle, are applicable to the robustness.” The physical plant is assumed to MIMO case. The (large) real-parameter uncer- belong to a “legal family” of possible plants, tainty set was subdivided into smaller parame- where the nominal plant together with frequency- ter subsets; each parameter subset gives rise to dependent upper bounds on unmodeled dynam- a different “plant model set” with reduced real- ics and upper- and lower-bounds on key uncer- parameter uncertainty. One then designed a set tain real parameters defines this “legal family” of of control gains or dynamic compensators for each plants. The performance specifications are explic- model set so that, if indeed the true parameter was itly stated in the frequency domain; they typically “close” to a specific model, then a “satisfactory” require superior disturbance-rejection in the lower performance was obtained. frequency region while safeguarding for excessive control action at higher frequencies. At present, In one of the multiple-model approaches, the identification of the most likely model is carried out by a “supervisor” which switches different con- performance in the presence of unmodeled dynam- trollers, based primarily on deterministic concepts ics and parametric uncertainty as well as unmea- (Morse, 1996, 1997, 1998, 2004; Narendra and surable plant disturbances and sensor noise. This Balakrishnan, 1997; Anderson et al., 2000, 2001; methodology, pioneered by Doyle et al., is often Hespanha et al., 2001). These proofs and results called the mixed-µ design method. The mixed- were presented for the case of SISO systems. The µ design method incorporates the state-of-the- second approach relied upon stochastic designs art in non-adaptive multivariable robust control that generated on-line posterior probabilities re- synthesis and exploits the proper use of frequency- flecting which of the models was more likely. In the domain weights to quantify desired performance. latter approach the controllers could be designed Typically, using the mixed-µ design method, one either by classical LQG methods (Willner, 1973; finds that as the size of the parametric uncer- Greene, 1975; Athans et al., 1977; Shomber, 1980; tainty is reduced the guaranteed desired perfor- Greene and Willsky, 1980; Schiller and Maybeck, mance, say disturbance-rejection, increases. Un- 1997; Maybeck and Griffin, 1997; Schott and Be- fortunately, very little has been done in integrat- quette, 1997) or by more sophisticated methods ing the non-adaptive mixed-µ design methodology (Fekri et al., 2004a,b,c; Fekri, 2005). In the latter with that of robust adaptive control studies; even approach one can deal with MIMO designs. A though it should be apparent that the mixed- different, more direct approach, called unfalsified µ design method should provide us guidance on control, also merits mention (Safonov and Tsao, the selection and number, N, of the models to 1995, 1997; Jun and Safonov, 1999); although we be used in any multiple-model adaptive control shall not discuss it in this paper. We also do not scheme. Notable exceptions are (Kosut and An- discuss numerous approaches to adaptive control derson, 1988; Anderson et al., 2000; Fekri et al., utilizing “intelligent” methods, such as neuro- 2004a,b,c; Fekri, 2005). fuzzy designs, since they are void of any analytical We now summarize our design philosophy regard- insights. ing adaptive control designs that employ multiple In all the proposed multiple-model adaptive meth- models. We assume that: ods the complexity of the adaptive feedback sys- (1). Independent of the size of uncertainty for the tem directly depends on the number of models plant real parameter(s), the plant always contains employed, N. By decreasing the size of the para- unmodeled dynamics whose size can be bounded metric subsets one obtains more models. Thus, a priori only in the frequency domain. Therefore, all multiple model approaches must address the the adaptive design must explicitly reflect these following: frequency-domain bounds upon the unmodeled (a) how to divide the initial large parameter dynamics. The presence of unmodeled dynamics uncertain set into N smaller parameter subsets, immediately brings into sharp focus the fact that we must use the state-of-the-art in nonadaptive (b) how to determine the “size” or “boundary” of robust control synthesis, i.e. mixed-µ synthesis each parameter subset, and (Young et al., 1992, 1995; Young, 1994; Zhou et (c) how large should N be? Presumably the al., 1996; Zhou, 1998; Balas, 2003; Balas et al., “larger” the N, the “better” the performance of 2004); and associated Matlab software (Balas, the adaptive system is. 2003; Balas et al., 2004). (2). The plant is subject to unmeasurable plant disturbances whose impact upon the chosen per- 1.2 Our Design Philosophy formance variables (error signals) must be minimized, i.e. we must have superior “disturbance- In this paper we shall focus upon “robust per- rejection”. The modern trend is to use frequency- formance” requirements on the adaptive system depended weights to emphasise and define supe- implemented by one of the available multiple- rior disturbance-rejection performance. This de- model methods. We follow the recent research on sign objective can also be accommodated by the “Robust Multiple-Model Adaptive Control (RM- mixed-µ design methodology. MAC),” (Fekri et al., 2004a,b,c; Fekri, 2005), (3). The plant measurements are not perfect; thus which will be discussed in more detail in the sensor measurements are corrupted by unmeasur- sequel. able sensor noise. The performance variables must If we turn our attention to the non-adaptive be “insensitive”, to the degree possible, to such literature there exists a well-documented de- sensor noise. sign methodology, and associated Matlab design (4). Performance requirements must be explicitly software, for linear time-invariant multivariable defined, up to constants whose values can be op- plants (both SISO and MIMO) that addresses timized for superior performance. In the mixed-µ simultaneously both robust-stability and robust- design methodology these “disturbance-rejection” that the adaptive system yield a performance that performance requirements are explicitly quanti- equals a certain multiple, say 10, of the (lower- fied by frequency-domain weights typically involv- bound) GNARC performance, if possible (there ing the selected “error signals” and the control may be inherent limitations due to non-minimum variables. phase zeros and/or unstable poles) and consistent with the FNARC upper-bound. Yet another ap- (5). Given the information in (1) to (4), we can de- proach is to fix the number N of the models, sign the best “global non-adaptive robust compen- i.e. specify the complexity of a multiple-model sator (GNARC)” for the entire (large) uncertain scheme, and maximize the performance for each real-parameter set and taking into account both model. Many other approaches are also possible the unmodeled dynamics and the performance which use both the GNARC lower-bound and the requirements. This non-adaptive feedback design FNARC upper-bound upon performance. must be optimized so as to yield the best possible performance, i.e. superior disturbance-rejection By following the above performance-driven metho- with reasonable control effort. The GNARC then dology one directly arrives at the number, N, of provides a yardstick (lower-bound) for perfor- required models in the adaptive multiple-model mance, so that any performance improvements by system, as well as the quantification of each more complex adaptive designs can be quantified. model. In general, the more stringent the perfor- It is self-evident that the GNARC must be de- mance requirements on any adaptive implemen- signed using the mixed-µ synthesis methodology. tation – consistent, of course, with the FNARC upper-bound – the larger the number of models (6). An upper-bound for adaptive performance can and the greater the complexity of the multiple- be obtained by optimizing the performance un- model adaptive system. We stress that such a der the assumption that the real-parameter val- systematic definition of the required models and ues are known exactly, but still reflecting the numerical specification would not be possible if presence of complex-valued unmodeled dynam- we did not explicitly pose the performance speci- ics and frequency-dependent performance require- fications and optimized performance to the extent ments. This implies that we compute for a large possible. number of grid points in the original parameter uncertainty set a “fixed non-adaptive robust The procedure summarized above can be used compensator (FNARC)” which defines the best with any of the adaptive multiple-model meth- possible performance for each parameter value. ods. We shall illustrate its detailed design and The FNARC design is carried out using the properties by using the multiple-model method complex-µ synthesis methodology, since we still in the context of dynamic hypothesis-testing, must take into account the unmodeled dynamics which involves generating the posterior probabil- and frequency-dependent performance specifica- ity for each model, the so-called Robust Multiple- tions. We use the same quantitative performance Model Adaptive Control (RMMAC) architecture. requirements as in part (5) above. The set of However, it can also be used in conjunction the FNARCs corresponds to having an infinite with the “switching” supervisory controllers de- number of models in the multiple-model imple- veloped by Morse, Anderson, Hespanha and oth- mentation. ers. The important point to remember is that all multiple-model adaptive schemes require the The difference between the lower-bound on per- definition of the minimum number of models formance from the GNARC in part (5) and the required to achieve both robust-stability and FNARC upper-bound from part (6) provides a robust-performance, and these can only be defined valuable quantitative decision aid to the designer after we pose realistic performance requirements on what performance improvements are possible for the adaptive system as discussed above. by some multiple-model adaptive control method. The designer must then make a quantifiable choice In Section 2 we present an overview of four dif- on the degree of performance improvement that ferent multiple-model architectures for adaptive he/she desires from the adaptive system. As we estimation, identification and control. In Section shall show in the sequel, this approach will then 3 we discuss the designs of the robust dynamic define the number of models (parameter subsets) compensators, such as the GNARC and FNARC required, N, and their numerical specification discussed above, and how to determine the num- (boundary of parameter subsets) in a natural ber N of models in the multiple-model architec- manner. tures, as well as the collection of the compensators. In Section 4 we focus upon the “identi- One possible approach is that the designer de- fication” aspect of the RMMAC. In Section 5 we mands that the adaptive performance equals or present numerical results, using the RMMAC, for exceeds a certain percentage, say 75%, of the (best a non-trivial dynamic control problem. Section possible) FNARC performance for each parameter 6 discusses the commonalties and differences of value. Another possible approach is to demand

& © ©

¡ ¢ ¡ ¢ ¢

the supervisory switching multiple model adaptive ξ (t) θ (t)

£ ¢ ¡ £ ¢ ¡ control (SMMAC) and RMMAC architectures and u( t ) y( t )

we summarize our conclusions in Section 7. ¢

xˆ1( t ) % 2. MULTIPLE-MODEL ARCHITECTURES IN r1 t)(

ADAPTIVE CONTROL xˆ2( t ) xˆ( t )

$ r2 t)( In this section we shall discuss the architectures

xˆN ( t )

that utilize multiple-models. We follow a historical rN t)(

development that traces the concepts over the

¢ ¡ £

S1 P1 ( t )

¢ ¡ £

¡ ¢£ ¤¥ ¦ §

¦ £ § £

past 40 years or so. First we shall discuss the S P( t )

¡ ¢ ¡ ¢

2 2

¨ ©

¦ £ ¦ ¡ ¢

! ¦ § ¥ ¦

¦ £ § £ ¡ ¢

architecture associated with multiple-model adap- " S ! # P( t ) tive estimation (MMAE). Second, we discuss the N N early extension of the MMAE concepts to classical multiple-model adaptive control (CMMAC). Fig. 1. The MMAE architecture for adaptive esti- Third, we present architectures associated with mation. SMMAC. Finally, we discuss the architecture associated with robust multiple-model adaptive con- mean of the state and one can calculate the true trol (RMMAC). conditional covariance matrix; see, for example, (Athans and Chang, 1976); (Anderson and Moore, 1979, Chap. 10). 2.1 Multiple-Model Adaptive Estimation (MMAE) Appendix I summarizes the notation and formulas associated with the MMAE algorithm One of the first uses of multiple-models was motivated by the need for accurate stochastic state- From a technical point of view the MMAE system estimation for dynamic systems subject to sig- of Fig. 1 blends optimal estimation concepts (i.e. nificant parameter uncertainty. In such problems Kalman filtering) and dynamic hypothesis-testing the estimation accuracy provided by standard concepts that lead to a system identification al- Kalman filters was not adequate. For some early gorithm. As explained in Appendix I, each KF references regarding MMAE see (Magill, 1965; generates a “local” state estimate,x ˆ (t t) and a k | Lainiotis, 1971; Athans and Chang, 1976; Hawkes residual (or innovations) signal, rk(t), which is the and Moore, 1976; Chang and Athans, 1978). We difference between the actual measurement and remark that MMAE algorithm was also referred the predicted measurement (the residual is pre- to as “partitioned estimation”, especially in the cisely the prediction error common to all adaptive research by Lainiotis. systems). Furthermore, the (steady-state) residual covariance matrix, S , k = 1, 2,...,N, associated Fig. 1 shows the architecture of the MMAE sys- k with each KF can be computed off-line. The key tem. It is assumed that a linear time-invariant to the MMAE algorithm is the so-called “posterior plant is driven by white process noise, as well probability evaluator (PPE)” which calculates, in as a known deterministic input signal, and gen- real time, the posterior conditional probability erates measurements that are corrupted by white that each model generates the data, i.e. P (t) = noise. If there is no parameter uncertainty in the k Prob H = H Y (t) , k = 1, 2,...,N. Thus, the plant, then the Kalman filter (KF) is the opti- k PPE{ represents| an} identification subsystem. The mal state-estimation algorithm; see, for example, “global” state-estimate is then obtained by the (Gelb, 1974; Anderson and Moore, 1979). More- probabilistic weighting of the local state-estimates over, under the usual linear-gaussian assumptions, as shown in Fig. 1; this “global” state estimate is the KF state-estimate is the true conditional mean precisely the true conditional mean of the state of the state. If the plant has an uncertain real- given the set Y (t) of past measurements and con- parameter vector, p, one can imagine that it is trols. The true conditional covariance can also be “close” to one of a finite discrete parameter set, calculated on-line; see Appendix I for mathemat- P = p ,p ,...,p . One can then design a D 1 2 N ical details. bank of{ standard Kalman} filters (KFs) where each KF uses one of the discrete parameters pk The key property of the MMAE algorithm is that, in its implementation, k=1,2, . . . , N. It turns under suitable assumptions, one of the posterior out that, if indeed the true plant parameter is probabilities, say Pj, Pj(t) 1, where j in- identical to one of its discrete values – and this dexes the model that is “closest”→ to the correct is modeled as the hypothesis H = Hk, then the hypothesis H = Hj , even though the actual conditional probability density of the state is the plant parameter is different than pj, as t . sum of gaussian densities. Then, the MMAE of These asymptotic convergence results hinge→ upon ∞ Fig. 1 will indeed generate the true conditional information-theoretic arguments and involve non-

5 () ) > )( M

04 0

trivial stationarity and ergodicity assumptions. 2

( ) > )(

3 0 0 2

The detailed convergence proofs involve either ξ (t) θ (t)

( ) (Y ( )

- ,2

A EF

the so-called “Baram Proximity Measure (BPM)”, u( t ) G y( t )

?@ @ C@ D @

(see Baram, 1976; Baram and Sandell, 1978b,a),

ST )

V 3

- U as discussed in Appendix II or the Kullback in- ;< )

xˆ1( t ) u1( t ) NOP formation metric (see Anderson and Moore, 1979, ;< = L r1 t)(

pp.267–279). The detailed proofs are beyond the xˆ2( t ) u2( t ) u( t )

NO ;< =K r t)( Q scope of this paper. These (asymptotic) conver- 2 gence results to the “nearest probabilistic neigh-

xˆN ( t ) uN ( t )

;< => R bor” using the BPM represent the key “system- rN t)(

identiﬁcation” algorithms associated with both

5 ) (

S 6 P( t )

0 0 2

1 2 1

5 ) (

0 0

2 2

* + .

'( )

: :* . * 8

5 6

the CMMAC and the RMMAC algorithms dis- ,-

2 -

7 8 9 7

() S P( t ) ()

2 2 6

()

0 3 4

1 -2 -

1 - ,- 2

9 : :*.*

()

0 -

cussed in the sequel. 2

I J

55H SN PN ( t ) It should be noted that the MMAE architecture is essentially identical to that of the “sum of Fig. 2. The CMMAC algorithm where the “local” Gaussians” estimators used extensively in nonlin- KF state estimates get multiplied by “local” ear filtering (Anderson and Moore, 1979) which LQ gains to form “local” controls which, in utilize banks of extended Kalman filters. Further- turn, generate a probabilistically weighted more, it is important to stress that the blend of “global” control. dynamic hypothesis-testing concepts and optimal estimation theory is the workhorse of all mod- 1975; Shomber, 1980; Athans et al., 1977; Greene ern defense and civilian surveillance and fusion and Willsky, 1980; Schiller and Maybeck, 1997; algorithms that employ several sensors, several Maybeck and Griffin, 1997; Schott and Bequette, targets (crossing, manoeuvring, disappearing, re- 1997). In the context of this paper it is important appearing, etc.) to stress that no robustness to unmodeled dynamics was considered in early CMMAC designs (such robustness issues were unknown in the 1970s) and 2.2 Classical Multiple-Model Adaptive Control that performance specifications were generated by (CMMAC) LQG “tricks” and not with the frequency-weight concepts widely adopted at present. The intriguing convergence properties of the MMAE algorithm, coupled with the robustness It is also noteworthy to mention that a “switch- shortcomings of MRAC systems to disturbances ing” version of the CMMAC can be implemented and sensor noise, gave rise to the classical MMAC in which the “local” control with the largest pos- algorithms which simply integrated design con- terior probability is used as the “global” control; cepts from Linear-Quadratic-Gaussian (LQG) con- such switching versions of the CMMAC are much trol system design (Kwakernaak and Sivan, 1972; more sensitive to the stochastic signals. Finally, Anderson and Moore, 1990) with the MMAE in retrospect, the fact that both the “local” KF architecture. The classical MMAC architecture state estimates and their residuals, via the pos- (CMMAC) is shown in Fig. 2. Each “local” state- terior probabilities, were combined in the genera- estimate of the (steady-state) KF,x ˆk(t t), k = tion of the “global” adaptive control has certain | 1, 2,...,N, is multiplied by the associated linear- shortcomings, because one mixes state-estimation quadratic (LQ) optimal control-gain matrix, Gk, and feedback control generation. In the CMMAC to generate a “local” control signal uk(t) by architecture no clear-cut “separation-principle” of an “identification” subsystem and a “control” uk(t) = Gkxˆk(t t), k = 1, 2,...,N (2.1) − | subsystem is made. Using precisely the same MMAE calculation by the PPE of the posterior probabilities P (t), the k 2.3 Supervisory Switching Multiple-Model Adapt- “global” control applied to the plant – and to ive Control (SMMAC) the bank of KFs – is obtained by probabilistic weighting of the local controls, i.e. During the past decade a different, more de- N terministic, approach to adaptive control using u(t) = Pk(t)uk(t) (2.2) multiple models has been initiated (Morse, 1996, kX=1 1997, 1998, 2004; Anderson et al., 2000, 2001; The CMMAC algorithm of Fig. 2 represented a Hespanha et al., 2001) and research along these purely ad-hoc approach to adaptive control. It lines is still in progress. The methodology is called is not optimal in a stochastic LQG sense (Will- Supervisory Switching Multiple-Model Adaptive ner, 1973). However, several adaptations, exten- Control (SMMAC). The architecture of the SM- sions and simulations have been reported (Greene, MAC algorithms is shown in Fig. 3 (which is an

k l y

] m pjzs]

j]im

s stable feedback performance for at least one dis-

mmim]

ξ (t) θ (t) s

l crete parameter p P . The basic idea is to

sp] mswsj ]

t k D

t uij mi ]

a e f

g y( t )

`b ` d `

u( t ) ` c _ ∈ use the monitoring signals µk(t) to “switch-in”

the suitable controller. This is accomplished by a

q lk { lk l

q lk { l qq

s] wp im]

uij mi sm] v

v switching logic S which generates a signal σ(t)

\ ^

Z e1 ( t ) u( t )

]

u 1

[

k lk

s z ij

m ∈

mmim]

s PD that can be used to switch-in the appropriate ^

\ u( t ) ^

\ e( t ) u ]

Z 2 2 u( t ) controller. A key property of the switching logic

]

}

| | | S is that it keeps its output σ(t) constant over

some suitably long “dwell time”; this avoids rapid ^

\ e( t ) Z ^

N \ u( t )

] ]

u N ~

~ switching of the controllers and allows most tran-

kl k

z jn

sients to die-out between controller switchings.

k k l

zp im µ j

1 (t )

kl k kl k

ij im jn z jn

µ σ µ

k k q o (t )

2 ( ) The details of the switching logic diﬀer in the cited

in z pj jp

n k k

q q k l

s ws sjsmp im µ N (t ) SMMAC references. The basic structural difference between the CM- Fig. 3. The SMMAC architecture. MAC and the SMMAC – compare Figs. 2 and 3 – is that in the SMMAC the “identification” adaptation of Fig. 1 in (Hespanha et al., 2001), process is completely separated from the “con- so that comparisons with the CMMAC and RM- trol” process. Even in the case of the CMMAC MAC become easier). We remark that in SMMAC that the largest posterior probability switches the architectures there exists a “separation” between corresponding LQ control gain, the identification identification and control (unlike the CMMAC). and control get mixed-up. The separation of the We briefly discuss the SMMAC architecture to identification and control processes in the SM- point out some similarities and differences to the MAC seems to have an advantage, coupled with CMMAC of Fig. 2 and the RMMAC architecture the idea of infrequent controller-switching. Oth- to be discussed below. The approach is determinis- erwise, the KFs in the CMMAC serve the same tic and the goal is to prove “local” bounded-input objective as the multi-estimators of the SMMAC. bounded-output stability of the SMMAC system Moreover, the multi-controllers in the SMMAC under certain assumptions. The plant-disturbance can be more complex than the simple LQ-gains in and sensor-noise signals are assumed bounded the CMMAC. Some potential shortcomings of the rather than being characterized as stochastic pro- SMMAC methodology will be discussed after we cesses as in CMMAC and RMMAC. The pres- present the RMMAC approach below. Unfortu- ence of unmodeled dynamics is also considered, nately, SMMAC numerical simulations have been although the bound on the unmodeled dynamics reported for only a couple of (very) academic SISO is simply an H∞ bound. The theoretical results plants. to date are restricted to single-input single-output (SISO) systems, although research is underway to extend them to the MIMO case. 2.4 Robust Multiple-Model Adaptive Control In reference to Fig. 3, the SMMAC employs a (RMMAC) finite number of stable deterministic estimators (Luenberger observers), called multi-estimators We now present the newest multiple-model ar- and denoted by Ek(s), k = 1, 2,...,N, designed chitecture which we call RMMAC, to empha- for a grid of distinct parameter values pk P , size the fact that both stability-robustness and where P is the compact real-parameter set.∈ The performance-robustness are addressed from the output of each estimator yk(t) is compared with start. Preliminary results on RMMAC can be the measured (true plant) output y(t) to form found in (Fekri et al., 2004b,c) and a more com- the estimation (prediction) errors ek(t) = yk(t) plete treatment is available in Fekri’s Ph.D. thesis y(t), k = 1, 2,...,N. We remark that the errors− (Fekri, 2005). We note that the RMMAC architec- ek(t) in Fig. 3 are completely analogous to the ture has a “separation” between identification and residuals rk(t) in the MMAE (Fig. 1), the CM- control, like the SMMAC and unlike the CMMAC. MAC (Fig. 2) and the RMMAC (Fig. 4). The The RMMAC architecture is shown in Fig. 4. It monitoring signal generator, M(s), is a dynamical represents a blend of the CMMAC, Fig. 2, and of system that generates monitoring signals µ (t); k the SMMAC, Fig. 3. these are suitably-defined integral norms of the estimation errors ek(t). The size of these monitoring As in the CMMAC, the RMMAC uses a bank signals indicates which of the multi-estimators is of (steady-state) Kalman Filters (KFs) and relies “closer” to the true plant. In addition to the bank on stochastic processes for the disturbance sig- of multi-estimators, it is assumed that a family nals and the sensor noise measurements. However, of multi-controllers, Ck(s), k = 1, 2,...,N, has unlike CMMAC the “local” KF state-estimates been designed so that each provides satisfactory (the xˆ (t t) in Fig. 2) are not used in gener- k |

§ ®

¨ totic identiﬁcation. See Appendix II. This requires

ξ (t) θ (t)

careful design of the KFs, perhaps robustiﬁed

±¨

¢ ¨

¯ °

u( t ) y( t )

through the use of “fake-white-plant-noise” – a

time-honoured design trick in linear and nonlinear

estimation practice. r1 (t) u1( t ) K1( s )

u( t )

r2 t)( 2 u( t ) K2 ( s )

2.5 Discussion

rN t)( uN ( t ) KN ( s )

The three multiple-model algorithms (CMMAC,

S P( t )

1 1

ª S P( t ) SMMAC, and RMMAC) presented above have the

2 2

§ ¨§

¡ ¢ ª

£ ¤

SN PN ( t ) following common characteristics. (a). One must design a set of N multi-controllers Fig. 4. The RMMAC architecture. or dynamic compensators. (b). One must design a set of N Kalman filters or ating the control signals. Only the KF residu- multi-estimators (observers). als, rk(t), k = 1, 2,...,N, generated on-line and their pre-computed residual covariance ma- (c). One must implement an identification process by which the actual (global) adaptive control is trices, Sk, are utilized by the posterior probability evaluator (PPE) to generate the posterior generated. probability signals Pk(t). The calculation of the Clearly, the complexity of the adaptive system will posterior probabilities is identical to that in the depend on the number, N, of models that are MMAE or CMMAC (see Appendix I). A crucial required to implement. Ideally, N should be as difference is that the “nominal” KF design uses small as possible. However, it should be intuitively explicitly the Baram Proximity Measure (BPM) obvious that if N is too small the performance of to ensure asymptotic convergence of the poste- the adaptive system may not be very good. On rior probabilities. Another key difference lies in the other hand, if N is very large, one may reach the construction of the bank of the “local” ro- the point of diminishing returns as far as adaptive bust compensators, Kj (s), j = 1, 2,...,N, in performance improvement is concerned. It follows Fig. 4, which are designed using the state-of-the- that we need a systematic procedure by which, art in robust mixed-µ synthesis (Young et al., starting from a compact parameter set p P , 1992; Young, 1994; Young et al., 1995; Zhou et to define a finite set, N, of discrete values (mod-∈ al., 1996; Balas et al., 2004; Balas, 2003). These els), P = p ,p ,...,p P, that are used D { 1 2 N } ⊂ compensators, Kj (s); j = 1, 2,...,N, referred to subsequently in designing the KFs in the CM- as the “local non-adaptive robust compensators MAC and RMMAC, the multi-estimators in SM- (LNARC)”, are designed so as to guarantee “lo- MAC as well as the compensators. The following cal” stability- and performance-robustness. Each section summarizes our suggested methodology “local” compensator Kj(s) generates a “local” which hinges upon the recent developments in ro- control signal uj(t). The “global” control is then bust feedback control synthesis using the so-called generated, as in the CMMAC case, by the proba- mixed-µ methodology and associated software. bilistic weighting of the local controls uj(t) by the We conclude this section with a few comments. posterior probability Pj(t) Even though all MMAC architectures are made N by piecing together LTI systems, the probabilis- u(t) = Pj(t)uj (t) (2.3) tic weighting in the RMMAC as well as the j=1 X supervisory switching logic of the SMMAC result in a highly nonlinear and time-varying closed A switching version of RMMAC can also be imple- loop MIMO feedback system. Hence, it is na¨ıve mented by finding the largest probability P (t), at j to expect foolproof global asymptotic stability re- each instant of time, and using the corresponding sults in the near future, because there does not, “local” u (t) as the “global” control u(t). More- j as yet, exists a solid mathematical theory for over, a “dwell-time” can be incorporated to avoid global (stochastic) nonlinear time-varying stabil- frequent switching. ity. Even in the simpler CMMAC, involving LQG One relies upon the convergence properties of the controllers, attempts to prove global stability posterior probabilities to the nearest probabilis- were not successful (Greene, 1975; Shomber, 1980; tic neighbor, using the Baram proximity mea- Greene and Willsky, 1980). Thus, it is the opinion sure (BPM) (Baram, 1976; Baram and Sandell, of the authors, what is needed in the short run is 1978b,a), to ensure that the RMMAC operates in additional pragmatic understanding of the differ- a superior manner, so as to ensure correct asymp- ent multiple-model approaches, their similarities and differences and consistent fair comparisons dominant closed-loop poles etc); these reflect the on performance improvement over non-adaptive common objective to have small tracking errors in designs. Thus, there are numerous opportuni- the low-frequency region and small control signals ties for future theoretical research to investigate in the high-frequency region. such global stability-robustness and performance- (d). Unmeasurable plant disturbances, perhaps robustness issues, especially in the MIMO case. with information on their power spectral densities. (e). Unmeasurable sensor noises, perhaps with 3. DESIGNING ROBUST COMPENSATORS information on their power spectral densities. From now on, we focus our attention to the 3.1 Introduction problem of “disturbance-rejection” in the presence of noisy sensor measurements so as to simplify the During the past several years a very sophisticated exposition. Adding “command-following” to the and complete non-adaptive design methodology, specifications is straight forward, but complicates accompanied by Matlab design software, has the exposition. What we want to stress relates been developed for the robust feedback control to the philosophy that we cannot design adaptive of MIMO linear time-invariant (LTI) uncertain control systems without explicit quantification of dynamic systems with simultaneous dynamic and desired performance. parametric errors. This design methodology is often called the “mixed-µ synthesis” method, which Assume that we have a state-space description of involves the so-called D,G-K iteration, and it re- the plant (excluding unmodeled dynamics) of the quires the design of different H∞ compensators at form each iteration. The outcome of the “mixed-µ syn- d x(t) = A(p)x(t) + B(p)u(t) + L(p)d(t) thesis” process is the definition of a non-adaptive dt (3.1) LTI MIMO dynamic compensator with fixed pa- y(t) = C1(p)x(t) + D(p)n(t) rameters, which guarantees that the closed-loop z(t) = C(p)x(t) feedback system enjoys stability-robustness and where x(t) is the state vector, u(t) the control performance-robustness, i.e. it meets the posed vector, d(t) the plant-disturbance vector, y(t) the performance specifications in the frequency do- (noisy) measurement vector, n(t) the sensor noise main (if such a compensator exists). The “mixed- and z(t) the performance (output or error) vector, µ synthesis”, loosely speaking, de-tunes an opti- i.e. the vector for which we wish to minimize the mal H∞ nominal design, that meets more strin- effects of the disturbance d(t) and noise n(t), i.e. gent performance specifications, to reflect the have superior disturbance-rejection. The system presence of inevitable dynamic and parameter matrices depend upon a real-valued parameter errors. In particular, if the bounds on key pa- vector p, where p is constrained to be in a (hyper) rameter errors are large, then the “mixed-µ syn- parallelepiped, p Pπ; this is a required µ- thesis” yields a robust LTI design albeit with synthesis constraint.∈ Thus, for each independent inferior performance guarantees as compared to real uncertain parameter we must have a lower- the H∞ nominal design. So the price for stability- and upper-bound (real-valued). robustness is poorer performance. Experience has shown that if the bounds on the parameter er- From a performance point of view, in order to rors decrease, then the mixed-µ synthesis yields achieve superior disturbance-rejection, the de- a design with better guaranteed performance. A signer specifies a frequency weight on z(t). Typi- recent application (Barros et al., 2005) illustrates cally, to achieve superior disturbance-rejection in this uncertainty/performance tradeoff in a clear the low-frequency region, the designer specifies, in manner. the µ-synthesis methodology, a frequency-weight, say of the form Whether we are dealing with non-adaptive or α m adaptive feedback designs we must take into ac- z¯(s) = A I z(s) (3.2) p s + α · · count the following engineering issues: µ ¶ which implies that superior disturbance-rejection (a). Complex-valued plant unmodeled dynamics is most important in the frequency range 0 ω (e.g. unmodeled time-delays, plant-order reduc- α. The larger the performance-gain parameter≤ A≤, tion, parasitic high-frequency poles and zeros, p the better the desired performance-rejection. high-frequency bending and torsional modes etc). To complete the robust design synthesis the de- (b). Errors in key real-valued plant parameters in signer must provide frequency-dependent bounds its state space realization. for all (structured or unstructured) unmodeled (c). Explicit definition of performance require- dynamics, and frequency weights for the control, ments typically in the frequency-domain (rather disturbance and sensor noise vectors. The control than just the shape of step responses, location of and sensor weights, together with the bound(s) on the unmodeled dynamics, safeguard against very Table 1. high-bandwidth feedback designs. List of Performance Evaluation Tools

(1) Magnitude (or singular-value) Bode plot of the 3.2 The GNARC Design closed-loop transfer function from the plant disturbance, d, to output, z, which measures the quality of disturbance-rejection vs frequency. Assuming that the Before a wise designer can make a decision on plant has no integrators, the magnitude of this trans- whether or not to implement an adaptive sys- fer function, in the low frequency region, will be ap- tem, he/she must have a solid knowledge on proximately 1/Ap. This is why we must maximize the what is the best robust non-adaptive design. We performance parameter Ap for superior disturbance- called the best “global non-adaptive robust com- rejection. pensator” GNARC. The GNARC is computed (2) Magnitude (or singular-value) Bode plot of the via mixed-µ synthesis and it takes into account closed-loop transfer function from the sensor noise, n, the frequency-domain bounds on unmodeled dy- to output, z, which measures the quality of insensitivity namics, and the various frequency weights that to sensor noise vs frequency quantify disturbance-rejection requirements, con- (3) Magnitude (or singular-value) Bode plot of the trol effort and, perhaps, power spectral densities closed-loop transfer function from the plant distur- for the plant-disturbances and sensor-noises. bance, d, to control, u, which measures the impact of the plant-disturbance on the control vs frequency We certainly do not intend to provide here a tuto- rial exposition of the mixed-µ synthesis method. (4) Magnitude (or singular-value) Bode plot of the closed-loop transfer function from the sensor noise, n, To compute the GNARC, one fixes the perfor- to control, u, which measures the impact of the sensor mance gain Ap to some initial value and exercises noise on the control vs frequency the Matlab software which, after a sequence of (5) Root-mean-square (RMS) tables assuming that so-called DG-K iterations, determines a compen- the plant-disturbance, d, and the sensor noise, n, are sator K(s) and generates an upper-bound µub(ω). stationary stochastic processes. Such RMS values are If readily evaluated via the solution of Lyapunov equations. Individual or combined RMS tables for the out- µub(ω) < 1 ω (3.3) ∀ put, z, and the control, u, as a function of the plant- then the resulting feedback design is guaranteed disturbance, d, and the sensor noise, n, can be com- to be stable for all “legal” unmodeled dynamics puted. and the entire parameter uncertainty p P . ∈ π (6) Time-domain responses, e.g. step- or sinusoidal- Moreover, in addition to stability-robustness, we disturbance, stochastic signals, etc. are guaranteed that we meet or exceed the posed performance requirements. It should be evident, that in order to find the “best” GNARC we must become acceptable if more controls and/or sensors maximize the performance-parameter A until the p are introduced and a MIMO GNARC analyzed, µ-upper bound is just below unity, say µ (ω) ub thereby eliminating the need for complex adaptive 0.995 ω. ≈ ∀ control. The GNARC is a single dynamic (SISO or MIMO) compensator that can be used by the designer to fully understand what is the best possible robust 3.3 The FNARC Design performance in the absence of adaptation. Since the feedback system is LTI a whole variety of If the GNARC analyses discussed above indicate performance evaluations are possible, using rep- the need for adaptive control, they will be used resentative values of the uncertain real-parameter to provide a lower-bound upon robust perfor- vector p Pπ for the plant, as summarized in mance. The “fixed non-adaptive robust compen- Table 1. ∈ sators (FNARC)” provide the means for quantifying an upper-bound on robust performance. “Ide- We believe that such a thorough understanding ally”, the FNARC analyses assumes an infinite of the non-adaptive GNARC is essential prior to number of models, N , in any multiple-model making a decision on using some sort of multiple- adaptive scheme. Thus,→ ∞ we understand what is model adaptive control. It has been our experience the best possible performance if we knew the real that such GNARC analyses point out which pa- parameter exactly. rameter uncertainty is most critical; this can be used to eliminate from further consideration non- In practice, to determine the FNARC one uses critical parameters. Moreover, the performance of a dense grid of parameters pj, j , in Pπ the GNARC, say for a SISO system, can change and determines the associated robust→ ∞ compen- drastically if one adds more measurements and/or sator for each pj using exactly the same bounds controls. Thus, an unacceptable performance for a on unmodeled dynamics and frequency weights SISO GNARC non-adaptive system may, indeed, employed in the GNARC design. Thus, we can make fair and meaningful comparisons. For each Ap AF ( p )

pj we use the complex-µ design methodology and p

È Ñ

³ ´ µ ¶ · ¸ ¹ º » ¼

Matlab software (Balas et al., 2004), because Ç

Ì È

both the performance weights and bounds on un- Ì

Ç Î modeled dynamics are complex-valued and there Í G

Ì Ap Ë are no real parameter uncertainties. For each p , È

j ÉÊ

² ³ ´ µ ¶ · ¸ ¹ º » ¼

È Ç we again maximize the performance-parameter Æ Ap in (3.2) until the complex-µ upper-bound,

p ¸ Å ¸ p

ÃÀÂÀÄ Â c L p ½ ¾ ¿ À Á À Â U µub(ω) is just below unity for all frequencies, say µc (ω) 0.995 ω, to be consistent with the ub ≈ ∀ GNARC upper-bound employed. Fig. 5. Hypothetical comparison of the performance parameter, A , for the GNARC and One can then analyze each FNARC design us- p FNARC for the case of a scalar uncertain real ing the six techniques outlined in Table 1. Our parameter, p, p p p . experience indicates that detailed analyses, espe- L ≤ ≤ U cially at the corners of the hyper-parallelepiped Pπ, provide useful insights regarding the impact The benefits decrease if the unknown parameter of the subset of the uncertain real parameters is closer to its lower-bound, i.e. p pL. Indeed F ≈ G that determine the need for sophisticated adaptive it may well happen that Ap (pL) = Ap . This control. can occur, for example, if the parameter p, in rad/sec, represents the value of a non-minimum phase zero which places inherent restrictions upon 3.4 The Potential Benefit of Adaptive Control disturbance-rejection (see, e.g. (Aström,˚ 2000)). Such an non-minimum phase system has been Recall that we had initially posed the follow- analyzed, using the RMMAC, in (Fekri et al., ing question: do we need adaptive control? The 2004a). GNARC and FNARC results provide the designer with the tool to answer this question. If we have an uncertain m-dimensional parameter vector, constrained in a hyper-parallelepiped, i.e. In Fig. 5 we visualize a hypothetical plot of the p P Rm, the calculations are more numerous outcome of the GNARC and FNARC designs be π to∈ construct⊂ the two hypersurfaces along the lines plotting the (maximized) performance parame- suggested by Fig. 5. However, the same philosophy ter A , see (3.2), as a function of a scalar un- p still applies. certain real parameter, p, pL p pU . We denote the (constant) value associated≤ ≤ with the G GNARC design by Ap . We denote the parameter- dependent value associated with the FNARC by AF (p), p p p . The difference, AF (p) p L ≤ ≤ U p − 3.5 Determining the Number of Models and Desig- AG 0 quantifies the impact of the uncertain p ≥ ning the LNARCs parameter p [pL,pU ] upon performance. The FNARC process∈ indicates that if the parameter p were known exactly, then the (low-frequency) Let us suppose that we have decided that there is F a substantial benefit in using adaptive control and disturbance-rejection is approximately 1/Ap (p). The GNARC process indicates that if the param- that we wish to use a multiple-model architecture. eter p were unknown, then the (low-frequency) As we remarked before, the adaptive complexity disturbance-rejection is, at worse, approximately is directly related to the number N of models in G either the SMMAC or RMMAC implementation. 1/Ap . Note that to obtain the FNARC benefits we must implement a multiple-model architecture Recall that the non-adaptive GNARC requires N = 1 model while the FNARC requires N = with an infinite number of models. In this man- ∞ ner we have quantified the potential performance models. Clearly, there must be a happy medium. benefits of adaptive control; the non-adaptive GNARC provides the lower-bound upon expected 3.5.1. A “Brute-Force” Approach. A “brute- performance, while the (infinite-model) adaptive force” approach is to decide on the number of FNARC provides the performance upper-bound. models, say N=4, and their parameter variation This information is critical in deciding whether to as illustrated in the hypothetical visualization of implement an adaptive control system. Fig. 6. Essentially, in this approach, the designer The shapes of the curves in Fig. 5 provide addi- fixes the “adaptive complexity”, quantified by tional valuable information. In the hypothetical N, of the multiple-model system. Each model, case of Fig. 5, we should expect the benefit of denoted by M#k (k=1,2,3,4) requires definition using adaptive control to be greatest if the pa- of its “local” lower-bound, pkL, k = 1, 2, 3, 4, and rameter was near its upper-bound, i.e. p p . upper-bound, p , k = 1, 2, 3, 4, i.e. ≈ U kU Ap â Ó Ô Õ Ö models and their numerical specification. This method fully exploits the information provided by

4 Ù

Ø A

á p

Ø 3 the FNARC curve in Fig. 5.

Ü Ap

Ý Ù

Ý 2

à Ap We have remarked that the number of models Ø

ß 1 Þ A Ý p

Ü required for any adaptive multiple-model design

Ù ÚÛ

Ù should be the natural outcome of performance de-

Ø ÒÓÔÕÖ

× sign speciﬁcations. In the so-called % FNARC ap-

ä å ã ä æ ã ä ç ã ä è ã p proach the designer specifies that the performance p= p p= p p= p p= p p= p L 11 12 21 22 31 32 41 44 U parameter, Ap, should be equal or greater than X% of the best possible performance as defined Fig. 6. Illustration of a “brute-force” selection of by the FNARC. four models. An ad-hoc decision is made on The basic idea is illustrated in Fig. 7, starting the number of models, N=4, and the spec- from the GNARC and FNARC curves of Fig. 5. ification of the “boundary”, [pkL,pkU ], k = k Using the designer-specified value of X%, we con- 1, 2, 3, 4, for each model M#k. The Ap de- struct the X% FNARC, shown in Fig. 7, and we note the maximized value of the performance proceed as follows. Since the FNARC is maximum parameter. at p = pU , we slowly increase, starting from the upper limit, the size of the parameter uncertainty Model #1 (M#1) : pL = p1L p p1U set Ω = p : α p p . For each value of α, we ≤ ≤ α { ≤ ≤ U } Model #2 (M#2) : p1U = p2L p p2U use the mixed-µ software to design the best robust Model #3 (M#3) : p = p ≤ p ≤ p 2U 3L ≤ ≤ 3U controller by maximizing the performance param- Model #4 (M#4) : p3U = p4L p p4U = pU α α ≤ ≤ eter Ap denoted by Ap max. As long as Ap max > F F The model upper and lower bounds must be (X%) Ap (α), where Ap (α) is the parameter value of· the FNARC at p = α, then we decrease α found by trial-and-error. ∗ until at α = α∗ we have Aα = (X%) AF (α∗). After the models are selected, all frequency- p max p The outcome of this process defines the· dashed dependent bounds and weights are fixed as in ∗ curve labeled Γ in Fig. 7. The point α is at the the GNARC and FNARC designs. Next, for each 1 intersection of the Γ curve with the X% FNARC model the performance parameter A – see (3.2) – 1 p curve. This defines Model #1 (M#1) with uncer- denoted now by Ak, k=1,2,3,4, is maximized using ∗ p tainty set Ω = p : α p p ; we also remark the mixed-µ software, in an iterative mode, for the 1 U that the µ-software{ determines≤ ≤ } in addition the smaller parameter uncertainty subset associated LNARC #1 controller denoted by K (s). with each model. Thus, for each model, we are 1 again attempting to attain as large a disturbance- The process is repeated from the (right) boundary ∗ ∗ rejection as possible. Moreover, at the end of this of M#1, α . Starting at p = α , we define the ∗ iterative optimization process, we obtain what set Ωβ = p : β p α . For each value of β, { ≤ ≤ } we call the “local non-adaptive robust compen- we use the mixed-µ software to design the best sator (LNARC)” which we denote by Kj(s), j = robust controller by maximizing the performance β 1, 2, 3, 4. parameter Ap denoted by Ap max. As long as Aβ > (X%) AF (β), where AF (β) is the The values of the Ak, k=1,2,3,4, plotted in Fig. 6 p max p p p parameter value of· the FNARC at p = β, then β is ∗ can be used as a “guide” for adjusting the bound- ∗ β decreased until at β = β we have Ap max = (X%) aries of each model or changing the number of F ∗ · Ap (β ). The outcome of this process defines the models. Each of the resulting LNARC designs ∗ can be evaluated in more detail by following the dashed curve labeled Γ2 in Fig. 7. The point β suggestions in Table 1. is at the intersection of the Γ2 curve with the

The RMMAC numerical results presented in ù ê ë ì í

F ü ý (Fekri et al., 2004a,c) followed such an ad-hoc Ap ( p ) ú û

A ù ê ë ì í

approach for determining the models and the as- p

ð ï

sociated LNARCs. The same process can be used ÿ

ó ô

to design the multi-controllers in the SMMAC, ð

ï ö

although this was not done in their numerical é ê ë ì í

ô ó

simulations. ð

ñò

ð ï î Ω Ω 2 1

3.5.2. A More Systematic Approach to Model Se- M #2 M #1

p p β* α* p lection and Definition: The % FNARC method. L U The numerical simulations for the RMMAC design in Section 5 utilize a much more systematic Fig. 7. Visualization of the % FNARC model approach to the determination of the number of definition process. X% FNARC curve. This defines Model #2 (M#2) PPE will lead to the correct model identifica- ∗ ∗ with uncertainty set Ω2 = p : β p α . It tion. Appendix II presents the summary concepts is noted that the µ-software{ also determines≤ ≤ } the leading to the on-line generation of the posterior LNARC #2 controller denoted by K2(s). The probabilities and contains the key equations for process is repeated until the parameter lower- calculating the Baram proximity measure (BPM). bound is reached. The design of the KFs must be done after the The % FNARC method is straightforward for number of models and their boundaries have been systems involving a single scalar uncertain real established using the procedures in Section 3.5. parameter. In the case of two, or more, uncer- Recall that the original parameter set, p P ∈ π tain parameters the procedure has to be modi- (a parallelepiped) is subdivided into N subsets fied. For the case of two or more uncertain pa- (parallelepipeds) denoted by Ωk, k = 1, 2,...,N, rameters, the GNARC, FNARC and % FNARC such that become surfaces. The above process yields inter- N Ωk = Pπ (4.1) section of surfaces (the equivalent of Γ1, Γ2,... are also surfaces). Unfortunately, the intersection k[=1 of these surfaces does not occur along rectangular and each Ωk defines the Model #k. We need to (or parallelepiped) parameter subsets. However, design a discrete-time steady-state KF for each the mixed-µ software requires rectangular (or par- Ωk. To accomplish this we need to specify the allelepiped) constraints for uncertain parameter “nominal” value of the parameter, denoted by sets. Suggested modifications to the % FNARC ∗ th pk Ωk, which will be used to design the k concept are discussed in (Fekri, 2005). Kalman∈ filter. The “na¨ıve” point of view would be to choose ∗ 3.6 Discussion pk at the center of Ωk. Unfortunately, this may lead to unpredictable behavior in terms of the convergence of the posterior probabilities. The In this section we presented an overview of what ∗ we believe is the proper way of designing com- correct way is to select the pk using the Baram pensators or multi-controllers for the RMMAC proximity measure. and SMMAC architectures. We are driven by the desire that we must guarantee (local) robust- stability and robust-performance. This implies 4.2 The Baram Proximity Measure (BPM) that we must exploit the state-of-the-art of the mixed-µ synthesis methodology and software. We Let p Ω and let p∗ denote the nominal value also stressed the value of having optimized per- used to∈ implement a KF. If p = p∗, then the KF formance lower-bounds (via the GNARC) and residual, r∗(t), would be a stationary white-noise upper-bounds (via the FNARC) to aid the control sequence with a specific covariance matrix S∗. If system designer in the selection of the models, the data were generated by a different LTI system, their number and their numerical specification with p = p∗, then the KF residual r(t) would (and hence complexity of the adaptive system). no longer6 be white. The BPM is a real-valued We presented two methods (there are more) for function, denoted by L (p,p∗) which measures defining the models and the associated compen- how large is a “stochastic distance” between the sators (the LNARCs) driven by designer-specified residuals r(t) and r∗(t). See Appendix II for performance or complexity specifications. details. Now suppose that p Ω, as above, but we have 4. DESIGNING KALMAN FILTERS designed two different∈ KFs, one (KF #1) with ∗ nominal value p1 Ω and another (KF #2) ∗ ∈ 4.1 Introduction with p2 Ω. Now we can calculate two BPMs, ∈ ∗ ∗ L1 L (p,p1) and L2 L (p,p2). In this section we discuss issues related to the ≡ ≡ Fig. 8 illustrates this for a scalar parameter, p, design of the Kalman filters (KFs) in the RMMAC where we visualize the BPMs L and L for all architecture. We remark that the design of the 1 2 p Ω. For the specific value p = pA shown, we can bank of KFs is much more systematic (and com- ∗ ∗ see∈ that L (p ,p ) < L (p ,p ). The implication plex) that the ad-hoc multi-estimators employed A 1 A 2 of this is that for a 2-model MMAE, when the in SMMAC architectures. true value is p = pA, the posterior probabilities The proper design of each KF in the RMMAC P1(t) 1, P2(t) 0, so that we have conver- architecture of Fig. 4 is crucial in order to satisfy gence→ to Model #1→ (defined by KF #1) occurs, the theoretical assumptions (Baram, 1976; Baram even though the Euclidean distances would indi- and Sandell, 1978b,a) which will imply that the cate the opposite ( p p∗ > p p∗ ). | A − 1| | A − 2| L model identification” by the posterior probabili- i L= Lpp( ,* ) 2 2 ties. We stress that a KF is optimal if indeed the data is generated by the same model as that used = * L1 Lpp( ,1 )

£ to design the KF. In adaptive control, however, ¢ ¡ the true parameter is NOT identical to that of the “closest” KF in the sense of Section 4.3. Also, the unmodeled dynamics are not taken into account * * p1 pA p2 p in the design of the KF. Moreover, the KFs and the PPE must perform well even when some of Fig. 8. Illustration of Baram proximity mea- the stochastic assumptions are violated. Thus, we sures (BPM). must “robustify” the KF to be tolerant of such “errors”. The time-honored engineering practice of using suitable “fake plant white noise” is a Fundamental convergence result: In (Baram, 1976) very useful tool. We have found (Fekri, 2005) that this result was generalized and proved for an judicious use of “fake plant white noise” (which arbitrary number of KFs designed for the nominal ∗ ∗ ∗ causes the KF gains to increase and pay more values p1,p2,...,pN . If the BPM satisﬁes the attention to the measurements) can be a very inequality valuable tool in improving the quality and speed ∗ ∗ of probability convergence. We believe that future L(p,pj ) < L(p,pk) k = j = 1, 2,...,N (4.2) ∀ 6 fundamental studies of robustifying KFs, in the then, under some additional stationarity and er- context of multiple-model adaptive control, are godicity assumptions – see Appendix II – the very relevant. Also, the identiﬁcation must work posterior probabilities converge almost surely to when the parameter changes “slowly” as a func- the correct model, i.e. tion of time.

lim Pj(t) 1 (4.3) t→∞ → It is also important to note that similar probability convergence results for the MMAE can be found in (Anderson and Moore, 1979, pp.267–279) 4.3 Integrating Probability Convergence Results and cited references therein) which used the so- for the RMMAC called Kullback information metric. The Kullback metric, however, is not a true norm (it violates The outcome of the controller design of Sec- the triangle inequality). Nonetheless, it would be tion 3 yielded the number of models, N, and highly desirable that future research studies the their boundaries, in terms of the parallelepipeds deep relationship of the BPM and the Kullback Ω1, Ω2,..., ΩN which deﬁne the models M#1, metric, especially with respect to posterior prob- M#2, . . . , M#N. The question now is: how do we ability convergence behavior. ∗ ∗ ∗ select the nominal values p1,p2,...,pN to design the KFs?

The idea, illustrated for three models in Fig. 9 for 5. RMMAC SIMULATIONS a scalar parameter, is to use an iterative algorithm to calculate the nominal KF values p∗,p∗,p∗, 1 2 3 We described above how the RMMAC architec- so that the BPMs are equal at the boundary of ture combines the state-of-the-art in mixed-µ ro- adjacent Ω . s bust synthesis and multiple model adaptive esti- In this manner, the fundamental probability con- mation (MMAE) system identiﬁcation. Further- vergence result will guarantee that p Ω more, we described the step-by-step design pro- ∈ j ⇒ Pj(t) 1 a.s. This, is the method we use in the cess required to implement a RMMAC design. numerical→ RMMAC simulations of Section 5. L This method becomes more complicated when we i L have two, or more, uncertain parameters. It be- 3

comes necessary to use some sort of genetic algo- ¦ L2 ¥ rithm to determine the nominal KF optimization ¤ L design points in an optimal manner (Fekri, 2005). 1

* * * p3 p2 p1 p Ω Ω Ω 4.4 Discussion 3 2 1 Ω We have presented a brief summary on how to optimize the design of the KFs in the RMMAC ar- Fig. 9. Optimizing the KF nominal design points chitecture using the BPM, so as to ensure “correct using the BPMs.

In this section we test and evaluate the disturbance-

rejection performance of the RMMAC feedback

system as compared to the “best” non-adaptive "

design obtained via mixed-µ robust synthesis. We § ©

follow the step-by-step design methodology of Sec- § ¨

# ¨ tion 3 and 4. © We present the outcome of several Monte Carlo (MC) simulations, which demonstrate the actual Fig. 10. The two-cart system. The spring-constant RMMAC performance improvements, as com- k is uncertain. pared to that of the best non-adaptive GNARC 1 system. Overall, for the example considered, the proposed RMMAC design yields significant per- m1 = m2 = 1, k2 = 0.15, b1 = b2 = 0.1, α = 0.1 formance improvements over the robust non- (5.3) adaptive one, thereby confirming the positive as- The upper and lower-bound for the uncertain pects of this adaptive control method. spring constant, k1, are: Ω = k : 0.25 k 1.75 (5.4) { 1 ≤ 1 ≤ } 5.1 The Two-Cart Example Dynamics The performance variable (output) z(t) is the position of mass m2, The RMMAC was tested and evaluated using z(t) x2(t) (5.5) the two-cart mass-spring-damper (MSD) system, ≡ shown in Fig. 10. A different topology of the two- All feedback loops utilize a single measurement cart system was analysed by RMMAC in (Fekri y(t), that includes additive white sensor noise θ(t), et al., 2004c). The system in Fig. 10 includes a defined by random colored disturbance force, d(t), acting on y(t) x2(t) + θ(t) ≡ − (5.6) mass m2 and sensor noise on the only measure- E θ(t) = 0, E θ(t)θ(τ) = 10 6 δ(t τ) ment of the position of mass m2. The control force { } { } − u(t) acts upon the mass m1. The disturbance force The desired disturbance-rejection requires that d(t) is a stationary stochastic process generated by the effects of d(t) and θ(t) be minimized so that driving a low-pass filter, Wd(s), with continuous- z(t) 0. time white noise ξ(t), with zero mean and unit ≈ intensity, as follows: Remark: The control problem is hard even if the α spring, k1, is known. Clearly, the control prob- d(s) = ξ(s) (5.1) s + α lem becomes much harder in our adaptive design, because the control u(t) is applied through the Wd(s) uncertain spring, so we are not sure how much The overall state-space| representation,{z } including force is exerted through the uncertain spring to the disturbance dynamics via the state variable the mass m2. Thus, we have a non-collocated actu- x5(t), is: ator problem because the control is not applied to x˙(t) = Ax(t) + Bu(t) + Lξ(t) the mass m2 whose position we wish to regulate. (5.2) y(t) = Cx(t) + θ(t) In addition to the uncertain spring stiffness, we where the state vector is assume that there is, in the control channel, an unmodeled time-delay τ whose maximum possible T x (t) = [x1(t) x2(t)x ˙ 1(t)x ˙ 2(t) d(t)] value is 0.05 sec, i.e. and τ 0.05 sec (5.7) ≤ 0 0 1 00 The frequency-domain upper-bound for the un- 0 0 0 10 modeled time-delay, which is a surrogate for un-  k1 k1 b1 b1  modeled dynamics, is required for mixed-µ syn- 0 thesis design and is the magnitude of the first A = −m1 m1 −m1 m1     k1 (k1 + k2) b1 (b1 + b2) 1  order transfer function    m2 − m2 m2 − m2 m2  2.1s  0 0 0 0 α Wunmod(s) = (5.8)  −  s + 40  1  BT = 0 0 0 0 ; C = 01000 m · 1 ¸ 5.2 Designing Global Non-Adaptive Robust Com- T £ ¤ L = 0 0 0 0 α pensator (GNARC) £ ¤ The following parameters in (5.2) are fixed and In this section we discuss the details behind the known: mixed-µ design of the GNARC system, which guarantees the “best” robust-stability and robust- ξ (t ) θ (t )

performance for the entire large parameter un-

) +

. .

34 9 3

6 78 6

: &:

certainty of (5.4). As explained in Section 3, the 5

) +

- + GNARC system will deﬁne what we can best ) ∆ * d( t ) δ

un k1

;

) +

expect in the absence of adaptation. . *

α , z( t )

s +α -

% '

= = As it is common in H2 or H∞ designs, we use the & z( t ) following frequency domain weights on the control

u< ( t )

) +

. *

and measurement noise, / +

u( t ) ) y( t ) ( 10(s + 10) * Control weight: W (s) = u s + 103 (5.9) − Meas. noise weight: W = 10 3 (constant) n Fig. 11. MSD system with weights for mixed-µ These, together with the unmodeled dynamics synthesis. weight (5.8), limit the bandwidth of the closed- loop system by penalizing large high-frequency we communicate to the µ-design that position control signals. The weights (5.8) and (5.9) are errors are most important below the “corner fre- incorporated in the definition of the nominal gen- quency” 0.1 rad/sec. The larger the performance eralized plant, together with (5.2) with k = 1.0. 1 parameter” A , the more one cares about position The weights (5.1), (5.8), and (5.9) will not change p errors at all frequencies. in any of the subsequent designs. The “control error”u ˜(t) is defined by To carry out the mixed-µ synthesis, the parameter s + 10 uncertainty of (5.4) is represented by u˜(s) = W (s) u(s) = 10 u(s) (5.13) u s + 103 0.25 k1 1.75 µ ¶ ≤ ≤ ⇒ (5.10) So W (s) is a high-pass filter that penalizes the k1 = 1.0 + 0.75 δk1 ; δk1 1 u | | ≤ system for using large controls at high frequencies. In order to design the “best possible” non- adaptive feedback system the following type of Using the mixed-µ software the performance pa- performance weight upon the output z(t) is used. rameter Ap in (5.11) is increased as much as 0.1 possible until the upper-bound on the mixed-µ, W (s) = A (5.11) p p s + 0.1 µub(ω), satisfied the inequality which reflects our specification for good disturbance- µ (ω) 1, ω (5.14) ub ≤ ∀ rejection for the frequency range ω 0.1 rad/sec ≤ which is only a sufficient condition for both where the disturbance d(t) has most of its power. stability- and performance-robustness. The largest Notice that performance weight Wp(s) penalizes value of the performance parameter Ap in (5.11) output error in the same frequency range as the thus determined was disturbance dynamics Wd(s) while the gain pa- G A = 50.75 (with µub 0.995) (5.15) rameter Ap in Wp(s) specifies our desired level of p ≈ disturbance-rejection. The larger Ap, the greater which leads to the GNARC design, the “best” the penalty on the effect of the disturbances on LTI non-adaptive compensator K(s) that guaran- the position. As we described in Section 3, for tees stability- and performance-robustness for the superior disturbance-rejection, Ap should be as entire parameter interval (5.4). As explained in large as possible; how large it can be is limited Section 3, the performance characteristics of the by the required guarantees on robust-stability and GNARC are to be used as the comparison-basis -performance inherent in the mixed-µ synthesis for evaluating performance improvement (if any) methodology. of our proposed RMMAC design. See Section 3.2. Fig. 11 shows the MSD plant with weights as required by mixed-µ synthesis. One can note that 5.3 Designing Local Non-Adaptive Robust Comp- there are two frequency-weighted “errors”z ˜(t) ensators (LNARCs) andu ˜(t). This figure is in fact a block diagram of the uncertain closed-loop MSD system illustrating Following the procedure presented in Section 3, the disturbance-rejection performance objective the plots of the (optimized) performance param- namely the closed-loop transfer function from eter Ap for the GNARC design and the FNARC ξ(t) z(t), or d(t) z(t). → → designs (requiring an infinite number of models) The “position error”z ˜ is our main perfor- is shown in Fig. 12. This figure shows that there mance variable for evaluating the quality of the is a potential 20-fold improvement in performance disturbance-rejection. Since by using adaptive control. 0.1 Remark: In this example, the GNARC and z˜(s) = Wp(s) z(s) = Ap z(s) (5.12) s + 0.1 FNARC look flat for all values of k1. Actually, µ ¶ F Table 2. Best GNARC and LNARCs 4 F F FNARC F 1000 3 2 1 ∗ Compensator Ω X% Ap Γ Γ 4 3 Γ Γ G 2 GNARC Ω = [0.25, 1.75] 5.1% Ap =50.75 1 1 800 LNARC #1 Ω1 = [1.02, 1.75] 70% Ap=694.5 2 LNARC #2 Ω2 = [0.64, 1.02] 70% Ap=694.5 3 600 LNARC #3 Ω3 = [0.40, 0.64] 70% Ap=694.5 70% of FNARC 4 LNARC #4 Ω4 = [0.25, 0.40] 70% A =694.5 p p A

400 ∗ Best performance gains for the GNARC and each of the four LNARCs used in subsequent designs vs “FNARC” GNARC 200 the FNARC get a little smaller as we approach small values of k , but this can not be noticed in 1 0 0.25 0.5 0.75 1 1.25 1.5 1.75 the figure. This “flatness” disappears if we change M #4 M #3 M #2 k Model #1(M #1) our control specifications (Fekri, 2005). 1 Next, we specify a desired level of performance to Fig. 13. Best local performance gains using mixed- be 70% of the FNARC, i.e. X% = 70%, following µ using 70% of the best FNARC performance the discussion of Section 3.5.2. designed for local uncertainty bounds. Fig. 13 shows how four covering models are selected to be adequate for constructing the RM- this occurs at k1 = 0.64 which yields the subset MAC using performance level of 70%. Ω2 = [0.64, 1.02], i.e. 0.64 k1 1.02, Model #2, and LNARC #2. Repeating≤ this≤ process leads to As illustrated in Fig. 13 for the specified per- the curves labeled Γ3 and Γ4 in Fig. 13 and the formance level of 70%, the large parameter un- four models summarized in Table 2. certainty interval of (5.4) is subdivided into four subintervals as summarized in Table 2. As a result, the whole initial uncertainty set of (5.4) is covered by four models using four local Explanation. We now provide a bit more detail compensators. Clearly, the reduction in parameter on how the four models of Fig. 13 are obtained. uncertainty allows larger performance gains for Starting at the North-East corner labeled F1, k1 = designing the LNARCs, resulting into guaranteed 1.75, (because the FNARC is maximum there) both stability- and -performance robustness over we slowly open-up the uncertain interval denoted the corresponding subintervals of Table 2. by Ωb = k1 : b k1 1.75 . For each interval, we{ iterate using≤ the≤ mixed-} µ software In fact the above model selection procedure us- to calculate the maximum performance parameter ing mixed-µ synthesis also generates four “local” b robust compensators, K1(s),...,K4(s), designed Ap which results in the dashed curve, labeled for each subinterval defined in Table 2; these are Γ1 in Fig. 13. When the curve Γ1 intersects the 70% FNARC curve we stop. In this example, referred to as LNARCs. In the mixed-µ synthesis, the weights (5.1), (5.8) and (5.9) were the same as this occurs at k1 = 1.02 which yields the subset in the GNARC design of Section 5.2. However, for Ω1 = [1.02, 1.75], i.e. 1.02 k1 1.75, Model M#1, and LNARC #1. The≤ left≤ boundary of each LNARC design, the performance parameter A in (5.11) is increased until the mixed-µ bound M#1 defines the point labeled F2 on the FNARC. p Starting from this point we slowly open up the of (5.14) is achieved, and their optimized values interval denoted by Ω = k : c k 1.02 . are shown in the last column of Table 2. c { 1 ≤ 1 ≤ } Once more, we iterate using the mixed-µ software Fig. 14 compares the GNARC design with the four to calculate the maximum performance parameter LNARCs, namely K1(s),...,K4(s) by looking at c Ap which results in the dashed curve, labeled their Bode magnitude plots. Γ2 in Fig. 13. When the curve Γ2 intersects the 70% FNARC curve we stop. In this example, Note that at low frequencies the LNARCs generate a loop-gain about 20 times as large compared to the GNARC and this, naturally, leads to the 3 10 performance improvements discussed below.

GNARC FNARC

p We emphasize that each individual LNARC closed-

A 2 10 loop design has guaranteed performance- and stability-robustness over its associated parameter

1 10 subinterval of Table 2. 0.25 0.75 1.25 1.75 k 1 Remark: In the design philosophy adopted in this paper we have stressed that the adaptive con- Fig. 12. Best GNARC and FNARC performance troller complexity, as measured by the number of gains for spring-stiﬀness uncertainty subin- models in the CMMAC, SMMAC and RMMAC tervals, as in (5.4). should be the natural by-product of the perfor- 4 10 probability convergence, the RMMAC essentially operates as an LTI stochastic feedback system! 3 LNARCs 10 In the spirit of Table 1, this allows us to calcu- 2 10 late two key transfer functions for disturbance- GNARC Mag. 1 10 rejection and control signal characteristics

0 10 Disturbance - rejection:

−1 10 z(s) z(s) −2 0 2 4 10 10 Freq. (rad/sec)10 10 Mξz(s) or Mdz(s) ≡ ξ(s) ≡ d(s) (5.16) Control - signal: Fig. 14. Frequency-domain characteristics (Bode u(s) u(s) plot) of the GNARC and the four LNARCs. Mξu(s) or Mdu(s) ≡ ξ(s) ≡ d(s)

for diﬀerent values of the unknown spring stiﬀness mance requirements. We have just demonstrated of (5.4), for the GNARC and for each LNARC that if we demand that the performance equals or design. exceeds X=70% of the FNARC, we require the four models summarized in Table 2. Fig. 15 illustrates the above using the actual spring constants indicated in (5.17) quantifying If we are willing to have somewhat inferior performance and select, say, X=50% then the procedure

1 outlined results in only two models. If we wish 10

0 to have much better performance and select, say, 10

−1 X=90%, then the outlined procedure yields nine 10 GNARC models. Clearly, as we demand better and better Mag. −2 10 performance we must increase the controller com- LNARC #1 −3 plexity, and this agrees with engineering intuition. 10 −4 10 −3 −2 −1 0 1 2 10 10 10 10 10 10 Freq. (rad/sec)

(a) GNARC and LNARC #1 for k1 = 1.2

5.4 Predicting Potential RMMAC Performance 1 10 Beneﬁts 0 10

−1 10 Testing the RMMAC requires signiﬁcant compu- GNARC

Mag. −2 tation using multiple Monte Carlo (MC) runs un- 10 LNARC #2 −3 der diﬀerent scenarios. 10

−4 10 −3 −2 −1 0 1 2 It is highly desirable, as explained in Section 3, to 10 10 10 10 10 10 use the LTI feedback designs, using the GNARC Freq. (rad/sec) and LNARCs, to quantify the potential beneﬁts (b) GNARC and LNARC #2 for k1 = 0.76

1 of using adaptive control in general, and the RM- 10

0 MAC in particular. From a pragmatic engineering 10

−1 perspective we must have tradeoﬀs that contrast 10 GNARC

the performance improvements (if any) of the very Mag. −2 10 sophisticated RMMAC vis-a-vis the much simpler LNARC #3 −3 non-adaptive GNARC design. To the best of our 10 −4 10 −3 −2 −1 0 1 2 knowledge, such performance tradeoffs have not 10 10 10 10 10 10 been quantified in other adaptive control studies. Freq. (rad/sec) (c) GNARC and LNARC #3 for k1 = 0.44 Referring to Fig. 4, the RMMAC requires the on- 1 line computation of its four Kalman filters (KFs) 10 0 10 as well as of its four dynamic LNARCs, K1(s), −1 10 ..., K4(s), in addition to the calculation of the GNARC

Mag. −2 four posterior probabilities, P1(s), ..., P4(s), by 10 LNARC #4 −3 the posterior probability evaluator (PPE) – a lot 10

of computations! −4 10 −3 −2 −1 0 1 2 10 10 10 10 10 10 In order to understand how one can easily predict Freq. (rad/sec) the potential RMMAC performance characteris- (d) GNARC and LNARC #4 for k1 = 0.34 tics, assume that one of the posterior probabilities converges to its nearest probabilistic neighbor Fig. 15. Potential improvement from RMMAC (which it does, as we demonstrate in the sequel); it visualized by Bode plots of disturbance- rejection transfer function M (jω) . follows that a specific LNARC is used. After the | ξz | the potential RMMAC improvement in disturbance- Table 3. Mismatched-Model Stability rejection. Model LNARC Similar plots can be made for other values of the # #1 #2 #3 #4 uncertain spring stiffness. Fig. 15 predicts that 1 S CU U U the RMMAC has the potential to significantly 2 CU S CU U improve disturbance-rejection; these predictions 3 U CU S S 4 U CU S S will be validated in the sequel, on the order of 1 . Ap Legend: S ≡ always stable ≡ Moreover, other similar plots, such as control U always unstable CU ≡ conditionally unstable signal characteristics, can be made for other values of the uncertain spring constant. Other transfer functions could also be computed (not shown) we use the LNARC #4 we always have instability from disturbance ξ(t) and the sensor noise θ(t) (U); if we use the LNARC #2 we have instability to the control u(t). for smaller values of k1, but have stability for larger values (CU). This is due to the fact that the Fig. 16 evaluates the potential performance im- mixed-µ upper-bound inequality (5.14) is “only provement of using RMMAC by using stochastic a sufficient condition for robust stability” and, metrics, namely by comparing the RMS errors of hence, each LNARC design will actually have a the output z and the control u, for different values wider robust-stability region. It turns out that of k1. Assuming that ξ and θ are indeed white for this example, LNARC #1 maintains stability noises, these RMS results are readily computed by for all k1 [0.69, 1.75], LNARC #2 for all k1 solving standard covariance algebraic Lyapunov ∈ ∈ [0.43, 1.58], LNARC #3 for all k1 [0.25, 1.01], equations for stochastic LTI systems. The graphs ∈ and LNARC #4 for all k1 [0.25, 0.73]. Of course, of Fig. 16 vividly suggest that RMMAC has the performance-robustness is∈ only guaranteed for the potential of decreasing the output RMS by a fac- subintervals defined in Table 2. tor of 2–5 over the GNARC system. Note that the potential improvement in output RMS requires controls with higher RMS values, as expected. 5.5 Designing the MMAE and RMMAC System Finally, it is important to construct what we call the “stability-mismatch” table shown in Table 3. For this example we follow the process to design The interpretation of Table 3 answers the ques- the four Kalman filters (KFs) outlined in the tion: what happens to closed-loop stability if we use RMMAC architecture of Fig. 4. the LNARC #K (s) when the true spring constant j As explained in Section 4, a great deal of care is in subinterval #i? The diagonal entries in this must be exercised in designing the Kalman fil- table are always robustly-stable, by construction. ters (KFs) in the RMMAC architecture, since the Examining the first row in Table 3 it is observed convergence of the appropriate posterior proba- that for Model #1, i.e. for all k [1.20, 1.75], if 1 ∈ bility to its nearest probabilistic neighbor is at

−2 the heart of the RMMAC identiﬁcation process ×10 8 (see Appendices I and II). The basic decision is how to select the nominal value of the uncertain 6 ∗ spring stiﬀness, k1, denoted by k1i, i = 1, 2, 3, 4, 4 GNARC for designing each of the four KFs. We stress that

RMS of z(t) 2 these nominal values are not at the centers of the LNARC#3 LNARC#2 LNARC#1 LNARC#4 sets Ωj deﬁned in Table 2. Rather, as explained in 0 ∗ 0.25 0.75 1.25 1.75 Section 4, the KF design points, k , i = 1, 2, 3, 4, (a) 1i 3 must be determined by an optimization process so LNARC#4 that the Baram proximity measures (BPMs) agree 2 LNARC#3 at the boundaries of the sets Ωj. The outcome of LNARC#2 this optimization process is shown in Fig. 17 using LNARC#1

RMS of u(t) 1 the optimized KF nominal values. GNARC In Fig. 17 the curves show the BPM for any value 0 0.25 0.75 1.25 1.75 uncertain k 0.25 k1 1.75 from each of the four optimized (b) 1 KFs.≤ This≤ process is required so that for any Fig. 16. Predicted potential RMS performance k1 Ωj , the corresponding posterior probability P ∈ 1, P 0 k = j. The speciﬁc nominal KF of the RMMAC vs GNARC. There is no j → k → ∀ 6 sensor noise for these plots. (a): Output RMS numerical values for this example are obtained by comparisons from ξ(t) to z(t). (b): Control iteration and are ∗ RMS comparisons from ξ(t) to u(t). = [1.2 0.76 0.45 0.34] (5.17) K1 −8.1 In the sequel some representative stochastic simulations are shown using the complete RMMAC i closed-loop system. Due to space limitations, we only show “typical” plots; however, our conclusions are based on thousands of other MC runs not explicitly shown in this paper (Fekri, 2005). M #4 M #3 Baram Proximities, L M #2 M #1

−8.13 5.6.1. “Easy” Identification, I: The dynamic 0.25 0.5 0.75 1 k 1.25 1.5 1.75 1 evolution of the four posterior probabilities when the true k1 = 1.65, well inside the Model #1 Fig. 17. Baram proximity measures (BPM) for the subinterval, and the corresponding outputs for the optimized four KFs. RMMAC and the GNARC systems are shown in Fig. 18. The correct model (Model #1) is identi- where k∗ = ∗[i] is the nominal spring constant fied quickly in about 2 secs. The improvement in 1i K1 k1 used in Model #i associated with the KF #i. disturbance-rejection by the RMMAC is evident as shown in 18(b). The GNARC and LNARC compensators are designed in continuous time, but the simulation was implemented in discrete time using a zero-order 5.6.2. “Easy” Identification, II: Again, the dy- hold with a sampling time of Ts = 0.01 secs. namic evolution of the four posterior probabilities The KFs in the MMAE algorithm were designed when the true k1 = 0.3, well inside the Model #4 in discrete-time using the sampling interval Ts.; subinterval, and the corresponding outputs for the all continuous-time dynamics (plant, weights, etc) RMMAC and the GNARC systems are shown in were transformed to their discrete-time equiva- Fig. 19(a). The correct model (Model #4) is iden- lents. In addition, the correct variances of the tified quickly in about 10 secs. The improvement discrete-time white noise sequences, ξ(.) and θ(.), in disturbance-rejection by the RMMAC is again were calculated and used to design the four KFs evident as shown in Fig. 19(b). and the posterior probability evaluator (PPE); these discrete-time numerical values were used in all Monte Carlo (MC) simulations in the sequel. 5.6.3. “Harder” Identification: When the actual spring constant is near the boundary between As explained in Section 2.4, the real-time KF two models, it takes longer (more data) to resolve residual sequences in Fig. 4, rj(t); j = 1,..., 4; t = 0, 1, 2,..., are used by the PPE to generate 1 on-line the four posterior probabilities, P (t);

j (t)

1 0.5 j = 1,..., 4; t = 1, 2,..., which are next used to P 0 generate the overall RMMAC control signal u(t) 1 by probabilistic weighting, i.e. (t)

2 0.5 4 P 0 u(t) = Pj(t) uj (t) (5.18) 1

j=1 (t)

3 0.5

X P where the uj(t) are the “local” controls gener- 0 ated by each LNARC, Kj (s), as designed in Sec- 0.4 (t) tion 5.3. 4 0.2 P

0 0 2 4 6 8 10 t(sec) 5.6 RMMAC Stochastic Simulations and Perfor- mance Evaluation (a) RMMAC Posterior probabilities

0.06 Unless stated otherwise: GNARC RMMAC 0.04 (a) all simulations use a stochastic disturbance and white measurement noise generated according 0.02 0 to (5.1). The true system includes an actual (but z(t) unmodeled) time-delay of 0.01 secs in the control −0.02 channel. −0.04

−0.06 0 20 40 60 80 100 (b) all initial model probabilities are initialized to time (sec) be P (0) = 0.25 (k = 1,..., 4) at t = 0 secs. k (b) Output performance comparisons (c) we present numerical averages for 5 MC simulations. Fig. 18. Simulation results for k1 = 1.65. 1 1 (t) (t) 1 1 0.5 0.5 P P

0 0 0 5 10 15 20 25 30 0 20 40 60 80 1 1 (t) (t) 2 2 0.5 0.5 P P

0 0 0 5 10 15 20 25 30 0 20 40 60 80 1 1 (t) (t) 3 0.5 3 0.5 P P

0 0 0 5 10 15 20 25 30 0 20 40 60 80 1 1 (t) (t) 4 0.5 4 0.5 P P

0 0 0 5 10 15 20 25 30 0 20 40 60 80 t(sec) t (sec)

(a) RMMAC Posterior probabilities (a) RMMAC Posterior probabilities

0.1 0.04 GNARC GNARC RMMAC RMMAC 0.05 0.02

0 0 z(t) z(t)

−0.05 −0.02

−0.1 −0.04 0 20 40 60 80 0 20 40 60 80 100 time (sec) t (sec) (b) Output performance comparisons (b) Output performance comparisons

Fig. 19. Simulation results for k1 = 0.3. Fig. 20. Simulation results for k1 = 0.405 (in Model #3).

the true hypothesis. In this example, k1 = 0.405 is selected which belongs to Model #3 but is also 0.025 “close” to Model #4, see Fig. 17. The probabilities vs time as well as output comparisons are shown

in Fig. 20. It takes about 50 secs to resolve the 0 ambiguity between Models #3 and #4. z(t)

Fig. 20(a) shows that the probabilistic weighing of LNARC #3 RMMAC the control according to (5.18) persists for about −0.025 0 10 20 30 40 50 50 secs. However, as evidenced by Fig. 20(b), there t (sec) is no significant degradation of the RMMAC performance as compared to the GNARC. In Fig. 21 Fig. 21. LNARC #3 and RMMAC performance the performance of the feedback system is shown output comparisons for k1 = 0.405. when we fix P3(t) = 1, t = 0, 1, 2,... (i.e. when we have “perfect” identification∀ from the start) with that of the RMMAC and compare the output Model #1 we have an unstable closed-loop system. response. As we can see, the probabilistic averag- To force this initial instability, the initial values of ing results in insignificant performance deteriora- the probability vector are selected to be tion. P1(0) = P2(0) = P3(0) = 0.01, P4(0) = 0.97

so that initially, at least at t = 0, the RMMAC 5.6.4. Initial Mismatch Instability: In Table 3 system is forced to be unstable. However, as illus- the mismatch-stability properties of the LNARC trated in Fig. 22, the RMMAC rapidly recovers to designs are summarized. We next evaluate the a stable conﬁguration. Fig. 22(a) shows that the RMMAC response when we force it to be unstable “correct probability” P (t) 1 within 1.3 secs, at time t = 0. 1 → starting from its initial value P1(0) = 0.01; the Fig. 22 illustrates a typical result selected from other three probabilities converge to zero within several diﬀerent MC simulations. In Fig. 22 the 1.3 secs as well. Fig. 22(b) shows the output re- true value of k1 is 1.75 in Model #1; its nearest sponse in which, after an initial period of brief probabilistic neighbor is KF #1. From Table 3 “instability”, the RMMAC recovers and returns we know that if we use LNARC #4, K4(s), with to its predictable superior disturbance-rejection. 1 1 (t) (t) 1 1 0.5 0.5 P P

0 0 0 1 2 3 4 5 0 50 100 150 1 1 (t) (t) 2 2 0.5 0.5 P P

0 0 0 1 2 3 4 5 0 50 100 150 1 1 (t) (t) 3 0.5 3 0.5 P P

0 0 0 1 2 3 4 5 0 50 100 150 1 1 (t) (t) 4 0.5 4 0.5 P P

0 0 0 1 2 3 4 5 0 50 100 150 t(sec) time (sec)

(a) Posterior probabilities transients (a) Posterior probabilities transients

0.2 0.06 RMMAC RMMAC GNARC 0.15 GNARC 0.04

0.02 0.05 0 z(t) z(t)

−0.02 −0.05 −0.04

−0.15 −0.06 0 20 40 60 80 100 0 50 100 150 time (sec) time (sec) (b) Output response (b) Output response

Fig. 22. The RMMAC recovers from an initial Fig. 23. The RMMAC recovers from a forced unstable conﬁguration at t = 0. unstable conﬁguration at T = 60 secs.

Fig. 23 illustrates another mismatch-instability 1976; Baram and Sandell, 1978b) assumes that all result, similar to the above case with k = 1.75. 1 MMAE signals are stochastic stationary random This test was suggested by Prof. B.D.O. Ander- processes. With some additional ergodicity condi- son. To force this instability the probabilities vec- tions, all results presented in Section 5.6 satisfied tor at time T = 60 secs are forced to be those assumptions, and we have indeed observed P1(T ) = P2(T ) = P3(T ) = 0.01, P4(T ) = 0.97 convergence to a nearest probabilistic neighbor. In so that the RMMAC system is forced to be this section we evaluate the RMMAC performance unstable at time t = T for 0.1 secs. Fig. 23 when we intentionally violate some of the above shows that the RMMAC rapidly recovers to a assumptions. We shall show that the RMMAC stable expected configuration. Fig. 23(a) shows still performs quite well; in a sense it appears that the “correct probability” P (t) 1 has been “robust” to violating the theoretical assumptions. 1 → forced to P1(60) = 0.01. Fig. 23(b) shows the We evaluated the RMMAC performance over a output response in which the RMMAC quickly wide variety of operating conditions, for different recovers and returns to its predictable superior values of the uncertain spring stiffness. In all the disturbance-rejection. RMMAC worked well and no instabilities were observed. The following representative cases are Remark: Provided that all convergence assump- presented. tions hold, (Baram, 1976), we observed in all our simulations that a temporary instability of the RMMAC will cause all signals to grow. As a con- 5.7.1. Step Disturbance: In this set of simula- sequence, the signal-to-noise ratio will increase, tions we used a deterministic periodic square-wave and it will be reflected in the size of the resid- disturbance, d(t) = 1.0, with a period of 60 secs. uals; this, in turn, appears to force the PPE to The sensor noise was± white as in Section 5.6. The adjust the posterior probabilities so that a stable KFs in the RMMAC were NOT aware of the configuration is re-established. square-wave disturbance; they continued to use (5.1) to model the disturbance dynamics. The true spring stiffness is k1 = 1.75. Fig. 24 shows the 5.7 Violating RMMAC Convergence Assumptions simulation result for one MC run. The theory which guarantees the convergence of Note that after one period (60 secs) of the distur- the posterior probabilities almost-surely (Baram, bance square-wave the probabilities converge to 1 1 (t) (t) 1 1 0.5 0.5 P P

0 0 0 50 100 150 200 0 5 10 15 1 1 (t) (t) 2 2 0.5 0.5 P P

0 0 0 50 100 150 200 0 5 10 15 1 1 (t) (t) 3 3 0.5 0.5 P P

0 0 0 50 100 150 200 0 5 10 15 1 1 (t) (t) 4 4 0.5 0.5 P P

0 0 0 50 100 150 200 0 5 10 15 t(sec) time (sec)

(a) Posterior probabilities transients (a) Posterior probabilities transients

0.4 0.15 RMMAC RMMAC GNARC GNARC 0.1 0.2

0 0 z(t) z(t)

−0.1 −0.2

−0.4 −0.2 0 100 200 300 350 0 20 40 60 80 time (sec) time (sec) (b) Output response (b) Output response

Fig. 24. RMMAC performance for a square-wave 10 GNARC disturbance. RMMAC 5 the correct Model #1, i.e. P1(t) 1 as shown 0 u(t) in Fig. 24(a). The RMMAC exhibits→ an oscilla- tory behavior during this time period Fig. 24(b) −5 which dies out after the probability P1(t) con- ¡ ¢ −10 verges to unity; afterwards the RMMAC perfor- 0 5 10 15 time (sec) mance is very good. (c) Control signals

5.7.2. Sinusoidal Sensor Noise: We also tested Fig. 25. RMMAC robustness to the sinusoidal “robustness to the assumptions” by using a high sensor noise. frequency sinusoid for the measurement noise, as θ(t) = 10−3 sin(10t), rather than pure white eters. In all the numerical simulations presented noise. Fig. 25 shows a representative simulation up to now in this section, we constrained the uncertain parameter to remain constant for all using the value k1 = 0.3, which is “close” to Model #4. Fig. 25(a) shows that the “correct” time. Of course, the presence of a time-varying spring stiﬀness violates the plant LTI assumption, probability P4(t) 1 within 5 secs. The improvement in disturbance-rejection→ is obvious from and hence all stationarity and ergodicity assump- Fig. 25(b). Fig. 25(c) shows a comparison of the tions required to prove the posterior probability control signals. Both GNARC and RMMAC con- convergence results do not hold. Nevertheless, it is trol signals have sinusoidal components at steady- important to understand, for any adaptive system, state. Nonetheless, as expected, the amplitude of its behavior and performance in the presence of the RMMAC control is slightly larger than that slow parameter variations. of the GNARC (this comes as no surprise) since In the following numerical MC simulations, the RMMAC yields improved disturbance-rejection. uncertain spring stiﬀness is assumed to be sinu- Only one MC run is shown. soidal with frequency 0.01 rad/sec, i.e. −2 k1(t) = 1 0.75 cos 10 t (5.19) 5.7.3. Slow Parameter variation: As mentioned − in Section 1, the driving engineering motivation as shown in Fig. 26(a). The dashed-lines indicate for using adaptive control was the need to deal the times that the spring crosses the boundaries with “slow” changes in the plant uncertain param- of the four models of Table 2. Fig. 26(b) shows the 5.8 Discussion of Numerical Results 1.75

M#1 The stochastic simulation results presented in this paper demonstrate the excellent performance of 1.02 M#2 the RMMAC system. Compared to the “best” 0.64 non-adaptive design, GNARC, the RMMAC con- M#3 sistently had superior disturbance-rejection. No M#4 0.40 0.25 0 200 400 600 800 1000 closed-loop instabilities were noted in thousands t (sec) of MC simulations. (a) Sinusoidal spring stiﬀness, k1(t) The example illustrated how to predict (and then

1 validate) the potential performance characteris-

(t) tics of the RMMAC. This was done by analyzing 1 0.5 P the corresponding LTI feedback loops involving 0 1 the GNARC and each of the four LNARCs so as

(t) to generate RMS predictions, frequency-domain

2 0.5 P visualizations etc. We emphasize that this con- 0 structive capability is critical so that quantitative 1 tradeoﬀs can be carried out, before one constructs (t)

3 0.5 P and tests the full-blown RMMAC design with 0 numerous MC simulations. 1 (t)

4 0.5 The RMMAC performance conclusions reported P

0 here validate earlier simulations by the authors. 0 200 400 600 800 1000 time (sec) In (Fekri et al., 2004a) we analyzed, using a ﬁve- model RMMAC, an academic SISO non-minimum (b) Posterior probabilities transients phase (NMP) plant with a single uncertain right-

0.15 half plane zero which poses fundamental limits to GNARC superior disturbance-rejection near its frequency; 0.1 RMMAC see, for example, (Aström,˚ 2000). In that example, the GNARC “hedged” for the minimum value 0.05 of the NMP zero. If the true zero was near its z(t) 0 minimum value, the RMMAC performance was essentially the same as for the GNARC (no magic −0.05 properties). The true value of the RMMAC shows when the true NMP zero is large; then the RM- −0.1 0 200 400 t (sec) 600 800 1000 MAC produces truly superior performance vis-à- (c) Output response vis the GNARC. The interested reader may also review the RMMAC results reported in (Fekri et Fig. 26. RMMAC responses for the sinusoidal al., 2004c) for a different two-cart mass-spring- parameter variations, as in (5.19). dashpot non-collocated system with a mass uncertainty. However, the previous results (Fekri et al., 2004a) and (Fekri et al., 2004c) did not dynamic evolution of the four posterior probabil- use the systematic FNARC based approach for ities, while Fig. 26(c) compares the GNARC and defining the models (in the spirit of Fig. 13) nor RMMAC performance output responses, which the optimized KF design using the BPM (in the shows that the RMMAC continues to work quite spirit of Fig. 17). Several other SISO and MIMO well. results and examples can be found in the doctoral It is tempting to interpret Fig. 26(b) as demon- dissertation of S. Fekri (Fekri, 2005). strating a “transient” in the identification process. This may well be true, but the reader should realize that the signals generated by the time- 6. RESEARCH NEEDS varying plant cannot be interpreted using “frozen model” reasoning. After all changing the spring We have presented a brief overview of different ap- stiffness according to (5.19) implies an exogenous proaches to adaptive control that utilize multiple- energy transfer as a function of time, which is not model architectures. Recent research in the area accounted for in a “frozen” model. This opens of the CMMAC and RMMAC architectures seems up avenues for future research. Nonetheless, the most promising, but we still have no widely ac- results of Fig. 26 (and many others not presented cepted solution to the robust adaptive control herein) demonstrate the ability of the RMMAC in problem. As a consequence, there is still a great dealing with slowly-varying uncertain parameters. deal of work that needs to be done, both theoretical and applied, to fully understand the similar- generated by a bank of multi-estimators, which ities and difference among the two architectures are a special class of Luenberger observers. In and the variants within each one, and suitable all cited SMMAC references, with the exception extensions. of (Corradini et al., 2004), each estimator uses the same “A ” matrix, i.e. all observers have Any future theoretical research must adopt a de- E identical poles. This was done to minimize on- sign methodology. We strongly believe that the line computation. An integral norm of the pre- adaptive problem formulation should always in- diction errors is used for the system identification clude unmodeled dynamics, unknown plant dis- function. Research should be done to remove the turbances and sensor noise, in addition to the common “A ” matrix assumption and use more (slowly-varying) uncertain real parameters. Fur- E general observers. Also, the assumption of scalar thermore, the design methodology should contain prediction errors must be removed to make the explicit performance specifications and performance- SMMAC architectures extendable to the MIMO robustness should be an integral part of the case. In (Corradini et al., 2004) this was done, but adaptive problem formulation. Thus, closed loop- the problem of designing suitable deterministic stability arguments are not enough. This im- MIMO Luenberger observers, say by eigenstruc- plies that the “local” compensators, the LNARCs, ture assignment, becomes quite complex. must be designed using mixed µ-synthesis, following the ideas presented in Section 3. Finally, we need to stress that in all available SMMAC and RMMAC results we cannot prove – The structure of the multi-controllers used in the as yet – global robust-stability because a suitable SMMAC architecture does not reflect a robust- theory is lacking for handling the nonlinear time- performance requirement. Indeed, for the most varying (stochastic) closed loop dynamics arising part, the SMMAC multi-controllers have a com- from these architectures. Needless to say, any mon A -matrix (so all controllers have the same c advances in stability theory along these directions poles). This was done for the purpose of compu- would be welcomed. At present, the promise of tational simplicity and “bump less transfer” to the SMMAC and RMMAC architectures is in avoid control transients. However, such controller their “local” and “asymptotic” results related to structures are not appropriate if performance- stability and performance. Equally important to robustness is desired. A recent exception (Corra- the theoretical research investigations would be dini et al., 2004) uses multi-controllers that have to fairly compare the different methods using an LQG structure, but they use pole-placement a variety of common “test-bed examples”, with ideas, which are also notorious for their lack-of- identical performance requirements. robustness properties. On the other hand, the RMMAC architecture deals directly with such robust-performance issues and the dynamic characteristics of the RMMAC compensators are quite 7. CONCLUSIONS different. We have witnessed the development and evolu- We believe that, as stressed in Section 3, one can- tion of a variety of novel multiple-model architec- not arrive at a systematic procedure for defining tures for robust adaptive control during the past the number of models required in any multiple- decade. In order to compare them in a fair manner model scheme without explicit performance spec- we have suggested that all should utilize the avail- ifications. Specification of complexity requires fix- able results and software associated with mixed-µ ing the number, N, of models; however, one still synthesis. If we explicitly state the specifications needs to properly calculate their “boundaries.” for required stability-robustness and performance- Specification of the required performance, such robustness for the adaptive closed-loop system, as in the % FNARC method, would naturally then we can evolve the present architectures so lead to the required number of models and their that they can be confidently used in real applica- boundaries. In either case, one needs to utilize the tions. mixed-µ methodology to maximize performance and derive the LNARCs. We now consider the “system identification” part ACKNOWLEDGEMENTS of the SMMAC and RMMAC architectures. In the RMMAC architecture the approach is stochastic We thank Profs. B.D.O. Anderson and J. Hes- and relies on the residuals generated by the bank panha for their suggestions, discussions and criti- of KFs. These must be designed very carefully so cisms. We also are grateful to Prof. G.J. Balas for as to meet the probability convergence results, providing us with the latest version of the mixed- as explained in Section 4 and Appendices I and µ software (Balas, 2003) and Prof. Y. Baram for II. In the SMMAC architectures the approach is his help to provide us a copy of his Ph.D. the- deterministic and relies upon the prediction errors sis (Baram, 1976). Appendix I. The MMAE filter shown in Fig. 1 is driven by the SUMMARY OF THE MMAE FILTER sequence of past controls and noisy sensor measurements while generates both a state-estimate In this appendix, the equations of the multiple vector and a corresponding error-covariance ma- model adaptive estimation (MMAE) algorithm, as trix. Let us first establish some notation for the depicted in Fig. 1, and some of its properties are discrete-time steady-state Kalman filter (KF). summarized. Here we shall restrict our attention Predict-cycle: to linear systems driven by white Gaussian inputs having time-invariant statistic. It is also assumed xˆk(n + 1 n) = Fkxˆk(n) + Jku(n) | (A.7) that the system has attained steady-state, i.e. that yˆk(n + 1 n) = Hkxˆk(n + 1 n) all signals of interest are stationary. | | Update-cycle: Consider the discrete-time (unknown) system xˆ (n + 1 n + 1) =x ˆ (n + 1 n) + K r (n + 1) k | k | k k x(n + 1) = F∗x(n) + J∗u(n) + G∗w(n) (A.8) (A.1) y(n) = H∗x(n) + v(n) The residual r ( ), residual covariance matrix S , k · k where u(n) is the deterministic (control) in- and the constant steady-state KF gain matrix, { } put and w(n) and v(n) are zero-mean white Kk, are respectively defined as follows. { } { } Gaussian sequences, mutually uncorrelated and Residual: uncorrelated with x(0) with r (n + 1) = y(n + 1) yˆ (n + 1 n) (A.9) T k k E w(n)w(n) = Q∗ − | { } T (A.2) Residual covariance: E v(n)v(n) = R∗ { } S = cov[r (n + 1); r (n + 1)] = H ΣpHT + R Let us assume that N discrete-time stochastic k k k k k k k (A.10) LTI models are given that include disturbance KF gain: dynamics (if any). These models blend a finite set − K = ΣpHT S 1 (A.11) of families of models k k k k The constant steady-state covariance equations are ℘ = : k = (Fk,Jk, Gk, Hk, Qk,Rk) ; p {M M Predict-cycle covariance Σk : k = (1, 2,...,N) (A.3) p T T } Σk = FkΣkAk + GkQkGk (A.12)

The k-th model is then described by the state Update-cycle covariance Σk : − space dynamics Σ = Σp ΣpHT S 1H Σp (A.13) k k − k k k k k x (n + 1) = F x (n) + J u(n) + G w(n) k k k k k (A.4) We stress that both the MMAE state-estimate (A.5) yk(n) = Hkxk(n) + v(n) and state-covariance matrix (A.6) represent true conditional state estimate and its conditional where the time index is n = 0, 1, 2,... and the covariance at steady-state (Athans and Chang, model index is k = 1, 2,...,N. 1976; Anderson and Moore, 1979). This is be- It is assumed that the true system that generates cause, one can explicitly calculate the conditional the data is one of the models in (A.4). probability density function p (x(n) Y (n)) , which turns out to be a weighted sum of gaussian| densi- The set of past controls, u(0),u(1),u(2),...,u(n − ties, where the weights are found from the poste- 1), and the set of past noisy measurements, in- rior probability evaluator (PPE); see Fig. 1. cluding the one at the present time n, y(1),y(2), ...,y(n 1),y(n), are also known at time n. We The problem reduces to a combination of a want to− determine the true steady-state condi- hypothesis-testing problem and a state-estimation tional mean of the present state vector, x(n), i.e. problem. The fact that one of the N models is as n the true one is modeled by a hypothesis random → ∞ variable that must belong to a discrete set of hypothesis , ,..., . It turns out that on- xˆ(n n) = E x(n) (A.5) {H1 H2 HN } | | line generation of the posterior conditional prob- u(0),u(1),...,u(n 1); y(1),...,y(n 1),y(n) abilities determines which hypothesis is true. Let © − − suppose that H indicates the hypothesis random Y (n) ª variable (scalar) which can attain only one of N | {z } and the true steady-state conditional covariance possible values, matrix of x(n), i.e. 1, 2,..., N (A.14) 0 H∈{H H H } Σ(n n) = E x(n) xˆ(n n) x(n) xˆ(n n) Y (n) The event = k means that the k-th system | − | − | | (A.6) is the trueH one,H i.e. the one that is generating £¡ ¢¡ ¢ ¤ the data. The prior probabilities at initial time and the exact conditional covariance matrix n = 0, Pk(0) Prob( = k), are assumed ≡ H 1 H N known (typically Pk(0) = N ), and Σ(n n) = Pk(n) Σk(n n) + N | · | k=1 Pk(0) 0, Pk(0) = 1 (A.15) X £ 0 ≥ xˆk(n n) xˆ(n n) xˆk(n n) xˆ(n n) (A.25) kX=1 | − | | − | The posterior probabilities, Pk(n) = Prob( = ¡ ¢¡ ¢ ¤ H A brief review on Multiple-Model Adaptive Es- k Y (n)), must also satisfy H | timation (MMAE) is given that was restricted N to LTI systems driven by white Gaussian inputs P (n) 0, P (n) = 1 (A.16) k ≥ k having time-invariant statistics. Again it is noted kX=1 that the above results hold in the sense that the and can be calculated on-line by the PPE in true (unknown) plant is assumed to belong to the the MMAE system. It turns out that the condi- model set of (A.4). tional PDF, p (y(n + 1) u(n), k,Y (n)) , is Gaus- sian with mean | H

E y(n + 1) u(n), k,Y (n) = Hk xˆk(n + 1 n) Appendix II. { | H } (A.17)| BARAM PROXIMITY MEASURE (BPM) and steady-state covariance The main purpose of this appendix is to summarize the underlying results from probability cov [y(n + 1); y(n + 1) u(n), ,Y (n)] = | Hk and estimation theory that yield the identiﬁca- p T , HkΣkHk + Rk Sk (A.18) tion procedure which converges to a model in the set which is closest to the true model in an Furthermore, information metric sense, as proved by (Baram, 1976). The posterior probabilities will converge p (y(n + 1) u(n), ,Y (n)) = | Hk if certain ergodicity and stationarity assumptions −1 1 − 1 rT (n+1) S r (n+1) e 2 k k k (A.19) of residuals (innovation signals) are true. These m/2 (2π) √det Sk · conditions are brieﬂy discussed in this appendix; By using Bayes rule we deduce that see also (Baram, 1976) for more details.

Pk(n + 1) = II.1 Stationarity and Ergodicity

p y(n + 1) k,u(n),Y (n) |H Pk(n) The purpose of this section is to provide def- N · initions and convergence results for ergodic se- Pj¡(n)p (y(n + 1) j ,u(n),Y¢ (n)) j=1 |H quences used in the paper. It is not intended to P (A.20) provide an elaborate presentation of the concept of ergodicity. For a precise development of ergod- For notational simplicity, define icity theory the reader is referred to, e.g. (Doob, 1953; Halmos, 1956). 0 −1 wj (n + 1) = rj (n + 1) Sj rj(n + 1) (A.21) − m/2 1 βj = (2π) det Sj (A.22) II.2 Definitions: where m ¡is the numberp of¢ measurements. 1. Consider a probability space (Ω,U,P ). A trans- Then, from (A.19)–(A.22) the posterior probabil- formation T from Ω to U is said to be measure ities can be computed on-line by the PPE using preserving if the recursive formula −1 1 P (T A) = P (A) − wk(n+1) βk e 2 Pk(n + 1) = Pk(n) for all A U. N 1 · ∈ − wj (n+1) βj e 2 Pj(n) 2. A stochastic sequence x(n) on (Ω,U,P ) is j=1 said to converge almost everywhere{ } (a.e.) or al- P (A.23) most surely (a.s.) to a random variable x on where P (0) are the prior model probabilities as k (Ω,U,P ) if in (A.15). lim x(n) = x a.e. Thus, at steady-state the MMAE generates the n→∞ state estimate (exact conditional mean) by 3. Given a measure preserving transformation T, N a U-measurable event A is said to be invariant if xˆ(n n) = Pk(n)x ˆk(n n) (A.24) − | | T 1A = A kX=1 ∗ 4. Let x(n) be a stochastic sequence on (Ω,U,P ) controllable. For each j , S is generated ∈ M j with values{ } in (R`,B`), where (R`) is the l- by the discrete Kalman filter #j corresponding to dimensional Euclidean space and (B`) is the σ the model = (F , G , H , Q ,R ). Mj j j j j j algebra of Borel sets of Rl; see (Gray, 2001, ` Assuming that index ’i’ denotes the true param- Chap. 3). Let B be the σ-algebra of Borel sets ∗ ∞ eter in , the dynamic equation generating of R` where R` = R` R` . Then x(n) ∞ ∞ simultaneouslyM the state x(n) and its updated is said to be stationary ×if for× each · · · κ 1 and{ for} ` ≥ estimation by the j-th Kalman filter,x ˆj(n), is every C B∞ ∈ P [ x(1),x(2),...,x(n) C] = x (n + 1) F 0 x (n) { } ∈ (B.1) i = i i P [ x(κ + 1),x(κ + 2),...,x(κ + n) C] xˆ (n + 1) F K H F (I K H ) xˆ (n) { } ∈ · j ¸ · j j i j − j j ¸ · j ¸ G 0 w(n) + i (B.4) 5. The covariance matrix defined by 0 F K v(n) · j j ¸ · ¸ R(κ) E x(n) xT (n + κ) ≡ { } where K is the (steady-state) Kalman filter (KF) plays a fundamental role in the study of stationary j gain corresponding to the model . The KF gain processes in the wide sense (Doob, 1953). A zero j matrix is given by M mean stationary Gaussian process is ergodic if − and only if (Gernander, 1959, pp.257–260); (Doob, K = Σ HT (H Σ HT + R ) 1 1953, pp.494) j j j j j j j n where Σj denote the prediction error covariance 1 2 lim (κ) = 0 (B.2) matrix according to M . n→∞ n + 1 |R | j κ=0 X For notational purposes, the following are used. where (κ) denotes the determinant of (κ). |R | R i Fi 0 Fj ≡ FjKjHi Fj(I Kj Hj) II.3 Calculating the BPM · − ¸ i Gi 0 Gj ≡ 0 FjKj Based on a bank of N discrete Kalman filters, a · ¸ i Qi 0 set of N models for the system of (A.1) can be Q ≡ 0 Ri · ¸ described by Hi H H j ≡ i | − j xj(n + 1) = Fjxj(n) + Gjw(n) (B.3) Then the matrix£ ¤ yj(n) = Hj xj(n) + v(n) that is in the absence of deterministic inputs but T i xi(n + 1) xi(n + 1) with the assumption of a stationary process. Note Ψj(n + 1) E ≡ xˆj(n + 1) xˆj(n + 1) that these results can be extended to the case (· ¸ · ¸ ) where the system (B.3) is driven by an additional is generated by the Lyapunov equation deterministic (known) input sequence. ∗ i i i iT i i i T Let = ℘ denotes the model set that also Ψj(n + 1) = Fj Ψj(n)Fj + Gj Q Gj (B.5) includesM the{∗∪ true} model, denoted by . ∗ The (steady-state) limit of the Lyapunov equation For each i, j ℘ let ∈ of (B.5) is T i i S E [y(n) yˆ (n)][y(n) yˆ (n)] = Ψj = lim Ψj(n) (B.6) j ≡ − j − j |Hi Hj n→∞ denote the© residual covariance matrix generatedª by the j-th Kalman filter and exists and is finite if and only if the augmented i state matrix Fj has all its eigenvalues inside the Γi E [y(n) yˆ (n)][y(n) yˆ (n)]T = j ≡ − j − j |Hi 6 Hj unit circle, i.e. the spectral radius satisfies denote the residual covariance matrix according i © ª ρ(Fj ) < 1 (B.7) to , when is the correct model. Mj Mi This is the case if for each j ∗, F has all its ∈ M j We shall use the following condition (Baram, eigenvalues inside the unit circle and (Fj , Hj ) is 1976, pp.77) in addition to the other stationarity observable. and ergodicity conditions. Therefore, to calculate the limit matrix Ψi of ∗ j Condition B1 (C.B1). For each j , the (B.5) we simply solve, using Matlab, the discrete- ∈ M residual covariance Sj exists and has a finite time algebraic Lyapunov equation positive definite value. i i i iT i i i T Ψj = Fj ΨjFj + Gj Q Gj (B.8) A sufficient condition for (C.B1) is that each model corresponding to j ∗ is detectable and It can be shown that ∈ M T Γi = HiΨi Hi + R (B.9) The Baram proximity measure (BMP) of the j-th j j j j i i filter denoted by Lj is generated by i where Ψj is the steady-state solution of (B.5). − ∗ Li log S + tr S 1Γi ; i, j (B.14) It can also be shown that the state covariance j ≡ | j | j j ∈ M matrix can be calculated as follows. © ª Therefore, i i iT i Hj ΨjHj + Ri = Γj ; κ = 0 1 ∗ ∗ κ ¯ (κ) = i i iT iT (B.10) In(k; j) = Lj Lk ; i, j (B.15) R ( Hj Ψj Fj Hj ; κ > 0 2 − ∈ M ³ ´ ¡ ¢ Also, This is shown in (Baram, 1976, pp.80–81). How- ever, note that the stability and observability of I¯n( ; j) 0 ; for each j (B.16) the models are only sufficient, and not necessary. ∗ ≥ ∈ M In fact, (Baram, 1976, Theorem 5.1) has proved It is easy to check that ergodicity of the state residuals x(n) xˆj(n) , which is not necessary, to show{ the ergodicity− } d( ; j) d( ; k) of y(n) yˆ (n) . Thus, in the sequel we shall ∗ ≥ ∗ { − j } directly use the following assumption. if and only if Condition B2 (C.B2). For each j ∗ the ∗ ∗ Lj Lk residual sequence y(n) yˆ (n) is ergodic.∈ M ≥ { − j } The ergodicity condition of (C.B2) is important in which the following condition is essential. and must be checked. If it is not satisfied, one Condition B3 (C.B3). There exists some parame- might use a “fake-white-plant-noise” as mentioned ter k such that in Section 2.4; see (Grewal et al., 2001, Chap. 7) ∈ M ∗ ∗ for more details. L < L for all j ; j = k (B.17) k j ∈ M 6 The conditional probability density function corresponding to Model #j (j ∗) is If the convergence conditions hold for the MMAE ∈ M to converge, i.e. (B.2) and (C.B1)–(C.B3), then it − 1 will converge to the j-th filter governed by p (y(n) Y (n 1)) = [(2π)m S ] 2 j | − | j | · 1 T −1 ∗ i exp [y(n) yˆ (n)] S [y(n) yˆ (n)] Lj =min Lj ; i = 1,...,N {−2 − j j − j } i { } (B.11) i where Lj is the BMP of the models as in (B.14). where m is the dimension of y(n), i.e. the number of measurements. In conclusion, the Baram proximity measure (BPM), can be an appropriate distance metric For each j, k ∗ the mean information in y(n) ∈ M in the stochastic systems between the true model favoring k against j is defined by (Baram, 78a, b) and each of the adopted models utilizing the corresponding bank of Kalman filters. pk y(n) Y (n 1), = k I¯n(k; j) = E log | − H H p y(n) Y (n 1), = j ¡ | − H Hj ¢ and the distance measure¡ is ¢ II.4 Remarks d (k; j) = I¯ (k; j) n n The application of deterministic inputs to dy- ∗ ¯ ¯ namic systems for the purpose of identification For each j , using ¯to denote¯ the true plant in ∗, we∈ have M ∗ and their optimal selection, covering the reference M tracking problems, is not addressed in this paper. Generally speaking, the convergence analysis E log p y(n) Y (n 1), = = j | − H Hj in the presence of deterministic inputs requires m 1 1 −1 ∗ a more elaborate analysis that is suggested for log¡ 2π log Sj tr S¢ Γ (B.12) − 2 − 2 | | − 2 j j future research. © ª and for each pair j, k ∗ (Baram, 1976, Chap. 6) discusses a case that any ∈ M deterministic input sequence, that satisfies a suit-

1 1 − ∗ able assumption of (Baram, 1976, Theorem 6.3), I¯ (k; j) = I¯ (j; k) = log S + tr S 1Γ n n 2 | j| 2 j j − will provide convergence in the mean of the identi- 1 1 − ∗ © ª fication procedures at a certain rate. see (Baram, log S tr S 1Γ (B.13) 2 | k| − 2 k k 1976) and (Yared, 1979) for more details. © ª (Ljung, 1976) uses the prediction error parameter Athans, M. and C.B. Chang (1976). Adaptive estimate obtained by minimizing a scalar function estimation and parameter identification using of the matrix multiple model estimation algorithms. Techni- t cal Report 1976-28. MIT Lincoln Lab., Lexing- [R1/2(τ) r(τ; p)][R1/2(τ) r(τ; p)]T (B.18) ton, MA, USA. τ=0 X Balas, G.J. (2003). Private communication re mixed-µ software. where R(τ) is some positive definite weighting ma- Balas, G.J. et al. (2004). µ-Analysis and Synthesis trix, and shows that under general conditions of Toolbox For Use with Matlab, Users Guide. bounded fourth moments of the residuals r(t; p), 4th ed., The MathWorks Inc. by searching over models leading to stable Kalman Baram, Y. (1976). Information, consistent estima- filters and overall system stability, this prediction tion and dynamic system identification. PhD error estimate converges into the set of models thesis. MIT. Cambridge, MA, USA. that give the same output prediction as the true Baram, Y. and N.R. Sandell (1978a). Consistent system in the following sense. estimation on finite parameter sets with ap- t 1 2 plication to linear systems identification. IEEE lim inf yˆ(τ; p0) yˆ(τ; p) = 0 (B.19) t→∞ t + 1 | − | Trans. on Automatic Control 23(3), 451–454. τ=0 X Baram, Y. and N.R. Sandell (1978b). An informa- which is similar to the ergodicity condition (B.2). tion theoretic approach to dynamical systems This set depends in general on the input signal modeling and identification. IEEE Trans. on and will be contained in the set of all models Automatic Control 23(1), 61–66. with same input-output relation as true model, Barros, D. et al. (2005). Robust mixed-µ synthesis if the input is general enough to excite all modes performance for a mass-spring benchmark sys- th of the system. It is shown in (Ljung, 1976) that tem with stiffness uncertainty. Proc. 13 IEEE a sufficient condition would be for u(t) to be Mediterranean Conference on Control and Au- independent of the process noise and persistently tomation (MED’05). Limassol, Cyprus. exciting. The input u(t) is persistently exciting if, Chang, C. and M. Athans (1978). State estimation for discrete systems with switching parameters. for all M, there exists δ(M) and N0(M) such that IEEE Trans. on Aerospace and Electronic Sys- N 1 0 1 tem 14(3), 418–425. δI < u (t)u (t) < I N M M δ Corradini, M.L. et al. (2004). Robust stabilization 1 X of multivariable uncertain plants via switching for N > N and where 0 control. IEEE Trans. on Automatic Control 0 0 0 u (t) [u (t) ...u (t M)]. 49(1), 107– 114. M ≡ − Doob, J.L. (1953). Stochastic Processes. John Wi- ley & Sons, Inc., New York. REFERENCES Fekri, S. et al. (2004a). A new robust adap- Anderson, B.D.O. and J.B. Moore (1979). Opti- tive control method using multiple-models. mal Filtering. Prentice Hall. Proc. 12th IEEE Mediterranean Conference on Anderson, B.D.O. and J.B. Moore (1990). Opti- Control and Automation (MED’04). Kusadasi, mal Control - Linear Quadratic Methods. Pren- Turkey. tice Hall. NJ, USA. Fekri, S. et al. (2004b). RMMAC: A novel robust Anderson, B.D.O. et al. (2000). Multiple model adaptive control scheme – Part I: Architecture. adaptive control. part 1: Finite controller cov- Proc. of the IEEE Conf. on Decision and Con- erings. Int. J. of Robust and Nonlinear Control trol. Paradise Island, Bahamas. 10(11-12), 909–929. Fekri, S. et al. (2004c). RMMAC: A novel robust Anderson, B.D.O. et al. (2001). Multiple model adaptive control scheme – Part II: Performance adaptive control with safe switching. Int. J. evaluation. Proc. of the IEEE Conf. on Decision of Adaptive Control and Signal Processing and Control. Paradise Island, Bahamas. 15(5), 445–470. Fekri, S. (2005). Robust adaptive MIMO control Aström,˚ K.J. (2000). Limitations on control sys- using multiple-model hypothesis testing and tem performance. European Journal of Control mixed-µ synthesis. PhD thesis. Instituto Supe- 6(1), 1–19. rior Técnico. Lisbon, Portugal. Aström,˚ K.J. and B. Wittenmark (1995). Adaptive Gelb, A. (1974). Applied Optimal Estimation. Control. 2nd ed., Addison-Wesley. MA, USA. MIT Press. MA, USA. Athans, M. et al. (1977). The stochastic control Gernander, U. (1959). Stochastic Processes and of the F-8C aircraft using the multiple model Statistical Inference. Vol. 1. Arkiv fürMathe- adaptive control (MMAC) method – part I: matic. Equilibrium flight. IEEE Trans. on Automatic Gray, R.M. (2001). Probability, Random Pro- Control 22(5), 768–780. cesses, and Ergodic Properties. Springer Verlag. Greene, C.S. (1975). An analysis of the multiple cations, (D. Normand-Cyrot, ed.). Springer- model adapative control algorithm. PhD thesis. Verlag. Berlin. MIT. Cambridge, MA, USA. Morse, A.S. (2004). Analysis of a supervised set- Greene, C.S. and A.S. Willsky (1980). An analysis point control containing a compact continuum of the multiple model adaptive control algo- of finite-dimensional linear controllers. Proc. rithm. Proc. of the IEEE Conf. on Decision and MTNS. Belgium. Control. pp. 1142–1145. Narendra, K.S. and A. Annaswamy (1988). Stable Grewal, M.S. et al. (2001). Global Positioning Adaptive Systems. Prentice Hall. NJ, USA. Systems, Inertial Navigation, and Integration. Narendra, K.S. and J. Balakrishnan (1997). Adap- John Wiley & Sons, Inc. tive control using multiple models. IEEE Trans. Halmos, P.R. (1956). Lectures on Ergodic Theory. on Automatic Control 42(2), 171–187. Chelsea Publ. Co., New York. Rohrs, C.E. et al. (1985). Robustness of Hawkes, R.M. and J. Moore (1976). Performance continuous-time adaptive control algorithms in of bayesian parameter estimators for linear sig- the presence of unmodeled dynamics. IEEE nal models. IEEE Trans. on Automatic Control Trans. on Automatic Control 30(9), 881–889. 21(4), 523–527. Safonov, M.G. and T.-C. Tsao (1995). The Unfal- Hespanha, J.P. et al. (2001). Multiple model adap- sified Control Concept: A Direct Path From Ex- tive control. part 2: switching. Int. J. of Robust periment To Controller. pp. 196–214. in Feed- and Nonlinear Control 11(5), 479–496. back Control, Nonlinear Systems, and Complex- Ioannou, P. and J. Sun (1996). Robust Adaptive ity, (B.A. Francis and A.R. Tannenbaum, eds.). Control. Prentice Hall. NJ, USA. Springer-Verlag. Jun, M. and M.G. Safonov (1999). Automatic Safonov, M.G. and T.-C. Tsao (1997). The un- PID tuning: An application of unfalsified con- falsified control concept and learning. IEEE trol. Proc. IEEE CCA/CACSD. pp. 328–333. Trans. on Automatic Control 42(6), 843–847. Kosut, R.L. and B.D.O. Anderson (1988). Adap- Sastry, S. and M. Bodson (1989). Adaptive Con- tive control via finite modelling and robust trol: Stability, Convergence and Robustness. control. IFAC Workshop on Robust Adaptive Prentice-Hall. NJ, USA. Control pp. 91–95. Schiller, G. and P.S. Maybeck (1997). Control of Kwakernaak, H. and R. Sivan (1972). Linear Op- a large space structure using MMAE/MMAC timal Control Systems. John Wiley. NY, USA. techniques. IEEE Trans. on Aerospace and Lainiotis, D.G. (1971). Optimal adaptive es- Electronic System 33(4), 1122–1130. timation: Structure and parameter adapta- Schott, K.D. and B.W. Bequette (1997). Multi- tion. IEEE Trans. on Automatic Control AC- ple Model Adaptive Control. pp. 269–292. in 16(2), 160–170. Multiple Model Approaches to Modelling and Landau, I.D. (1979). Adaptive Control: The Model Control, (R. Murray-Smith and T.A. Johnsen Reference Approach. Marcel Dekker. NY, USA. eds.). Taylor & Francis. London. Ljung, L. (1976). On the Consistency of Predic- Shomber, H.R. (1980). An extended analysis of tion Error Identification Methods. pp. 121–164. the multiple model adaptive control algorithm. in System Identification: Advances and Case PhD thesis. MIT. Cambridge, MA, USA. Studies, (R.K. Mehra and D.G. Lainiotis, eds.). Willner, D. (1973). Observation and control of Academic Press. New York. partially unknown systems. PhD thesis. MIT. Magill, D.T. (1965). Optimal adaptive estimation Cambridge, MA, USA. of sampled stochastic processes. IEEE Trans. Yared, K.I. (1979). Maximum likelihood identifi- on Automatic Control AC-10(4), 434–439. cation of state space models for linear dynamic Maybeck, P.S. and J. Griffin (1997). MMAE/ systems. PhD thesis. MIT. Cambridge, MA, MMAC control for bending with multiple un- USA. certain parameters. IEEE Trans. on Aerospace Young, P.M. et al. (1992). Practical computation and Electronic System 33(3), 903–912. of the mixed-µ problem. Proc. of the American Morse, A.S. (1996). Supervisory control of families Control Conf.. Chicago, IL. pp. 2190–2194. of linear set-point controllers – part 1: Exact Young, P.M. et al. (1995). Computing bounds for matching. IEEE Trans. on Automatic Control the mixed-µ problem. Int. J. of Robust and 41(10), 1413–1431. Nonlinear Control 5, 573–590. Morse, A.S. (1997). Supervisory control of fam- Young, P.M. (1994). Controller design with mixed ilies of linear set-point controllers – part 2: uncertainties. Proc. of the American Control Robustness. IEEE Trans. on Automatic Control Conf.. Baltimore, MD. pp. 2333–2337. 42(11), 1500–1515. Zhou, K. et al. (1996). Robust and Optimal Con- Morse, A.S. (1998). A bound for the distur- trol. Prentice-Hall. NJ, USA. bance - to - tracking - error gain of a su- Zhou, K. (1998). Essential of Robust Control. pervised set point control system. pp. 23–41. Prentice-Hall. NJ, USA. in Perspectives in Control Theory and Appli-