Structured Stochastic

Bassam Bamieh Department of Mechanical University of California at Santa Barbara Santa Barbara, CA, 93106 [email protected]

Abstract— We consider linear time invariant systems in over the time interval Z+. This is done in contrast with the feedback with structured stochastic . This setting standard setting over Z since stability arguments involve the encompasses linear systems with both additive and multiplica- growth of signals starting from some initial time. tive noise. We provide a purely input-output treatment of these systems without recourse to state space models, and For any random variable (or vector) v, we use := v⇤v to denote its variance, and ⌃ := vv⇤ thus our results are applicable to certain classes of distributed v E{ } v E{ } systems. We derive necessary and sufficient conditions for mean to denote its covariance matrix. A u is + square stability in terms of the spectral radius of a linear a one-sided sequence of random variables uk; k Z . matrix operator whose dimension is that of the number of { 2 } We will thus denote by := u⇤u the sequence uncertainties, rather than the dimension of any underlying state uk E{ k k} of its variances, and by ⌃ = u u⇤ the sequence space models. Our condition includes the case of correlated uk E{ k k} uncertainties, and reproduces earlier results for uncorrelated of its inter-component correlation matrices. A process u is uncertainties. termed second order if it has finite covariances ⌃uk for each k Z+. Although the processes we consider are 2 I. INTRODUCTION technically not stationary (stationary processes are defined The setting we consider is that of a discrete-time Linear over the doubly infinite time axis), it can be shown that they Time Invariant (LTI) system in feedback with gains , are asymptotically stationary in the sense that their G 1 ..., n (see Figure 3). These gains are random processes that become approximately stationary in the limit of large time, are temporally independent, but possibly mutually correlated. or quasi-stationary in the terminology of [13]. This fact is The setting of LTI systems in feedback with structured not used in our treatment here and the preceding comment uncertainties is common in the robust controls literature is only included for clarification. where the uncertainties are typically norm-bounded opera- tors, real or complex deterministic gains [1]–[8]. The setting A. Input-output definition of mean square stability where the uncertainties are stochastic has been relatively less Let be a linear time invariant (MIMO) system. The studied [9]–[12], but it is well known that the necessary G system is completely characterized by its impulse response and sufficient condition for mean square stability in the G + which is a matrix valued sequence Gk; k Z . The presence of structured stochastic uncertainties is a bound { 2 } action of on an input signal u to produce an output signal on the spectral radius of a matrix of H2 norms of all the G y is given by the convolution sum subsystems of . G Our aim is to provide a rather elementary and purely k input-output treatment and derivation of the necessary and yk = Gk l ul. (1) sufficient condition for mean square stability. In the process, l=0 X we define a new object, a linear matrix operator, which If the input u is a second order stochastic process, then captures how a feedback system amplifies covariances of it is clear from (1) that y has finite variance for any k, signals in a loop. A pleasant side effect is that the conditions k even in the cases where this variance may be unbouded in in the case of correlated uncertainties (which have been time. This leads to the following input-output definition of unknown) are almost as easy to state as the ones for un- Mean-Square Stability. correlated uncertainties. Those earlier results on uncorrelated Definition 1: The linear time invariant system is called uncertainties are easy to reproduce from the conditions we G provide. Mean-Square Stable (MSS) if for each second order white input process u with uniformly bounded variance, the cor- II. PRELIMINARIES responding output process y = u has uniformly bounded G variance All the signals we consider are defined on the half-infinite, discrete-time interval Z+ =[0, ) Z. The dynamical 1 ⇢ y := y⇤yk M sup u , (2) systems we consider are maps between various signal spaces k E{ k }  k ✓ k ◆ Work supported by AFOSR award AFOSR FA9550-10-1-0143 where M is a constant independent of k and the process u. In this paper we deal exclusively with this kind of stability, where is the Kroneker delta function, and ⌃uk is the time and we therefore refer to MSS stable systems as simply varying, inter-component correlation matrix of uk. By think- stable. ing of the vector component index as a “spatial” variable, A standard calculation shows that we refer to such signals as temporally white and spatially k correlated. 2 yk = Gl uk l (3) A standard calculation then shows that for y and u related | | l=0 by (1), their instantaneous correlations matrices are related X when u is a white process. A uniform bound can be deduced by k from the following inequality ykyk⇤ =: ⌃yk = Gk l ⌃ul Gk⇤ l. (6) E{ } l=0 1 2 X sup yk Gl sup uk . (4) k  | | ! k C. Multiplication of processes Xl=0 ✓ ◆ Such quantities will occur often in the sequel, so we adopt Given any two vector-valued stationary stochastic pro- the notation cesses u and h, their element-by-element product process

y := sup yk , y1(k) h1(k) u1(k) k k1 k . . and note that the bound (4) can be rewritten in terms of the 2 . 3 := 2 . 3 2 y (k) h (k) u (k) H norm of as 6 n 7 6 n n 7 G 4 5 4 5 2 is also a stationary process. y 2 u . (5) k k1 kGk k k1 It is easy to see that equality holds when u has constant III. SISO UNSTRUCTURED UNCERTAINTY variance. Conversely, if does not have finite H2 norm, G We now consider the simplest case of uncertainty analysis equation (3) shows that any input with constant variance depicted in Figure 2. is a strictly causal LTI system, d G causes yk to grow unboundedly, and thus the bound (2) will not hold for any finite M. In summary, we can conclude that a linear time invariant d u y system is MSS if and only if it has finite H2 norm, and in G G that case the inequality (5) holds, with equality in the case of the input having equal (in time) variance. z e w For a feedback interconnection, we define the MSS sta- bility of the overall system in a manner parallel to the con- ventional scheme of injecting exogenous disturbance signals into all loops. Consider the feedback system 1 with d1 and Fig. 2. LTI system in feedback with unstructured stochastic uncertainty d being white second order processes, and and are 2 G1 G2 linear causal systems. We say that the feedback system is and w are exogenous white processes with uniform variance, MSS if all signals u1, u2, y1 and y2 have finite variance and is a white process with uniform variance and uniformly in time. independent of the signals d and w. We assume to have G finite H2 norm. The stability conditions we derive follow from a type of “small gain” analysis of the feedback interconnection in u G1 y d1 1 1 Figure 2 based on the variances of the signals in the loop. We therefore begin by deriving some basic relations between

y2 u2 d2 those variances. 2 An important consequence of the independence of and G the exogenous signals is that the block “whitens” its input signal e, i.e. even though e will in general be colored, z is white. This can be seen by letting k2 >k1 and calculating Fig. 1. MSS stability for a feedback interconnection z z = e e E{ k1 k2 } E{ k1 k1 k2 k2 } = e e =0, B. The MIMO case E{ k1 k1 k2 }E{ k2 } When is a MIMO system, the convolution input-output due to the independence of k of the other signals. In fact, a G 2 description (1) still holds with u and y as vector signals and consequence of the strict causality assumption is that current G a sequence of matrices. Consider input signals that are and future values , d and w are independent of past values { k} temporally white but with correlated components such as of any of the internal signals. When k1 = k2, we get 2 2 2 uk1 uk⇤ = k1 k2 ⌃uk , zk = ek = k ek = ek . (7) E 2 1 E E Similarly, the signal u is also white as can be seen from the Consider any time horizon k¯, and note that the quantity k¯ 2 following calculation with k2 >k1 ↵ := G can be made arbitrarily close to 1. l=0 l kGk2 The monotonicity of the sequence u gives the following uk1 uk2 = (dk1 + zk1 )(dk2 + zk2 ) P E{ } E{ } lower bounds = dk zk + dk zk E{ 1 2 } E{ 2 1 } nk¯ 2 = dk1 k2 ek2 +0 G + E{ } unk¯ nk¯ l ul d = d e =0. l=0 E{ k1 k2 }E{ k2 } X nk¯ For the case of k2 = k1 we get 2 Gnk¯ lul + d = + = + . (8) l=(n 1)k¯ uk dk zk d ek X k¯ For the other summing junction we observe that even though 2 Gl min ul + d y is colored in general, it is uncorrelated with w, which 0 1 (n 1)k¯ l nk¯ results in the relation Xl=0   = @↵u(n 1)Ak¯ + d. ek = yk + w. (9) This is a difference inequality (in n) which has the initial Finally we recall the variance inequality (5) between the condition u0 = d (this follows from the strict causality of signals u and y which follows from the assumption that G ). A simple induction argument gives is MSS together with the conclusion above that u is white. G n ¯ (↵ + + ↵ + 1) . (10) We are now in a position to state the main stability result nk ··· d for unstructured stochastic perturbations. Now if > 1, then we can choose a time horizon k¯ Lemma 3.1: Consider the system in Figure 2 with a kGk2 G such that ↵ > 1, and (10) shows that u ¯ (and thus uk ) stable LTI system and a white process with variance . nk is a geometrically increasing sequence. The case =1 The feedback system is Mean-Square Stable if and only if kGk2 is slightly more delicate. We can choose k¯ such that ↵ is as 2 < 1. close to 1 as desired. For ↵ < 1 we also have that kGk2 Proof: We assume =1. The general case follows by the 1 lim (↵n + + ↵ + 1) = . usual simple scaling. n ··· 1 ↵ “if ”) This is similar to standard sufficiency small gain !1 Thus n can be chosen such that arguments, but using variances rather than signal norms. First 1 observe that (8) yields ¯ ✏ nk 1 ↵ u e + d, k k1 k k1 for any ✏ > 0. Now given any lower bound B, choose k¯ and while (9) with (5) yields n such that ↵ is sufficiently close to 1 and ✏ is sufficiently 2 small so that e 2 u + w. k k1 kGk k k1 1 ¯ ✏ >B. These two bounds can be combined as nk 1 ↵ 2 u 2 u + w + d This proves that u is an unbounded sequence even though k k1 kGk k k1 2 2 it may not have geometric growth. e 2 e + 2d + w. k k1 kGk k k1 kGk Two remarks are in order regarding the necessity part of Combining these bounds with the condition 2 < 1 gives kGk2 the previous proof. First is that we did not need to construct bounds for the internal signals u and e in terms of the a so-called “destabilizing” perturbation as is typical in worst exogenous signals d and w case perturbation analysis. Perturbations here are described 1 statistically rather than members of sets, and variances will ( + ) , u 2 w d always grow when the stability condition is violated. Second, k k1  1 2 kGk the necessity argument can be interpreted as showing that 1 2 e 2 2d + w . 1 (1 (z)) 1 1 2 implies that the transfer function has k k  2 kGk kGk G 1 kGk a zero in the interval [0, ), and thus (1 (z)) has 1 G In addition, the remaining internal signals z and y also have an unstable pole. The argument presented above however is bounded variances as follows from (7) and (5) respectively. more easily generalizable to the MIMO case we consider in “only if”) We assume that 2 1 and show that if d kGk2 the sequel. is a white, constant variance process and w =0, then uk is an unbounded sequence. IV. STRUCTURED UNCERTAINTY From (8), (9), and (5) we have We now consider the situation where the uncertainty is k diagonal as in Figure 3, i.e. 2 uk = Gk lul + d. = diag(1,...,n). Xl=0 k u y Ʃd Ʃu Ʃy d Gk l⌃ul Gk⇤ l G Xl=0

1 ⌃ ⌃ z . e w ek Ʃ Ʃ .. Ʃz e w 2 3 n 4 5 Fig. 4. The feedback dynamics of the covariance matrices. The forward path is a matrix convolution operation while the feedback path is a memoryless matrix-valued gain represented by the Hadamard product with Fig. 3. A feedback system with a structured, diagonal perturbation ⌃. When signals are considered to take values in the cone of positive semi- definite matrices, then this feedback system can be shown to be monotone.

Assume the i’s to be temporally white, but possi- bly mutually correlated. Let (k) denote the vector Theorem 4.1: Consider the system in Figure 3 and the T (k):= (k) (k) . The instantaneous linear matrix operator 1 ··· n correlations of the ’s can be expressed with the matrix 1 ⇥ ⇤ (X):=⌃ G XG⇤ , (11) L l l ⌃ := (k) ⇤(k) , l=0 ! E{ } X where ⌃ is correlation matrix of the uncertainties and G { k} which we will assume to be independent of k. is the matrix-valued impulse response sequence of the LTI For later reference, we will need to calculate quantities system . The system is Mean Square Stable (MSS) if and G like M for some matrix M only if E{ } ⇢ ( ) < 1. L 1 1 We note that is a finite-dimensional object, it is a on . . L M⇤ = .. M .. operator mapping n n matrices to n n matrices. It has a E{ } E 82 3 2 39 ⇥ ⇥ < n n = finite number of eigenvalues. In the absence of any additional = ⌃ 4M, 5 4 5 structure, this calculation involves at worst the calculation of : ; the eigenvalues of an n2 n2 matrix as follows. Let vec (X) ⇥ the Hadamard (element-by-element) product of ⌃ and M. denote the “vectorization” operation of converting a matrix Thus, if e and z are the input and output signals (respectively) X into a vector by stacking up its columns. It is then not to the block then difficult to show that (11) can be equivalently written as

1 ⌃zk = kekek⇤k⇤ = ⌃ ⌃ek . E{ } vec ( (X)) = diag vec (⌃d) Gl Gl vec (X) . L ⌦ l=0 ! In the special case where the perturbations are uncorrelated ⇣ ⌘ X matrix representation of and all have unit variance, then ⌃ = I, and we get the L simple expression Therefore, the eigenvalues| (and{z corresponding eigenmatri-} ces) of can be found by calculating the eigenvalues/vectors L2 2 ⌃zk = diag (⌃ek ) , of its n n representation above using standard meth- ⇥ ods. However, with special structure, this calculation can where diag (M) is a diagonal matrix made up of the diagonal be significantly simplified (as in the case of uncorrelated entries of the matrix M. Observe that if the ’s are white and uncertainties. See Appendix D). mutually uncorrelated, then the vector signal z is temporally We now present the proof of this theorem. It amounts and spatially uncorrelated even though e may have both types to small gain calculations for the arrangement in Figure 3. of correlations. In other words, a structured perturbation with Expressing ⌃uk by following signals in the loop uncorrelated components will “spatially whiten” its input. ⌃ = ⌃ + ⌃ = ⌃ + ⌃ ⌃ The key to the mean square stability analysis of the uk d zk d ek system in Figure 3 is to consider the deterministic system = ⌃d + ⌃ (⌃w + ⌃yk ) of evolution of covariance matrices in Figure 4. The signals k in this feedback system are matrix-valued and they take = ⌃d + ⌃ ⌃w + Gk l⌃ul Gk⇤ l ,(12) ! values in the cone of positive semi-definite matrices. We now Xl=0 give the main result of this paper which is the mean square where the last equation follows from (6) and the fact that uk stability condition for the system in Figure 3. The proof of is temporally white. Since ⌃ul is a non-decreasing sequence this result is contained in the paragraphs of the remainder of we can bound the summation by this section. The specialization of this result to uncorrelated k k uncertainties which recovers the standard result is shown in Gk l⌃ul Gk⇤ l Gl⌃uk Gl⇤. Appendix D.  Xl=0 Xl=0 A Theorem of Schur [14, Thm 2.1] implies that for any in (14) with infinity) is clearly an upper bound, i.e. for any matrices M M and H 0, we have H M H M . k 1  2 1  2 The last two facts allow us to replace (12) with the bounds ⇢( ) ⇢( ). Lk  L ) Lk  L k Since ⇢( k) is thus a non-decreasing sequence of real ⌃ ⌃ G ⌃ G + ⌃ + ⌃ ⌃ L uk l uk l⇤ d w numbers with an upper bound, it must converge to some  ! l=0 real number which can be shown to be ⇢( ). X L 1 We conclude that ⇢( k) is a monotonic sequence with ⌃ Gl⌃u G⇤ + ⌃d + ⌃ ⌃w. L  k l l=0 ! ⇢( )= lim⇢( k). X L k L !1 Note that now all quantities other than ⌃u are independent k Now we complete the necessity proof in a similar manner to of k and the next step is see under what conditions this last ¯ that in Lemma 3.1. If ⇢( ) > 1, then k such that ⇢( k¯)= bound gives a uniform bound on the sequence ⌃uk . The key L 9 L ↵ > 1. Furthermore, there exists a non-zero semidefinite is to rewrite the above bounds in the following form eigenmatrix X such that [17]

I (⌃uk ) ⌃d + ⌃ ⌃w, L  k¯(X)=⇢( k¯) X = ↵X. (15) L L where is⇣ the linear⌘ matrix operator L Referring back to (12) we obtain the following bounds 1 nk¯ (X):=⌃ GlXGl⇤ . ⌃ = ⌃ G ¯ ⌃ G⇤ + ⌃ L ! unk¯ nk l ul nk¯ l d l=0 0 1 X l=0 It is easy to show that this operator maps positive semi- X @ ¯ A definite matrices to positive semi-definite matrices. It is thus nk ⌃ Gnk¯ l⌃ul Gn⇤k¯ l + ⌃d “cone-invariant” in the terminology of [15] for the cone of 0 1 l=(n 1)k¯ positive semi-definite matrices. It then follows [15, Thm. 4] X that we can bound @ A k¯ ⌃u(n 1)k¯ + ⌃d. L (1 ⇢( )) ⌃ I (⌃ ) ⌃ + ⌃ ⌃ . ⇣ ⌘ L uk  L uk  d w A simple induction argument shows that if we use the eigenmatrix X from (15) for ⌃ , we obtain We thus arrive at the sufficient⇣ ⌘ condition d ⌃ (↵n + + ↵ + 1) X. ⇢( ) < 1 unk¯ ··· L Since ↵ > 1 and X is a non-zero semidefinite matrix, then for the MSS of the feedback system. This gives the uniform ⌃ is a geometrically growing sequence. (in k) bound u We note that the above argument produces a sort of worst 1 ⌃ (⌃ + ⌃ ⌃ ) . case covariance ⌃d as the eigenmatrix of the operator . The uk  (1 ⇢( )) d w L L significance of this is yet to be investigated. The stability of G then implies in addition that all other signals in Figure 3 have bounded covariances. For the converse, we assume w to be zero and recall REFERENCES equation (12) [1] M. Dahleh and Y. Ohta, “A necessary and sufficient condition for k robust bibo stability,” Systems & control letters, vol. 11, no. 4, pp. 271–275, 1988. ⌃uk = ⌃ Gk l⌃ul Gk⇤ l + ⌃d ! [2] A. Megretski, “Necessary and sufficient conditions of stability: A Xl=0 multiloop generalization of the circle criterion,” Automatic Control, k IEEE Transactions on, vol. 38, no. 5, pp. 753–756, 1993. ⌃ Gk l⌃d Gk⇤ l + ⌃d. (13) [3] B. Bamieh and M. Dahleh, “On robust stability with structured time- l=0 ! invariant perturbations,” Systems & control letters, vol. 21, no. 2, pp. X 103–108, 1993.

The lower bound follows from the fact that ⌃ul is monotonic, [4] J. Doyle, J. Wall, and G. Stein, “Performance and robustness analysis for structured uncertainty,” in Decision and Control, 1982 21st IEEE ⌃u = ⌃d, and the previously invoked theorem of Schur 0 Conference on, vol. 21. IEEE, 1982, pp. 629–636. stating that ⌃ M ⌃ M whenever M M .To 1 2 1 2 [5] M. Khammash and J. Pearson Jr, “Performance robustness of discrete- study (13) we define the sequence of linear matrix operators time systems with structured uncertainty,” Automatic Control, IEEE Transactions on, vol. 36, no. 4, pp. 398–412, 1991. k [6] J. Shamma, “Robust stability with time-varying structured uncer- k (X):=⌃ Gk lXGk⇤ l . (14) tainty,” Automatic Control, IEEE Transactions on, vol. 39, no. 4, pp. L l=0 ! 714–724, 1994. X [7] M. Fan, A. Tits, and J. Doyle, “Robustness in the presence of mixed This sequence is itself non-decreasing, in the sense that parametric uncertainty and unmodeled dynamics,” Automatic Control, IEEE Transactions on, vol. 36, no. 1, pp. 25–38, 1991. k(X) k+1(X) for any symmetric X. It has been L  L [8] J. Doyle, “Structured uncertainty in control system design,” in Deci- shown [16] that this implies that ⇢( ) ⇢( ). The Lk  Lk+1 sion and Control, 1985 24th IEEE Conference on, vol. 24. IEEE, operator := (replacing the upper limit in the sum 1985, pp. 260–265. L L1 [9] J. Lu and R. Skelton, “Mean-square small gain theorem for stochastic is clearly a linear operator, and thus the ma- D control: discrete-time case,” Automatic Control, IEEE Transactions on, trix representation (on diagonal matrices) of the operator vol. 47, no. 3, pp. 490–494, 2002. diag (G XG⇤)+diag (G XG⇤) is simply G G +G G . [10] N. Elia, “Remote stabilization over fading channels,” Systems & 1 1 2 2 1 1 2 2 Control Letters, vol. 54, no. 3, pp. 237–249, 2005. Clearly this fact also holds for any finite or infinite sum, e.g. [11] D. Hinrichsen and A. Pritchard, “Stability radii for infinite dimensional the matrix representation of the operator systems with stochastic uncertainty,” in Proc. 3rd. European Control Conf. Citeseer, 1995, pp. 3291–3293. [12] A. El Bouhtouri and A. Pritchard, “Stability radii of linear systems X diag GkXGk⇤ (16) with respect to stochastic perturbations,” Systems & control letters, 7! ! vol. 19, no. 1, pp. 29–33, 1992. Xk [13] L. Ljung, System identification. Wiley Online Library, 1999. on diagonal matrices X is [14] R. Horn and R. Mathias, “Block-matrix generalizations of schur’s basic theorems on hadamard products,” Linear Algebra and its Ap- Gk Gk. (17) plications, vol. 172, pp. 337–346, 1992. [15] P. Parrilo and S. Khatri, “On cone-invariant linear matrix inequalities,” Xk Automatic Control, IEEE Transactions on, vol. 45, no. 8, pp. 1558– In particular, the (possibly non-zero) eigenvalues of the 1563, 2000. [16] H. Tigelaar, “On monotone linear operators and the spectral radius operator (16) are simply the eigenvalues of the matrix (17). of their representing matrices,” SIAM Journal on Matrix Analysis and Furthermore, since the matrix (17) is non-negative, its spec- Applications, vol. 12, p. 726, 1991. tral radius is achieved by a real positive eigenvalue. This [17] A. Berman and R. Plemmons, Nonnegative matrices in the mathemat- ical sciences. Society for Industrial , 1994, vol. 9. statement is of course also valid for the operator (16).

APPENDIX Misc facts A. Given C 0 and A B, then tr (AC) tr (BC) . This can be seen by using the dyadic decomposition of C. B. If u and y are inputs and outputs respectively, then ⌃ ⌃ 0 ⌃ ⌃ 0 uk+1 uk ) yk+1 yk C. For any two symmetric matrices A and B (A)tr(B) tr (AB) (A)tr(B) . min   max The roles of A and B in the above formula can of course be reversed.

D. The operator X diag ( G XG⇤) 7! k k k First, consider the followingP operator on square matrices

(X) := diag (GXG⇤) . D The eigenmatrices of this operator must clearly be diagonal matrices, so we can restrict attention to understanding its action on diagonal matrices. Let V := diag (v1,...,vn) be a diagonal matrix, then W = diag (GXG⇤) is also a diagonal matrix and can thus be written as W := diag (w1,...,wn). Evaluating element-by-element leads to the following equa- tion between the entries of V and W 2 2 w1 g11 g1n v1 . . ···. . . 2 . 3 = 2 . .. . 3 2 . 3 . w g2 g2 v 6 n 7 6 n1 ··· nn 7 6 n 7 Thus, when4 restricted5 4 to diagonal matrices,5 4 the operator5 D has as its matrix representation the matrix G G, the element- by-element square of the matrix G.