1 Foundations of Matrix Analysis In this chapter we recall the basic elements of linear algebra which will be employed in the remainder of the text. For most of the proofs as well as for the details, the reader is referred to [Bra75], [Nob69], [!al58]. F#rther res#lts on eigen$al#es can be found in [Hou75] and [%il65]. 1.1 Vector Spaces Definition 1.1& vector space o$er the n#meric 'eldK(K)R orK)C* is a nonempty setV , whose elements are called vectors and in which two operations are de'ned, called addition and scalar multiplication, that en+oy the following properties, 1. addition is comm#tati$e and associati$e. 2. there exists an element0∈V (the zero vector or null vector* s#ch that v00)v for eachv∈V. 3. 2·v)0,-·v)v, for eachv∈V , where 2 and - are re specti$ely the 3ero and the #nity ofK. 4. for each elementv∈V there exists its oppos ite,−v, inV s#ch that v0(−v*)0. 5. the following distrib#tive properties hold ∀α∈K,∀v,w∈V, α(v0w*)αv0αw, ∀α,β∈K,∀v∈V,(α0β*v)αv0βv. 6. the following associati$e property holds ∀α,β∈K,∀v∈V,(αβ*v)α(βv*. � 4 1 Foundations of Matrix Analysis Example 1.1 Remarkable instances of vector spaces are: -V=R n (respectivelyV=C n): the set of then-tuples of real (respect ively complex) numbers!n≥1" P n k -V= n: the set of polynomialsp n(x) = k=0 akx #ith real (or complex) coefficientsa k havin% de%ree less than or equal to�n!n≥'" -V=C p([a, b]): the set of real (or complex) valued functions #hich are contin uous on (a, b) up to theirp th derivative! '≤p<∞* • Definition 1.2 %e say that a nonempty partW ofV isa vector subspace of V i5W is a $ector space o$erK. � ∞ Example 1.2 +he vector spaceP n is a vector subspace ofC (R)! #hich is the space of in,nite continuously differentiable functions on the real line* A trivial sub space of any vector space is the one containin% only the .ero vector*• In partic#lar, the setW of the linear combinat ions of a system ofp $ectors ofV,{v 1,...,v p}, is a $ector s#bspace ofV , called the generated subspace or span of the $ector system, and is denoted by W ) span{v 1,...,v p} ){v)α 1v1 0...0α pvp withα i ∈K, i)-, . , p}. 6he system{v 1,...,v p} is called a system of generators forW. IfW 1,...,Wm are $ector s#bspaces ofV, then the set S){w,w)v 1 0...0v m withv i ∈W i, i)-,...,m} is also a $ector s#bspace ofV . %e say thatS is the direct sum of the s#bspaces Wi if any elements∈S admits a #niq#e repr esentation of the forms) v1 0...0v m withv i ∈W i andi)-,...,m. In s#ch a case, we sha ll write S)W 1 ⊕...⊕W m. Definition 1.3 & system of $ectors{v 1,...,v m} of a $ector spaceV is called linearly independent if the relation α1v1 0α 2v2 0...0α mvm )0 withα 1, α2, . , αm ∈K implies thatα 1 )α 2 )...)α m ) 2. 8therwise, the system will be called linearly dependent. � %e call a basis ofV any system of linearly independent generators ofV. If {u1,...,u n} is a basis ofV , the expressionv)v 1u1 0...0v nun is called the decomposition ofv with respect to the bas is and the scalarsv 1, . , vn ∈K are the components ofv with respect to the gi$ en basis. 9oreo$er, the following property holds. 1*/ Matrices 0 Property 1.1 LetV be a vector space which admits a basis ofn vectors. Then every system of linearly independent vectors ofV has at mostn elements and any other basis ofV hasn elements. The numbern is called the dimension of V and we write dim(V*)n. If, instead, for anyn there always existn linearly independent v ectors ofV, the vector space is called infinite dimensional. Example 1.3 For any integerp the spaceC p([a, b]) is in,nite dimensional. +he spacesR n andC n have dimension equal ton* +he usual basis forR n is the set of unit vectors{e 1,...,e n} #here (ei)j =δ ij for i,j=1,...n! #hereδ ij denotes the Kronecker symbol e&ual to ' ifi=� j and 1 ifi=j* +his choice is of cour se not the only one that is possible (see 1xercise /)* • 1.2 Matrices :etm andn be two positi$e intege rs. %e call a matrix ha$ingm rows and n col#mns, or a matrixm×n, or a matrix (m, n), with elements inK, a set of mn scalarsa ij ∈K, withi)-,...,m andj)-, . n, represented in the following rectang#lar array a11 a12 . a1n a21 a22 . a2n &) . (-.1) . a a . a m1 m2 mn %henK)R orK)C we shall respecti$ely w rite &∈R m×n or &∈C m×n, to explicitly o#tline the n#merical 'elds which the elements of & belong to. ;apital letters will be #sed to denote the matrices, while the lower case letters corresponding to those #pper case letters will denote the matrix entries. %e shall abbre$iate (-.-* as & ) (aij * withi)-,...,m andj)-,...n. 6he indexi is called row index, whilej is the col#mn index. 6he set (ai1, ai2, . , ain* is called the i-th row of &. li<ewise, (a1j , a2j , . , amj * is the j-th column of &. Ifn)m the matrix is called s uared or ha$ing ordern and the set of the entries (a11, a22, . , ann* is called its main diagonal. & matrix having one row or one col#mn is called a row vector or column vector respecti$ely. =nless otherwise speci'ed, we shall always ass#me that a $ector is a col#mn $ector. In the casen)m ) 1, the matrix will s imply denote a scalar ofK. >ometimes it t#rns out to be #sef#l to disting#ish within a matrix the set made #p by speci'ed rows and col#mns. 6his prompts #s to introd#ce the following de'nition. Definition 1.4 Let & be a matrixm×n. :et -≤i 1 < i2 < . < ik ≤m and -≤j 1 < j2 < . < jl ≤n two sets of contiguous indexes. 6he matrix >(k×l* 2 1 Foundations of Matrix Analysis of entriess pq )a ipjq withp)-,...,k,q)-,...,l is called a submatrix of A. Ifk)l andi r )j r forr)-,...,k, > is called a principal submatrix of &.� Definition 1.5 & matrix &(m×n* is called block partitioned or said to be partitioned into submatrices if &11 &12 ...& 1l &21 &22 ...& 2l &) . , . .. & & ...& k1 k2 kl where &ij are s#bmatrices of A. � &mong the possible partitions of &, we recall in partic#lar the partition by columns &)(a1,a 2,...,a n*, ai being thei?th col#mn $ector of & . In a similar way the partition by rows of & can be defined. 6o 'x the notations, if & is a matrixm×n, we shall denote by &(i1 ,i 2, j1 ,j 2*)(aij *i 1 ≤i≤i 2, j1 ≤j≤j 2 the s#bmatrix of & of si3e (i2 −i 1 0 -*×(j 2 −j 1 0 -* that lies between the rowsi 1 andi 2 and the col#mnsj 1 andj 2. :i<ewise, ifv is a $ector of si3en, we shall denote byv(i 1 ,i 2* the $ector of sizei 2 −i 1 0 - made #p by thei 1?th to thei 2?th components ofv. 6hese notations are con$enient in view of programming the algorithms that will be presented throughout the $ol#me in the 9&6:&B lang#age. 1.3 Operations with Matrices :et & ) (aij * and B ) (bij * be two matricesm×n o$erK. %e say that & is e ual to B, ifa ij )b ij fori)-,...,m,j)-, . , n. 9oreo$er, we de'ne t he following operations: @ matrix sum, the matrix sum is the matrix & 0 B ) (aij 0b ij *. 6he ne#tral element in a matrix s#m is the null matrix, still denoted by 2 and made #p only by n#ll entries. @ matrix multiplication by a scalar, the m#ltiplication of & byλ∈K, is a matrixλ&)(λa ij *. @ matrix product, the prod#ct of two matrices & and B of si3es (m, p* and p (p, n* respecti$ely, is a matrix ;(m, n* whose entries arec ij ) aikbkj, k�=1 fori)-, . , m,j)-, . , n. 1*3 4perations #ith Matrices 5 6he matrix prod#ct is associative and distrib#ti$e with respect to the matrix s#m, b#t it is not in general comm#tati$e. 6he s7#are matrices for which the property &B ) B& holds, will be called commutative. In the case of s7#are matrices, the ne#tral element in the matrix prod#ct is a s7#are matrix of ordern called the unit matrix of ordern or, more fre7#ently, the identity matrix gi$en by In )(δij *. 6he identity matrix is, by definition, the only matrixn×n s#ch that &I n )In& ) & for all s7#are matrices &. In the following we shall omit the s#bscriptn #nless it is strictly nec essary. 6he identity matrix is a special instance of a diagonal matrix of ordern, that is, a s7#are matrix of the type A ) (diiδij *. %e will #se in the following the notation A ) diag(d11, d22, .