Supplementary Notes on Linear Algebra
Mariusz Wodzicki
May 3, 2015
1 Vector spaces
1.1 Coordinatization of a vector space 1.1.1
Given a basis B = {b1,..., bn} in a vector space V , any vector v ∈ V can be represented as a linear combination
v = β1b1 + ··· + βnbn (1) and this representation is unique, i.e., there is only one sequence of coeffi- cients β1, ··· , βn for which (1) holds.
1.1.2 The correspondence between vectors in V and the coefficients in the ex- pansion (1) defines n real valued functions on V ,
∗ ∗ bi : V → R, bi (v) ˜ βi, (i = 1, . . . , n).(2)
1.1.3 If 0 0 0 v = β1b1 + ··· + βnbn (3) is another vector, then
0 0 0 v + v = (β1 + β1)b1 + ··· + (βn + βn)bn which shows that
∗ 0 0 ∗ ∗ 0 bi (v + v ) = βi + βi = bi (v) + bi (v ), (i = 1, . . . , n).
Similarly, for any number α, one has
αv = (αβ1)b1 + ··· + (αβn)bn which shows that
∗ ∗ bi (αv) = αbi (v), (i = 1, . . . , n).
3 ∗ In other words, each function bi : V → R is a linear transformation from the vector space V to the one-dimensional vector space R, and the corre- spondence β1 ˜ . v 7−→ [v]B . (4) βn is a linear transformation from V to the n-dimensional vecor space of column vectors Rn .
1.1.4 The coordinatization isomorphism of V with Rn
The kernel of (4) is {0} since vectors b1,..., bn are linearly independent. n The range of (4) is the whole R since vectors b1,..., bn span V . Thus, the n correspondence v 7→ [v]B identifies V with the vector space R . We shall refer to (4) as the coordinatization of the vector space V in basis B.
∨ 1.2 The dual space V 1.2.1 Linear transformations V → R are referred to as (linear) functionals on V (they are also called linear forms on V ). Linear functionals form a vector ∨ space of their own which is called the dual of V . We will denote it V (pronounce it “V dual” or “V check”).
1.2.2 An example: the trace of a matrix The trace of an n × n matrix A = αij is the sum of the diagonal entries,
tr A ˜ α11 + ··· + αnn.(5) The correspondence A 7→ tr A is a linear functional on the vector space Matn(R) of n × n matrices.
Exercise 1 Calculate both tr AB and tr BA and show that
tr AB = tr BA (6) where α11 ... α1n . . A = . . (7) αm1 ... αmn
4 denotes an arbitrary m × n matrix and β11 ... β1m . . B = . . (8) βn1 ... βnm denotes an arbitrary n × m matrix.
1.2.3 An example: the dual of the space of m × n matrices For any n × m matrix (8), let us consider the linear functional on the space of m × n matrices:
φB : A 7−→ tr AB (A ∈ Matmn(R)).(9)
τ τ Exercise 2 Calculate φB(B ), where B denotes the transpose of B, and show that it vanishes if and only if B = 0. Deduce that φB = 0 if and only if B = 0.
1.2.4 The correspondence
∨ φ : Matnm(R) −→ Matmn(R) , B 7−→ φB,(10) is a natural linear transformation from the space of n × m matrices into the dual of the space of m × n matrices. In view of Exercise 2 it is injective. By considering bases in V , in the next sections we will show that ∨ the dimension of V equals the dimension of V if the latter is finite. In ∨ particular, this will imply that dim Matmn(R) = dim Matmn(R). Since the transposition of matrices,
τ A 7−→ A (A ∈ Matmn(R)), is an isomorphism of vector spaces, it will follow that
∨ dim Matnm(R) = dim Matmn(R) .
A corollary of this is that the dual space Matmn(R) is naturally identified with the vector space of n × m matrices, via identification (10).
5 1.2.5 The duality between the spaces of row and column vectors
n In particular, the space of column vectors R = Matn1(R) is naturaly identified with the dual of the space of row vectors Mat1n(R) and, vice- versa, the space of row vectors Mat1n(R) is naturally identified with the n dual of the space of column vectors R = Matn1(R).
1.2.6 The coordinate functionals The coordinatization isomorphism of V with Rn is made up of n coordi- ∨ nate functionals, cf. (2). They span V . Indeed, given a linear functional φ : V → R, let α1 ˜ φ(b1),..., αn ˜ φ(bn). Then, for any vector v ∈ V , one has
φ(v) = φ(β1b1 + ··· + βnbn)
= β1φ(b1) + ··· + βnφ(bn)
= β1α1 + ··· + βnαn
= α1β1 + ··· + αnβn ∗ ∗ = α1b1 (v) + ··· + αnbn(v) ∗ ∗ = α1b1 + ··· + αnbn (v) which shows that the linear functional φ is a linear combination of func- ∗ ∗ tionals b1,..., bn , ∗ ∗ φ = α1b1 + ··· + αnbn.
1.2.7 ∗ ∗ The coordinate functionals b1,..., bn are linearly independent. Indeed, if a linear combination ∗ ∗ α1b1 + ··· αnbn is the zero functional, then its values on v = b1,..., bn are all zero. But those values are: α1,..., αn, since ( ∗ 1 if i = j bi (bj) = .(11) 0 if i , j
6 1.2.8 The dual basis B∗ ∗ ˜ ∗ ∗ Thus, B {b1,..., bn} forms a basis of the dual space. Note, that ∨ dim V = dim V.
1.3 Scalar products 1.3.1 Bilinear pairings A function of two vector arguments h , i : V × V → R (12) is said to be a bilinear pairing on V if it is a linear functional in each argument. (Bilinear pairings are also called bilinear forms on V .)
1.3.2 We say that the bilinear pairing is nondegenerate if, for any nonzero vector v ∈ V , there exists v0 ∈ V , such that
hv, v0i , 0.
1.3.3 We say that the bilinear pairing is symmetric if, for any vectors v, v0 ∈ V , one has hv0, vi = hv, v0i.
1.3.4 Orthogonality We say that vectors v and v0 are orthogonal if hv, v0i = 0. We denote this fact by v ⊥ v0 .
1.3.5 If X is a subset of V , the set of vectors orthogonal to every element of X is denoted X⊥ ˜ {v ∈ V | v ⊥ x for all x ∈ X } (13)
Exercise 3 Show that X⊥ is a vector subspace of V and
X ⊆ X⊥⊥.(14)
7 1.3.6 We say that the bilinear pairing is positively defined if, for any vector v ∈ V , one has hv, vi ≥ 0. Theorem 1.1 (The Cauchy-Schwarz Inequality) Let h , i be a positively de- fined symmetric bilinear pairing on a vector space V . Then, for any vectors v, v0 ∈ V , one has the following inequality hv, v0i2 ≤ hv, vihv0, v0i.(15)
1.3.7 We shall demonstrate (15) by considering the second degree polynomial p(t) ˜ htv + v0, tv + v0i = hv, vit2 + hv, v0i + hv0, vit + hv0, v0i
= at2 + bt + c where a = hv, vi, b = 2hv, v0i and c = hv0, v0i. In view of the hypothesis, p(t) ≥ 0 for all real number t. This is equivalent to the inequality b2 ≤ 4ac which yields inequality (15).
1.3.8 An immediate corollary of the Cauchy-Schwarz Inequality is that a sym- metric bilinear pairing is nondegenerate and positively defined if and only if hv, vi > 0 for any nonzero vector in V .
1.3.9 Scalar products A nondegenerate positively defined symmetric bilinear pairing on V is called a scalar product.
Exercise 4 Show that a set of nonzero vectors {v1,..., vn}, mutually orthogonal with respect to some scalar product on V , is linearly independent. (Hint: for a linear combination representing the zero vector,
α1vq + ··· + αnvn = 0 calculate the scalar product of both sides with each vi .)
8 1.3.10 The associated norm For any scalar product, the functional q v 7−→ kvk ˜ hv, vi (16) is called the associated norm. Using the norm notation, we can rewrite the Cauchy-Schwarz Inequality as 0 0 hv, v i ≤ kvkkv k.(17)
1.3.11 The Triangle Inequality Note that kv + v0k2 = kvk2 + 2hv, v0i + kv0k2 while 2 kvk + kv0k = kvk2 + 2kvkkv0k + kv0k2. In view of the Cauchy-Schwarz Inequality, the bottom expression is not less than the top expression. Equivalently,
kv + v0k ≤ kvk + kv0k,(18) for any pair of vectors v and v0 in V . This is known as the Triangle Inequality.
1.3.12 The associated norm satisfies also the following two conditions
kαvk = |α|kvk,(19) for any real number α and any vector v ∈ V , and
kvk > 0 (20) for any nonzero vector v ∈ V .
1.3.13 Norms on a vector space Any function V −→ [0, ∞] that satisfies the Triangle Inequality (18) and conditions (19) and (20) is called a norm on V .
9 1.3.14 Polarization Formula In terms of the associated norm, the scalar product is expressed by means of the identity
1 hv, v0i = kv + v0k2 − kvk2 − kv0k2.(21) 2 known as the Polarization Formula. If a norm k k on a vector space V is associated with a scalar product, then the right-hand-side of (21) must depend on v linearly. If it does not, then that norm is not associated with a scalar product.
1.3.15 Quadratic forms A function q : V → R is called a quadratic form if the pairing assigning the number 1 hv, v0i ˜ q(v + v0) − q(v) − q(v0) (22) 2 to a pair of vectors v and v0 in V , is bilinear. Note that the pairing defined by (22) is symmetric. Vice-versa, for any symmetric bilinear pairing h , i, the function q(v) ˜ hv, vi (v ∈ V),(23) is a quadratic form on V .
1.3.16 We obtain a natural one-to-one correspondence between symmetric bilinear pairings and quadratic forms on V
symmetric bilinear pairings quadratic forms ←→ .(24) h , i : V × V −→ R q : V −→ R
1.3.17 Nondegenerate symmetric pairings correspond to nondegenerate quadratic forms, i.e., the ones that satisfy
q(v) = 0 if and only if v = 0.
10 1.3.18 Positively defined symmetric bilinear pairings correspond to positively defined quadratic forms, i.e., the ones that satisfy
q(v) ≥ 0 (v ∈ V).
1.3.19 An example: the dot product 0 Given a basis B = {b1,..., bn} in V , the dot product v ·B v of two vectors (1) and (3) is defined as
0 ˜ 0 0 v ·B v β1β1 + ··· + βnβn.(25)
It is the only scalar product on V for which B is orthonormal, i.e., ( 1 if i = j hbi, bji = .(26) 0 if i , j
1.3.20 In the special case of V = Rn and B being the standard basis
1 0 . . e1 = . ,..., en = . ,(27) 0 1 we obtain the dot product on Rn .
n 1.3.21 An example: the lp -norms on R For a positive number p > 0, consider the following functional on Rn ,
1 p p p x 7−→ kxkp ˜ |x1| + · · · |xn| .(28)
This functional satisfies the Triangle Inequality if and only if p ≥ 1. For any p > 0, it satisfies the other two properties of a norm. Only for p = 2 the right-hand-side of the Polarization Formula is linear in v. In that case, n the scalar product is the dot product on R and the l2 -norm is known n as the Euclidean norm. The vector space R equipped with the l2 -norm is referred to as the n-dimensional Euclidean space.
11 1.3.22 An example: the Killing scalar product
Consider the bilinear pairing on the vector space Matmn(R) of m × n matrices hA, Bi ˜ tr Aτ B.(29) The pairing is known under the name of Killing form and plays a very important role in Representation Theory.
Exercise 5 Calculate tr AτA and show that tr AτA > 0 for all nonzero m × n matrices. Explain why tr Aτ B = tr BτA.
1.3.23
n Note that the Killing scalar product on Matn1(R) = R coincides with the standard dot product on Rn .
1.3.24 Isometries A linear transformation T : V → V0 between vector spaces equipped with bilinear pairings h , i and, respectively, h , i0 , is called an isometry if it preserves the value of the pairing, i.e., if
0 hTv1, Tv2i = hv1, v2i for any pair of vectors v1 and v2 in V .
1.3.25
n The coordinatization isomorphism []B : V → R is an isometry between n V equipped with the dot product ·B and R equipped with standard dot product: [v1]B · [v2]B = v1 ·B v2.(30)
1.3.26 Description of bilinear pairings on a vector space with a basis
Given a basis B = {b1,..., bn} and an arbitrary bilinear pairing (12), let us consider the n × n matrix Q = qij where
qij ˜ hbi, bji (1 ≤ i, j ≤ n).(31)
12 For a pair of vectors, as in (1) and (3), one has
0 0 0 hv, v i = h β1b1 + ··· + βnbn, β1b1 + ··· + βnbn i, 0 = ∑ βihbi, bjiβj 1≤i,j≤n
= ∑ βiqijβj (32) 1≤i,j≤n 0 = [v]B · Q[v ]B (33)