A Review of Linear Algebra Gerald Recktenwald Portland State University Mechanical Engineering Department [email protected] These slides are a supplement to the book Numerical Methods with Matlab: Implementations and Applications, by Gerald W. Recktenwald, c 2000–2006, Prentice-Hall, Upper Saddle River, NJ. These slides are copyright c 2000–2006 Gerald W. Recktenwald. The PDF version of these slides may be downloaded or stored or printed only for noncommercial, educational use. The repackaging or sale of these slides in any form, without written consent of the author, is prohibited. The latest version of this PDF file, along with other supplemental material for the book, can be found at www.prenhall.com/recktenwald or web.cecs.pdx.edu/~gerry/nmm/. Version 0.99 August 22, 2006 page 1 Primary Topics • Vectors • Matrices • Mathematical Properties of Vectors and Matrices • Special Matrices NMM: A Review of Linear Algebra page 2 Notation Variable type Typographical Convention Example scalar lower case Greek σ, α, β vector lower case Roman u, v, x, y, b matrix upper case Roman A, B, C NMM: A Review of Linear Algebra page 3 Defining Vectors in Matlab • Assign any expression that evaluates to a vector >>v=[1357] >> w = [2; 4; 6; 8] >> x = linspace(0,10,5); >> y = 0:30:180 >> z = sin(y*pi/180); • Distinquish between row and column vectors >> r = [1 2 3]; % row vector >> s = [1 2 3]’; % column vector >>r-s ??? Error using ==> - Matrix dimensions must agree. Although r and s have the same elements, they are not the same vector. Furthermore, operations involving r and s are bound by the rules of linear algebra. NMM: A Review of Linear Algebra page 4 Vector Operations • Addition and Subtraction • Multiplication by a scalar • Transpose • Linear Combinations of Vectors • Inner Product • Outer Product • Vector Norms NMM: A Review of Linear Algebra page 5 Vector Addition and Subtraction Addition and subtraction are element-by-element operations c = a + b ⇐⇒ ci = ai + bi i =1,...,n d = a − b ⇐⇒ di = ai − bi i =1,...,n Example: 2 3 2 3 1 3 a = 425 b = 425 3 1 2 3 2 3 4 −2 a + b = 445 a − b = 4 0 5 4 2 NMM: A Review of Linear Algebra page 6 Multiplication by a Scalar Multiplication by a scalar involves multiplying each element in the vector by the scalar: b = σa ⇐⇒ bi = σai i =1,...,n Example: 2 3 2 3 4 2 a a = 465 b = = 435 2 8 4 NMM: A Review of Linear Algebra page 7 Vector Transpose The transpose of a row vector is a column vector: 2 3 1 ˆ ˜ T u = 1, 2, 3 then u = 425 3 Likewise if v is the column vector 2 3 4 T ˆ ˜ v = 455 then v = 4, 5, 6 6 NMM: A Review of Linear Algebra page 8 Linear Combinations (1) Combine scalar multiplication with addition 2 3 2 3 2 3 2 3 u1 v1 αu1 + βv1 w1 6 u 7 6 v 7 6 αu + βv 7 6 w 7 α 6 2 7 + β 6 2 7 = 6 2 2 7 = 6 2 7 4 . 5 4 . 5 4 . 5 4 . 5 um vm αum + βvm wm Example: 2 3 2 3 −2 1 r = 4 1 5 s = 405 3 3 2 3 2 3 2 3 −4 3 −1 t =2r +3s = 4 2 5 + 405 = 4 2 5 6 9 15 NMM: A Review of Linear Algebra page 9 Linear Combinations (2) Any one vector can be created from an infinite combination of other “suitable” vectors. Example: » – » – » – 4 1 0 w = =4 +2 2 0 1 » – » – 1 1 w =6 − 2 0 −1 » – » – 2 −1 w = − 2 4 1 » – » – » – 4 1 0 w =2 − 4 − 2 2 0 1 NMM: A Review of Linear Algebra page 10 Linear Combinations (3) 4 [2,4] [1,-1] 3 2 [4,2] [-1,1] 1 [0,1] [1,1] 0 [1,0] Graphical interpretation: 0123456 • Vector tails can be moved to convenient locations • Magnitude and direction of vectors is preserved NMM: A Review of Linear Algebra page 11 Vector Inner Product (1) In physics, analytical geometry, and engineering, the dot product has a geometric interpretation Xn σ = x · y ⇐⇒ σ = xiyi i=1 x · y = x2 y2 cos θ NMM: A Review of Linear Algebra page 12 Vector Inner Product (2) The rules of linear algebra impose compatibility requirements on the inner product. The inner product of x and y requires that x be a row vector y be a column vector 2 3 y1 ˆ ˜ 6 7 6y27 x1 x2 x3 x4 4 5 = x1y1 + x2y2 + x3y3 + x4y4 y3 y4 NMM: A Review of Linear Algebra page 13 Vector Inner Product (3) For two n-element column vectors, u and v, the inner product is Xn T σ = u v ⇐⇒ σ = uivi i=1 The inner product is commutative so that (for two column vectors) uT v = vT u NMM: A Review of Linear Algebra page 14 Computing the Inner Product in Matlab The * operator performs the inner product if two vectors are compatible. >> u = (0:3)’; % u and v are >> v = (3:-1:0)’; % column vectors >> s = u*v ??? Error using ==> * Inner matrix dimensions must agree. >> s = u’*v s= 4 >> t = v’*u t= 4 NMM: A Review of Linear Algebra page 15 Vector Outer Product The inner product results in a scalar. Example: Outer product of two 4- The outer product creates a rank-one element2 column3 vectors matrix: u1 6 7 ˆ ˜ T 6u27 T uv = 4 5 v1 v2 v3 v4 A = uv ⇐⇒ ai,j = uivj u3 u4 2 3 u1v1 u1v2 u1v3 u1v4 6 7 6u2v1 u2v2 u2v3 u2v47 = 4 5 u3v1 u3v2 u3v3 u3v4 u4v1 u4v2 u4v3 u4v4 NMM: A Review of Linear Algebra page 16 Computing the Outer Product in Matlab The * operator performs the outer product if two vectors are compatible. u = (0:4)’; v = (4:-1:0)’; A = u*v’ A= 00000 43210 86420 129630 16 12 8 4 0 NMM: A Review of Linear Algebra page 17 Vector Norms (1) Compare magnitude of scalars with the absolute value ˛ ˛ ˛ ˛ ˛α˛ > ˛β˛ Compare magnitude of vectors with norms x > y There are several ways to compute ||x||. In other words the size of two vectors can be compared with different norms. NMM: A Review of Linear Algebra page 18 Vector Norms (2) Consider two element vectors, which lie in a plane b = (2,4) a = (4,2) c = (2,1) a = (4,2) Use geometric lengths to represent the magnitudes of the vectors p √ p √ p √ 2 2 2 2 2 2 a = 4 +2 = 20,b = 2 +4 = 20,c = 2 +1 = 5 We conclude that a = b and a >c or a = b and a > c NMM: A Review of Linear Algebra page 19 The L2 Norm The notion of a geometric length for 2D or 3D vectors can be extended vectors with arbitrary numbers of elements. The result is called the Euclidian or L2 norm: ! Xn 1/2 ` 2 2 2 ´1/2 2 x2 = x1 + x2 + ...+ xn = xi i=1 The L2 norm can also be expressed in terms of the inner product √ √ T x2 = x · x = x x NMM: A Review of Linear Algebra page 20 p-Norms For any integer p ` p p p´1/p xp = |x1| + |x2| + ...+ |xn| The L1 norm is sum of absolute values Xn x1 = |x1| + |x2| + ...+ |xn| = |xi| i=1 The L∞ norm or max norm is x = max (|x1|, |x2|,...,|xn|) = max (|xi|) ∞ i Although p can be any positive number, p =1, 2, ∞ are most commonly used. NMM: A Review of Linear Algebra page 21 Application of Norms (1) Are two vectors (nearly) equal? Floating point comparison of two scalars with absolute value: ˛ ˛ ˛α − β˛ ˛ ˛ <δ ˛α˛ where δ is a small tolerance. Comparison of two vectors with norms: y − z <δ z NMM: A Review of Linear Algebra page 22 Application of Norms (2) Notice that y − z <δ z is not equivalent to y−z <δ. z This comparison is important in convergence tests for sequences of vectors. See Example 7.3 in the textbook. NMM: A Review of Linear Algebra page 23 Application of Norms (3) Creating a Unit Vector T Given u =[u1,u2,...,um] , the unit vector in the direction of u is u uˆ = u2 Proof: ‚ ‚ ‚ u ‚ 1 uˆ = ‚ ‚ = u =1 2 ‚u ‚ u 2 2 2 2 The following are not unit vectors u u u1 u∞ NMM: A Review of Linear Algebra page 24 Orthogonal Vectors From geometric interpretation of the inner product u · v = u2 v2 cos θ u · v uT v cos θ = = u2 v2 u2 v2 Two vectors are orthogonal when θ = π/2 or u · v =0. In other words uT v =0 if and only if u and v are orthogonal. NMM: A Review of Linear Algebra page 25 Orthonormal Vectors Orthonormal vectors are unit vectors that are orthogonal. A unit vector has an L2 norm of one. The unit vector in the direction of u is u uˆ = u2 Since √ u2 = u · u it follows that u · u =1if u is a unit vector. NMM: A Review of Linear Algebra page 26 Matrices • Columns and Rows of a Matrix are Vectors • Addition and Subtraction • Multiplication by a scalar • Transpose • Linear Combinations of Vectors • Matrix–Vector Product • Matrix–Matrix Product • Matrix Norms NMM: A Review of Linear Algebra page 27 Notation The matrix A with m rows and n columns looks like: 2 3 a11 a12 ··· a1n 6 a a a 7 A = 6 21 22 2n 7 4 .