Background Material A computer vision "encyclopedia": CVonline. http://homepages.inf.ed.ac.uk/rbf/CVonline/ Linear Algebra Review Linear Algebra: and Eero Simoncelli “A Geometric View of Linear Algebra” Matlab Tutorial http://www.cns.nyu.edu/~eero/NOTES/geomLinAlg.pdf Michael Jordan slightly more in depth linear algebra review Assigned Reading: http://www.cs.brown.edu/courses/cs143/Materials/linalg_jordan_86.pdf •Eero Simoncelli “A Geometric View of Linear Algebra” http://www.cns.nyu.edu/~eero/NOTES/geomLinAlg.pdf Online Introductory Linear Algebra Book by Jim Hefferon. http://joshua.smcvt.edu/linearalgebra/ Notation Overview 2 Standard math textbook notation Vectors in R Scalars are italic times roman: n, N Scalar product Vectors are bold lowercase: x Outer Product Row vectors are denoted with a transpose: xT Bases and transformations Matrices are bold uppercase: M Inverse Transformations Tensors are calligraphic letters: T Eigendecomposition Singular Value Decomposition Warm-up: Vectors in Rn Vectors in Rn We can think of vectors in two ways: Notation: Points in a multidimensional space with respect to some coordinate system translation of a point in a multidimensional space ex., translation of the origin (0,0) Length of a vector: n 2 2 2 2 x = x1 + x2 +Lxn = ∑ xi i=1 1 Dot product or scalar product Scalar Product Dot product is the product of two vectors Notation Example: x1 y1 x⋅ y = ⋅ = x y + x y = s x,y 1 1 2 2 y x2 y2 1 T x⋅y = x y = []x1 L xn M It is the projection of one vector onto another yn We will use the last two notations to denote the dot x ⋅y = x y cosθ product θ x.y Scalar Product Norms in Rn Commutative: x⋅y = y ⋅x Euclidean norm (sometimes called 2-norm): Distributive: ()x + y ⋅z = x⋅z + y ⋅z n 2 2 2 2 Linearity x = x = x⋅x = x + x + + x = x 2 1 2 L n ∑ i ()cx ⋅y = c (x⋅y ) i=1 x⋅()cy = c (x⋅y ) The length of a vector is defined to be its (Euclidean) norm. A unit vector is of length 1. (c1x)⋅(c2y) = (c1c2 )(x⋅y) Non-negativity properties also hold for the norm: Non-negativity: Orthogonality: ∀x ≠ 0,y ≠ 0 x⋅y = 0 ⇔ x ⊥ y Bases and Transformations Linear Dependence We will look at: Linear combination of vectors x1, x2, … xn Linear Independence Bases c1x1 + c2x2 +L+ cnxn Orthogonality Change of basis (Linear Transformation) A set of vectors X={x1, x2, … xn} are linearly dependent if there Matrices and Matrix Operations exists a vector xi ∈ X that is a linear combination of the rest of the vectors. 2 Linear Dependence Bases (Examples in R2) n In R sets of n+1vectors are always dependent there can be at most n linearly independent vectors Bases Bases n A basis is a linearly independent set of vectors that spans the Standard basis in R is made up of a set of unit vectors: “whole space”. ie., we can write every vector in our space as linear combination of vectors in that set. ˆ ˆ e1 e 2 eˆ n n n Every set of n linearly independent vectors in R is a basis of R A basis is called We can write a vector in terms of its standard basis: orthogonal, if every basis vector is orthogonal to all other basis vectors orthonormal, if additionally all basis vectors have length 1. eˆ 1 eˆ 2 eˆ 3 Observation: -- to find the coefficient for a particular basis vector, we project our vector onto it. xi = eˆi ⋅x Change of basis Outer Product m Suppose we have a new basis B = [] b 1 L b n , bi ∈R m and a vector x ∈ R that we would like to represent in x1 terms of B T b x o y = xy = M []y1 ym = M 2 ~x x x 2 xn ~ x2 b ~ 1 x1 A matrix M that is the outer product of two vectors is a matrix of rank 1. ~ −1 Compute the new components x = B x T b x When B is orthonormal 1 ~ ~ x is a projection of x onto bi x = M T Note the use of a dot product b x n 3 Matrix Multiplication – dot product Matrix Multiplication – outer product Matrix multiplication can be expressed using a sum of outer Matrix multiplication can be expressed using dot products products aT a a 1 1 L n BA = b1 L bn M BA = T T b1 b ⋅a b ⋅a 1 1 1 n an M O = b aT + b aT + b aT 1 1 2 2 L n n n b T m bm ⋅a1 bm ⋅an = ∑bi oai i=1 Rank of a Matrix Singular Value Decomposition: D=USVT = V D U S I1x I2 A matrix D ∈ R has a column space and a row space SVD orthogonalizes these spaces and decomposes D D= USV T ( U contains the left singular vectors/eigenvectors ) ( V contains the right singular vectors/eigenvectors ) Rewrite as a sum of a minimum number of rank-1 matrices R D = ∑ σ r u r o v r r =1 T Matrix SVD Properties: D=USV Matrix Inverse R Rank Decomposition: D= ∑ σ r ur vr r =1 o sum of min. number of rank-1 matrices σ1 +σ 2 ….. +σ R D = T T T v1 v2 vR u 1 u2 uR R1 R2 Multilinear Rank Decomposition: D= σ ur vr ∑ ∑ r1r 2 1 o 2 r1=1 r2 =1 = V D U S 4 Some matrix properties Matlab Tutorial 5.