Linear Algebra Review Linear Algebra Primer Slides by Juan Carlos Niebles*, Ranjay Krishna* and Maxime Voisin *Stanford Vision and Learning Lab 27 - Sep - 2018 Another, very in-depth linear algebra review from CS229 is available here: http://cs229.stanford.edu/section/cs229-linalg.pdf And a video discussion of linear algebra from EE263 is here (lectures 3 and 4): https://see.stanford.edu/Course/EE263 1 Stanford University Outline Linear Algebra Review • Vectors and matrices – Basic Matrix Operations – Determinants, norms, trace – Special Matrices • Transformation Matrices – Homogeneous coordinates – Translation • Matrix inverse 27 - • Matrix rank Sep - • Eigenvalues and Eigenvectors 2018 • Matrix Calculus 2 Stanford University Outline Vectors and matrices are just Linear Algebra Review • Vectors and matrices collections of ordered numbers – Basic Matrix Operations that represent something: location in space, speed, pixel brightness, – Determinants, norms, trace etc. We’ll define some common – Special Matrices uses and standard operations on • Transformation Matrices them. – Homogeneous coordinates – Translation • Matrix inverse 27 - • Matrix rank Sep - • Eigenvalues and Eigenvectors 2018 • Matrix Calculus 3 Stanford University Vector Linear Algebra Review • A column vector where • A row vector where 27 - Sep - 2018 denotes the transpose operation 4 Stanford University Vector • We’ll default to column vectors in this class Linear Algebra Review 27 - Sep - • You’ll want to keep track of the orientation of your 2018 vectors when programming in python 5 Stanford University Some vectors have a geometric interpretation, others don’t… Linear Algebra Review • Other vectors don’t have a • Some vectors have a geometric interpretation: geometric – Vectors can represent any kind of interpretation: 27 data (pixels, gradients at an - – Points are just vectors image keypoint, etc) Sep - from the origin. – Such vectors don’t have a 2018 – We can make geometric interpretation calculations like “distance” between 2 – We can still make calculations like vectors “distance” between 2 vectors 6 Stanford University Matrix Linear Algebra Review • A matrix is an array of numbers with size m by n, i.e. m rows and n columns. 27 - Sep - 2018 • If , we say that is square. 7 Stanford University 7 Images = Linear Algebra Review • Python represents an image as a matrix of pixel brightnesses 27 - • Note that the upper left corner is [x, y] = (0,0) Sep - 2018 Python indices row column start at 0 8 Stanford University 8 Images can be represented as a matrix of pixels. Images can also be represented as a vector of pixels Linear Algebra Review 27 - Sep - 2018 9 Stanford University Color Images Linear Algebra Review • Grayscale images have 1 number per pixel, and are stored as an m×n matrix. • Color images have 3 numbers per pixel – red, green, and blue brightnesses (RGB) - and are stored as an m×n×3 matrix 27 - Sep - = 2018 10 Stanford University Basic Matrix Operations Linear Algebra Review • We will discuss: – Addition – Scaling – Dot product – Multiplication – Transpose 27 - Sep - – Inverse / pseudoinverse 2018 – Determinant / trace 11 Stanford University Matrix Operations • Addition Linear Algebra Review – We can only add a matrix with matching dimensions, or a scalar. Good to know for Python assignments J 27 - Sep - 2018 • Scaling 12 Stanford University Linear Algebra Review 13 27-Sep-2018 Examples of vector norms • Vector Norms Stanford University Vector Norms Linear Algebra Review • More formally, a norm is any function that satisfies the following 4 properties: • Non-negativity: For all • Definiteness: f(x) = 0 if and only if x = [0,0…0]. 27 - • Homogeneity: For all Sep - • Triangle inequality: For all 2018 14 Stanford University Vector Operation: inner product Linear Algebra Review • Inner product (dot product) of two vectors – Multiply corresponding entries of two vectors and add up the result – x·y is also |x||y|cos( the angle between x and y ) 27 - Sep - 2018 15 Stanford University Vector Operation: inner product Linear Algebra Review • Inner product (dot product) of two vectors –If B is a unit vector: • Then A·B = |A||B|cos(Θ) = |A|x 1 x cos(Θ) = |A|cos(Θ) • A·B gives tHe lengtH of A whicH lies in tHe direction of B 27 - Sep - 2018 16 Stanford University Linear Algebra Review 27-Sep-2018 17 product of two matrices The Stanford University • Matrix Operations Matrix Operations • Multiplication Linear Algebra Review • The product AB is: 27 - Sep - • EacH entry in tHe result is: 2018 (tHat row of A) dot product witH (tHat column of B) • Many uses, which will be covered later 18 Stanford University Matrix Operations • Multiplication example: Linear Algebra Review – Each entry of the matrix product is made by taking: the dot product of the corresponding row in the left matrix, with the corresponding column in the right one. 27 - Sep - 2018 19 Stanford University Linear Algebra Review 27-Sep-2018 20 The product of two matrices Stanford University • Matrix Operations Matrix Operations Linear Algebra Review • Powers – By convention, we can refer to the matrix product AA as A2, and AAA as A3, etc. – Important: only square matrices can be multiplied tHat way! (make sure you understand why) 27 - Sep - 2018 21 Stanford University Matrix Operations Linear Algebra Review • Transpose a matrix: flip the matrix, so row 1 becomes column 1 27 - • A useful identity: Sep - 2018 22 Stanford University Matrix Operations • Determinant Linear Algebra Review – returns a scalar – Represents area (or volume) of the parallelogram described by the vectors in the rows of the matrix – For , – Properties: 27 - Sep - 2018 23 Stanford University Matrix Operations Linear Algebra Review • Trace – Invariant to a lot of transformations, so it’s used sometimes in proofs. (Rarely in this class though.) 27 – Properties: - Sep - 2018 24 Stanford University Matrix Operations Linear Algebra Review • Vector Norms (we’ve talked about tHem earlier) 27 - Sep • Matrix norms: Norms can also be defined for - 2018 matrices, such as the Frobenius norm: 25 Stanford University Special Matrices Linear Algebra Review • Identity matrix I Square matrix, 1’s along diagonal, 0’s elsewhere I · [a matrix A] = [that matrix A] [a matrix A] · I = [that matrix A] • Diagonal matrix Square matrix with numbers along diagonal, 0’s 27 - elsewhere Sep - 2018 [diagonal matrix A] · [another matrix B] scales the rows of matrix B 26 Stanford University Special Matrices Linear Algebra Review • Symmetric matrix • Skew-symmetric matrix 0 2 5 20− −7 2 − 3 57 0 27 - 4 5 Sep - 2018 27 Stanford University Outline Linear Algebra Review • Vectors and matrices – Basic Matrix Operations – Determinants, norms, trace Matrix multiplication can be – Special Matrices used to transform vectors. • Transformation Matrices A matrix used in this way is – Homogeneous coordinates called a transformation – Translation • Matrix inverse matrix. 27 - • Matrix rank Sep - • Eigenvalues and Eigenvectors 2018 • Matrix Calculus 28 Stanford University Transformation: scaling Linear Algebra Review • Matrices can be used to transform vectors in useful ways, through multiplication: Ax = x’ • Simplest transformation is scaling: 27 - Sep - (Verify to yourself that the matrix multiplication works out this way) 2018 Scaling Initial Scaled 29 Stanford Universitymatrix vector vector Transformation: scaling Linear Algebra Review Scaled vector: Initial vector: 27 - Sep - Multiply by the 2018 scaling matrix: 30 Stanford University Transformation: rotation Linear Algebra Review Rotated vector: Initial ° vector: � = 30 27 - Sep - 2018 31 Stanford University Before learning how to rotate a vector, let’s understand how to do something slightly different • How can you convert a vector represented in frame “0” to a new, Linear Algebra Review rotated coordinate frame “1”? y0 yy00 y1 x1 27 - Sep - 2018 x0 x0 32 Stanford UniversityIt is the same data point, but it is represented in a new, rotated frame. Before learning how to rotate a vector, let’s understand how to do something slightly different • How can you convert a vector represented in frame “0” to a new, Linear Algebra Review rotated coordinate frame “1”? y0 yy00 y1 x1 y 27 - Sep - 2018 This is y This is x coordinate in coordinate frame “0” in frame “0” x x0 x0 33 Stanford UniversityIt is the same data point, but it is represented in a new, rotated frame. Before learning how to rotate a vector, let’s understand how to do something slightly different • How can you convert a vector represented in frame “0” to a new, Linear Algebra Review rotated coordinate frame “1”? y0 yy00 y1 x1 y 27 - This is x Sep This is x - x' coordinate 2018 This is y coordinate in frame coordinate in frame “0” This is y “1” in frame coordinate in y' “0” x x0 frame “1” x0 34 Stanford UniversityIt is the same data point, but it is represented in a new, rotated frame. Before learning how to rotate a vector, let’s understand how to do something slightly different • How can you convert a vector represented in frame “0” to a new, Linear Algebra Review rotated coordinate frame “1”? • Remember what a vector is: [component in direction of the frame’s x axis, component in direction of y axis] y0 yy00 y1 x1 y 27 - This is x Sep - x' coordinate 2018 This is y in frame coordinate “1” in frame y' “0” x x0 x0 This is x coordinate in This is y coordinate in 35 Stanford University frame “0” frame “1” Before learning how to rotate a vector, let’s understand how to do something slightly different • How can you convert a vector represented in frame “0” to a new,