Linear Algebra II: linear combinations & matrices Math Tools for Neuroscience (NEU 314) Fall 2016 Jonathan Pillow Princeton Neuroscience Institute & Psychology. Lecture 3 (Thursday 9/22) accompanying notes/slides Linear algebra “Linear algebra has become as basic and as applicable as calculus, and fortunately it is easier.” - Glibert Strang, Linear algebra and its applications today’s topics • linear projection (review) • orthogonality (review) • linear combination • linear independence / dependence • matrix operations: transpose, multiplication, inverse Did not get to: • vector space • subspace • basis • orthonormal basis Linear Projection Exercise w = [2,2] v1 = [2,1] v2 = [5,0] Compute: Linear projection of w onto lines defined by v1 and v2 linear combination is clearly a vector space [verify]. • scaling and summing applied to a group of vectors Working backwards, a set of vectors is said to span a vector space if one can write any v vector in the vector space as a linear com- 1 v3 bination of the set. A spanning set can be redundant: For example, if two of the vec- tors are identical, or are scaled copies of each other. This redundancy is formalized by defining linear• a independence group of vectors.Asetofvec- is linearly tors {⃗v1,⃗v2,...⃗vdependentM } is linearly independent if one can if be written as v2 (and only if) thea only linear solution combination to the equation of the others • otherwise,αn⃗vn =0 linearly independent !n is αn =0(for all n). A basis for a vector space is a linearly in- dependent spanning set. For example, con- sider the plane of this page. One vector is not enough to span the plane. Scalar multi- v ples of this vector will trace out a line (which v v v 2 is a subspace), but cannot “get off the line” vv 2 1 v v to cover the rest of the plane. But two vec- 1 v v tors are sufficient to span the entire plane. 1 Bases are not unique: any two vectors will do, as long as they don’t lie along the same line. Three vectors are redundant: one can always be written as a linear combination of the other two. In general, the vector space N R requires a basis of size N. e Geometrically, the basis vectors define a set of coordinate axes for the space (although e ˆ2 they need not be perpendicular). The stan- e dard basis is the set of unit vectors that lie along the axes of the space: 1 0 0 eˆ1 ⎛ 0 ⎞ ⎛ 1 ⎞ ⎛ 0 ⎞ eˆ1 = ⎜ 0 ⎟, eˆ2 = ⎜ 0 ⎟,...eˆN = ⎜ 0 ⎟. ⎜ . ⎟ ⎜ . ⎟ ⎜ . ⎟ ⎜ . ⎟ ⎜ . ⎟ ⎜ . ⎟ ⎜ ⎟ ⎜ ⎟ ⎜ ⎟ v ⎜ 0 ⎟ ⎜ 0 ⎟ ⎜ 1 ⎟ S x ⎝ ⎠ ⎝ ⎠ ⎝ ⎠ 5 x S e ) v x e x S x S e x e v x S e S x) v x e x S x S e x e o o o o o 3 o o o o o 3 matrices n × m matrix can think of it as: m column vectors n row vectors r1 … c1 … cm or rn matrix multiplication One perspective: dot product with each row: matrix multiplication Other perspective: linear combination of columns u1 v1 • • c1 … cm • • • • un vm = v1• c1 + v2• c2 + … + vm• cm transpose • flipping around the diagonal T 1 4 7 1 2 3 square 2 5 8 = 4 5 6 matrix 3 6 9 7 8 9 T 1 4 1 2 3 2 5 = 4 5 6 non-square 3 6 • transpose of a product inverse • If A is a square matrix, its inverse A-1 (if it exists) obeys • inverse of a product.