Linear Algebra II: linear combinations & matrices
Math Tools for Neuroscience (NEU 314) Fall 2016
Jonathan Pillow Princeton Neuroscience Institute & Psychology.
Lecture 3 (Thursday 9/22) accompanying notes/slides Linear algebra
“Linear algebra has become as basic and as applicable as calculus, and fortunately it is easier.” - Glibert Strang, Linear algebra and its applications today’s topics
• linear projection (review) • orthogonality (review)
• linear combination • linear independence / dependence • matrix operations: transpose, multiplication, inverse
Did not get to: • vector space • subspace • basis • orthonormal basis Linear Projection Exercise
w = [2,2] v1 = [2,1] v2 = [5,0]
Compute: Linear projection of w onto lines defined by v1 and v2 linear combination is clearly a vector space [verify]. • scaling and summing applied to a group of vectors
Working backwards, a set of vectors is said to span a vector space if one can write any v vector in the vector space as a linear com- 1 v3 bination of the set. A spanning set can be redundant: For example, if two of the vec- tors are identical, or are scaled copies of each other. This redundancy is formalized by defining linear• a independence group of vectors.Asetofvec- is linearly tors {⃗v1,⃗v2,...⃗vdependentM } is linearly independent if one can if be written as v2 (and only if) thea only linear solution combination to the equation of the others • otherwise,αn⃗vn =0 linearly independent !n is αn =0(for all n).
A basis for a vector space is a linearly in- dependent spanning set. For example, con- sider the plane of this page. One vector is not enough to span the plane. Scalar multi- v ples of this vector will trace out a line (which v v v 2 is a subspace), but cannot “get off the line” vv 2 1 v v to cover the rest of the plane. But two vec- 1 v v tors are sufficient to span the entire plane. 1 Bases are not unique: any two vectors will do, as long as they don’t lie along the same line. Three vectors are redundant: one can always be written as a linear combination of the other two. In general, the vector space N R requires a basis of size N.
e Geometrically, the basis vectors define a set of coordinate axes for the space (although e ˆ2 they need not be perpendicular). The stan- e dard basis is the set of unit vectors that lie along the axes of the space:
1 0 0 eˆ1 ⎛ 0 ⎞ ⎛ 1 ⎞ ⎛ 0 ⎞ eˆ1 = ⎜ 0 ⎟, eˆ2 = ⎜ 0 ⎟,...eˆN = ⎜ 0 ⎟. ⎜ . ⎟ ⎜ . ⎟ ⎜ . ⎟ ⎜ . ⎟ ⎜ . ⎟ ⎜ . ⎟ ⎜ ⎟ ⎜ ⎟ ⎜ ⎟ v ⎜ 0 ⎟ ⎜ 0 ⎟ ⎜ 1 ⎟ S x ⎝ ⎠ ⎝ ⎠ ⎝ ⎠ 5 x S e ) v x e x S x S e
x e v x S e S x) v x e x S x S e
x e
o o o o o
3
o o o o o
3 matrices
n × m matrix
can think of it as:
m column vectors n row vectors
r1 … c1 … cm or
One perspective: dot product with each row: matrix multiplication Other perspective: linear combination of columns
u1 v1 • •
c1 … cm • • • • •
un vm
= v1• c1 + v2• c2 + … + vm• cm transpose
• flipping around the diagonal
T 1 4 7 1 2 3 square 2 5 8 = 4 5 6 matrix 3 6 9 7 8 9 T 1 4 1 2 3 2 5 = 4 5 6 non-square 3 6
• transpose of a product inverse
• If A is a square matrix, its inverse A-1 (if it exists) obeys
• inverse of a product