sign in sign up
<<
Home , Euclidean vector

What is a vector?

Linear in a Nutshell: in a Nutshell: ñ A (finite) list of real values is called a vector: Vector Spaces Vector Spaces

S. Evert S. Evert x~ = (x1, x2, . . . , xn) Introduction Linear Algebra in a Nutshell Introduction What is a vector? What is a vector? Examples Part 1: Vector Spaces Examples ñ x1, . . . , xn are the components of x~ Vector spaces Vector spaces ñ n is also referred to as dimensionality of x~ Geometric Geometric interpretation interpretation Formal definition Formal definition Techniques Stefan Evert Techniques ñ Vectors are used as a mathematical representation of Linear algebra Linear algebra the features of an object (or the state of a system) Linear combination & coordinates Institute of Cognitive Science Basis & coordinates ñ specimen of a plant (size, , colour, . . . ) Linear subspaces University of Osnabrück, Germany Linear subspaces algebra Matrix algebra ñ car model (weight, horsepower, miles per gallon, . . . ) Practice with R [email protected] Practice with R ñ Volume & Volume & determinant word (formal and distributional features) Linear maps Linear maps ñ sentence (synactic structure coded as features) Equation systems Rovereto, 19 March 2007 Equation systems Coordinate Coordinate ñ transformations transformations text (categorisation based on features) Hyperplanes Hyperplanes

You did it! You did it!

What is a vector? Flower-spotting: The Iris data set Examples of vector representations

Linear Algebra Linear Algebra in a Nutshell: ñ Numerical features: in a Nutshell: ñ Recognise flowers by shape of Vector Spaces Vector Spaces ñ continuous vs. discrete values their petals and sepals S. Evert S. Evert sepal ñ Categorical features: ñ quantified by width & length Introduction Introduction ñ What is a vector? finite set of non-numerical values (e.g. colours, POS) What is a vector? Examples ñ Examples binary features (yes/no = 1/0) x~ = (lp, wp, ls, ws) width Vector spaces Vector spaces ñ multiple categories can be coded as set of length Geometric Geometric interpretation binary indicator features (e.g. blue?, red?) interpretation lp . . . length of petal Formal definition Formal definition Techniques Techniques w . . . width of petal ñ Focus on real-valued vector spaces (implicit p petal Linear algebra Linear algebra ls . . . length of sepal Linear combination assumption: continuous data), but we will also Linear combination Basis & coordinates Basis & coordinates ws . . . width of sepal Linear subspaces encode discrete and categorical data Linear subspaces Matrix algebra Matrix algebra ñ One can distinguish between different species of iris Practice with R Practice with R Volume & determinant Volume & determinant flowers based on the shape of their petals and sepals Linear maps ñ The set of all possible vectors of length n is the Linear maps Equation systems n Equation systems ñ Coordinate n-dimensional Euclidean R Coordinate x~ describes the shape of a particular specimen transformations transformations Hyperplanes (or a subspace if there are constraints on the values) Hyperplanes ñ Anderson, E. (1935). The irises of the Gaspe Peninsula. You did it! You did it! Bulletin of the American Iris Society, 59, 2–5. Recognition of handwritten numbers Text categorisation & register variation Examples of vector representations Examples of vector representations

Linear Algebra Linear Algebra in a Nutshell: ñ Recognize handwritten digits (e.g. ZIP codes) in a Nutshell: ñ Texts can be represented by content-oriented (lexical) Vector Spaces Vector Spaces ñ machine learning: classifier for handwritten numbers or form-oriented (morpho-syntactical) features S. Evert S. Evert ñ digits may be shifted, rotated, blurred, cropped Introduction ñ input data: 16 × 16 grayscale image Introduction ñ Text categorisation e.g. for filtering e-Mail spam What is a vector? What is a vector? Examples ñ Vector representation of image Examples ñ mostly content-oriented features: presence of Vector spaces Vector spaces keywords (Viagra, lose weight, Russian girls) Geometric Geometric interpretation ~ = interpretation ñ also relative frequencies of part-of-speech (POS) Formal definition x (p1, p2, . . . , p256) Formal definition Techniques Techniques n-grams, HTML tags, special characters (Vi@gr@), etc. Linear algebra Linear algebra pi ∈ [0, 1] . . . grey level of i-th pixel ñ Linear combination Linear combination Goal: accurate classification of incoming e-mails Basis & coordinates Basis & coordinates Linear subspaces Linear subspaces as spam vs. ham (legitimate e-mails) Matrix algebra Matrix algebra Practice with R Practice with R Volume & determinant Volume & determinant Linear maps Linear maps Equation systems Equation systems Coordinate Coordinate transformations transformations Hyperplanes Hyperplanes

You did it! You did it!

Text categorisation & register variation Word space & distributional similarity Examples of vector representations Examples of vector representations

Linear Algebra Linear Algebra in a Nutshell: ñ Speakers use different registers of language in a Nutshell: ñ Vector representation of contexts in which a certain Vector Spaces Vector Spaces depending on the communicative situation (genre) word or phrase occurs, e.g. bucket (n.) S. Evert S. Evert ñ written: term paper / formal letter / letter to a friend ñ Vector x~ = cooccurrence frequencies of typical Introduction ñ spoken: conference talk / party conversation Introduction What is a vector? What is a vector? collocates of the keyword Examples Examples ñ Alternative representation: occurrence frequencies of Vector spaces ñ Vector representation of texts for studies of register Vector spaces Geometric Geometric word in different documents (e.g. Wikipedia articles) interpretation variation uses features such as: interpretation Formal definition Formal definition Techniques ñ frequency of passive verbs, nouns, etc. (§ formal) Techniques noun f local MI verb f local MI adjective f local MI water 183 1023.77 throw 36 168.87 large 37 114.79 Linear algebra ñ Linear algebra spade 31 288.11 fill 30 139.45 single-record 5 64.53 frequency of active verbs, 1st person, etc. (§ informal) plastic 36 225.83 empty 14 96.73 full 21 63.23 Linear combination Linear combination size 41 195.89 randomize 9 96.11 cold 13 55.52 ñ Basis & coordinates frequency of different semantic classes of verbs & Basis & coordinates record 38 163.95 hold 31 78.93 small 21 45.61 slop 14 162.62 put 37 77.96 galvanized 4 43.47 Linear subspaces Linear subspaces mop 16 155.47 carry 26 71.95 ten-record 3 40.17 Matrix algebra adjectives (evaluative, experiential, etc.) Matrix algebra ice 22 125.76 tip 10 59.30 empty 9 38.41 Practice with R Practice with R bucket 18 125.49 kick 12 59.28 old 20 35.67 ñ use of native vs. Latinate roots (§ learned) seat 21 89.21 chuck 7 44.85 steaming 4 31.89 Volume & determinant Volume & determinant coal 16 77.25 use 31 42.31 clean 7 27.47 Linear maps Linear maps density 11 63.64 weep 7 41.73 leaky 3 25.91 ñ brigade 10 62.31 pour 9 40.73 wooden 6 25.50 Equation systems Goal is not classification but identifying the major Equation systems sand 12 61.32 take 42 37.57 bottomless 3 25.17 Coordinate Coordinate algorithm 9 60.77 fetch 7 35.13 galvanised 3 24.70 transformations dimensions of register variation transformations shop 17 59.49 get 46 34.73 big 12 23.86 Hyperplanes Hyperplanes container 10 59.10 douse 4 33.03 iced 3 22.62 champagne 10 56.79 store 7 31.82 warm 6 19.55 shovel 7 56.50 drop 10 31.49 hot 6 17.05 You did it! You did it! oats 7 54.93 pick 11 28.89 pink 3 11.15

collocational “profile” of bucket (n.) Why vector spaces? The geometric interpretation of vectors Vectors as points

Linear Algebra Linear Algebra in a Nutshell: ñ Vector spaces encode basic geometric intuitions in a Nutshell: ñ vectors like u~ = (4, 2) Vector Spaces Vector Spaces x2 geometric interpretation of numerical feature lists and v~ = (3, 5) can be S. Evert S. Evert v = (3, 5) one reason why linear algebra is so useful 6 understood as the Introduction Introduction What is a vector? What is a vector? 5 n coordinates of points Examples ñ Interpretation of vectors x,~ y,~ . . . ∈ R as points in Examples in the Euclidean Vector spaces n-dimensional Euclidean (= intuitive) space Vector spaces 4 Geometric Geometric interpretation interpretation ñ alternative notation: ñ n = 2 § Euclidean plane 3 Formal definition Formal definition u = (4, 2) Techniques ñ n = 3 § three-dimensional Techniques u instead of u~ 2 Linear algebra Linear algebra ñ Linear combination Linear combination in this interpretation, Basis & coordinates ñ Exploit geometric intuition for analysis of data set as Basis & coordinates 1 Linear subspaces Linear subspaces vectors refer to Matrix algebra group of points or arrows in Euclidean space Matrix algebra specific locations Practice with R Practice with R ñ Volume & determinant , length, direction, , dimension, . . . Volume & determinant 1 2 3 4 5 6 x1 Linear maps Linear maps ñ 2 3 Equation systems intuitive in R and R Equation systems Coordinate ñ Coordinate transformations can be generalised to higher dimensions transformations Hyperplanes may refer to objects in a data set as “data points” Hyperplanes You did it! You did it!

The geometric interpretation of vectors The geometric interpretation of vectors Vectors as arrows & vector Vectors as arrows

Linear Algebra Linear Algebra in a Nutshell: ñ vectors can also be in a Nutshell: ñ vectors as arrows are Vector Spaces x2 Vector Spaces x2 interpreted as -independent S. Evert v = (3, 5) S. Evert y-x = v-u 6 6 “ arrows” ñ y~ − x~ = v~ − u~ if the y = (4, 6.5) = (-1, 3) Introduction Introduction What is a vector? between points 5 What is a vector? relative positions of x~ 5 Examples v-u = (-1, 3) Examples Vector spaces ñ the arrow from u~ to v~ 4 Vector spaces and y~ are the same as 4 Geometric Geometric interpretation is described by the interpretation those of u~ and v~ 3 3 Formal definition u = (4, 2) Formal definition Techniques vector (−1, 3) Techniques ñ regardless of location x = (6, 3.5) 2 2 Linear algebra ñ calculated as Linear algebra in plane Linear combination Linear combination Basis & coordinates pointwise difference 1 Basis & coordinates 1 Linear subspaces Linear subspaces Matrix algebra between components Matrix algebra Practice with R Practice with R Volume & determinant of v~ and u~: v~ − u~ = 1 2 3 4 5 6 x1 Volume & determinant 1 2 3 4 5 6 x1 Linear maps Linear maps Equation systems (v1 − u1, v2 − u2) Equation systems Coordinate Coordinate transformations transformations Hyperplanes ñ general : Hyperplanes You did it! vector addition You did it! The geometric interpretation of vectors The geometric interpretation of vectors Direction & Linking points and arrows

Linear Algebra Linear Algebra in a Nutshell: ñ intuitively, arrows in a Nutshell: ñ points in the plane Vector Spaces x2 Vector Spaces x2 have a length and can be represented by S. Evert S. Evert (3, 5) direction 6 displacement arrows 6 Introduction Introduction What is a vector? ñ two arrows in 5 What is a vector? from a fixed reference 5 Examples Examples point Vector spaces the same direction iff 4 u Vector spaces 4 Geometric 2u Geometric interpretation they are multiples of interpretation ñ natural reference 3 3 Formal definition Formal definition (4, 2) Techniques each other: scalar Techniques point is the 2 2 Linear algebra multiplication Linear algebra 0~ = (0, 0) Linear combination -u Linear combination Basis & coordinates λu~ = (λu1, λu2) with 1 Basis & coordinates ñ these arrows are 1 Linear subspaces Linear subspaces Matrix algebra constant factor λ ∈ R Matrix algebra given by the same Practice with R Practice with R Volume & determinant ñ for λ < 0, arrows have 1 2 3 4 5 6 x1 Volume & determinant vectors as the point 1 2 3 4 5 6 x1 Linear maps Linear maps Equation systems opposite directions Equation systems coordinates Coordinate Coordinate origin transformations transformations (0,0) Hyperplanes ñ −u~ = (−1) · u~ is the Hyperplanes You did it! inverse arrow of u~ You did it!

The n-dimensional Euclidean space The axioms of a general vector space

Linear Algebra n Linear Algebra in a Nutshell: ñ The n-dimensional real space R is in a Nutshell: ñ General vector space over the real numbers R Vector Spaces Vector Spaces the set of all real-valued vectors x~ = (x1, . . . , xn) of = set V of vectors u~ ∈ V with operations S. Evert S. Evert length n, allowing the following operations: ñ u~ + v~ for u,~ v~ ∈ V (addition) Introduction ñ vector addition: u~ + v~ Í (u1 + v1, . . . , un + vn) Introduction ñ λu~ for λ ∈ R, u~ ∈ V () What is a vector? What is a vector? ñ scalar multiplication: λu~ Í (λu , . . . , λu ) for λ ∈ R Examples 1 n Examples ñ Addition and s-multiplication must satisfy the axioms Vector spaces Vector spaces ñ Important properties of the addition and 1. (u~ + v)~ + w~ = u~ + (v~ + w)~ Geometric n Geometric interpretation s-multiplication operations in R interpretation ~ ~ Formal definition Formal definition 2. u~ + 0 = 0 + u~ = u~ Techniques 1. (u~ + v)~ + w~ = u~ + (v~ + w)~ Techniques 3. ∀u~ ∃u~0 : u~ + u~0 = u~0 + u~ = 0~ Linear algebra 2. u~ + 0~ = 0~ + u~ = u~ Linear algebra 4. u~ + v~ = v~ + u~ Linear combination ~ Linear combination Basis & coordinates 3. ∀u~ ∃(−u)~ : u~ + (−u)~ = (−u)~ + u~ = 0 Basis & coordinates 5. (λ + µ)u~ = λu~ + µu~ Linear subspaces + = + Linear subspaces Matrix algebra 4. u~ v~ v~ u~ Matrix algebra 6. (λµ)u~ = λ(µu)~ Practice with R 5. (λ + µ)u~ = λu~ + µu~ Practice with R · = Volume & determinant Volume & determinant 7. 1 u~ u~ Linear maps 6. (λµ)u~ = λ(µu)~ Linear maps 8. λ(u~ + v)~ = λu~ + λv~ Equation systems Equation systems Coordinate 7. 1 · u~ = u~ Coordinate transformations transformations for any u,~ v,~ w~ ∈ V and λ, µ ∈ R Hyperplanes 8. λ(u~ + v)~ = λu~ + λv~ Hyperplanes ñ 0~ is the unique neutral element of V, You did it! for any u,~ v,~ w~ ∈ Rn and λ, µ ∈ R You did it! and the unique inverse u~0 of u~ is often written as −u~ Further properties of vector spaces Vector space techniques & applications

Linear Algebra Linear Algebra in a Nutshell: ñ Further properties of vector spaces: in a Nutshell: Vector Spaces Vector Spaces ñ 0 · u~ = 0~ ñ clustering The Iris data set S. Evert S. Evert ñ ~ ~

λ0 = 0 based on = 3.0 Introduction ñ λu~ = 0~ ⇒ λ = 0 ∨ u~ = 0~ Introduction similarity of features ● ●●

What is a vector? What is a vector? 2.5 ● ● ñ − = − = − Î − ●●●● ● ● ● ● Examples ( λ)u~ λ( u)~ (λu)~ λu~ Examples [2 major clusters] ● ● ● ●●●● ● ● ●●●● ● ●

Vector spaces n Vector spaces 2.0 ●● ● ● ñ It is easy to show these properties for R , but they also ñ classification ●● ● ●● ● ● ● Geometric Geometric ● ● ● ● ● ● interpretation interpretation supervised learning ● ●●● ●●● follow directly from the general axioms § must hold 1.5 ● ● ●●● ● Formal definition Formal definition ● ●●●●●●● Techniques Techniques ●● ● ● ●

distinctive features petal width ● ●● for all vector spaces ● ● ● ●● Linear algebra Linear algebra [linear classifiers] 1.0 Linear combination Linear combination ● ●

Basis & coordinates ñ Basis & coordinates 0.5 ● ●●● ● A non-trivial example: vector space C[a, b] of ñ main dimensions ●●● ● Linear subspaces Linear subspaces ● ●●●●●● ● ● ●● Matrix algebra continuous real functions over the interval [a, b] Matrix algebra transform to axes of Practice with R Practice with R 0.0 Volume & determinant ñ Volume & determinant largest variation vector addition: ∀f , g ∈ C[a, b], 0 1 2 3 4 5 6 7 Linear maps Linear maps Equation systems we define f + g by (f + g)(x) Í f (x) + g(x) Equation systems [size vs. shape] Coordinate Coordinate petal length transformations ñ s-multiplication: ∀λ ∈ R and ∀f ∈ C[a, b], transformations Hyperplanes we define λf by (λf )(x) Í λ · f (x) Hyperplanes You did it! You did it! show that C[a, b] satisfies vector space axioms

Vector space techniques & applications Vector space techniques & applications

Linear Algebra Linear Algebra in a Nutshell: in a Nutshell: Vector Spaces Vector Spaces ñ clustering The Iris data set ñ clustering The Iris data set S. Evert S. Evert

based on distances = 3.0 based on distances = 3.0 ● I. setosa Introduction similarity of features ● I. versicolor Introduction similarity of features ● I. virginica ● ●● ● ●●

What is a vector? 2.5 ● ● What is a vector? 2.5 ● ● ●●●● ● ● ● ● ●●●● ● ● ● ● Examples [2 major clusters] ● ● ● Examples [2 major clusters] ● ● ● ●●●● ● ● ●●●● ● ● ●●●● ● ● ●●●● ● ●

Vector spaces 2.0 ●● ● ● Vector spaces 2.0 ●● ● ● ñ classification ●● ● ●● ● ● ● ñ classification ●● ● ●● ● ● ● Geometric ● ● Geometric ● ● ● ● ● ● ● ● ● ● interpretation interpretation supervised learning ● ●●● ●●● supervised learning ● ●●● ●●●

1.5 ● ● ●●● ● 1.5 ● ● ●●● ● Formal definition ● ●●●●●●● Formal definition ● ●●●●●●● Techniques ●● ● ● ● Techniques ●● ● ● ●

distinctive features petal width ● ●● distinctive features petal width ● ●● ● ● ● ●● ● ● ● ●● Linear algebra [linear classifiers] 1.0 Linear algebra [linear classifiers] 1.0 Linear combination ● Linear combination ● ● ●

Basis & coordinates 0.5 ● ●●● ● Basis & coordinates 0.5 ● ●●● ● ñ main dimensions ●●● ● ñ main dimensions ●●● ● Linear subspaces ● ●●●●●● ● Linear subspaces ● ●●●●●● ● ● ●● ● ●● Matrix algebra transform to axes of Matrix algebra transform to axes of Practice with R 0.0 Practice with R 0.0 Volume & determinant largest variation Volume & determinant largest variation 0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7 Linear maps Linear maps Equation systems [size vs. shape] Equation systems [size vs. shape] Coordinate petal length Coordinate petal length transformations transformations Hyperplanes Hyperplanes

You did it! You did it! Clustering vs. classification Clustering vs. classification

Linear Algebra Linear Algebra in a Nutshell: in a Nutshell: Vector Spaces Clustering Vector Spaces Clustering

S. Evert S. Evert

Introduction Introduction What is a vector? What is a vector? Examples Examples

Vector spaces Vector spaces Geometric Geometric interpretation interpretation Formal definition Formal definition Techniques Techniques

Linear algebra Linear algebra Linear combination Linear combination Basis & coordinates Basis & coordinates Linear subspaces Linear subspaces Matrix algebra Matrix algebra Practice with R Practice with R Volume & determinant Volume & determinant ? Linear maps Linear maps Equation systems Equation systems Coordinate Coordinate transformations transformations Hyperplanes Hyperplanes

You did it! You did it!

Clustering vs. classification Linear algebra

Linear Algebra Linear Algebra in a Nutshell: in a Nutshell: Overview of basic topics in linear algebra Vector Spaces Classification Vector Spaces

S. Evert S. Evert ñ linear combinations & independence ñ Introduction Introduction basis & coordinates What is a vector? What is a vector? Examples Examples ñ linear subspaces

Vector spaces Vector spaces ñ Geometric Geometric matrix algebra interpretation interpretation Edible Formal definition Formal definition ñ linear algebra on the Techniques Techniques

Linear algebra Linear algebra ñ volume & determinant Linear combination Linear combination Basis & coordinates Basis & coordinates ñ linear maps Linear subspaces Linear subspaces Matrix algebra Matrix algebra ñ Practice with R Practice with R linear equation systems Volume & determinant Volume & determinant Linear maps Linear maps ñ coordinate transformations Equation systems Equation systems Coordinate Coordinate transformations Inedible transformations Hyperplanes Hyperplanes

You did it! You did it! Linear combinations & dimensionality Linear combinations & dimensionality

Linear Algebra Linear Algebra in a Nutshell: ñ linear combination of vectors u~(1),..., u~(n): in a Nutshell: ñ The largest number n ∈ N for which there exists a set Vector Spaces Vector Spaces of n linearly independent vectors u~(i) ∈ V is called the S. Evert (1) (2) (n) S. Evert λ1u~ + λ2u~ + · · · + λnu~ dimension dim V of V Introduction Introduction n What is a vector? What is a vector? ñ It can be shown that dim R = n Examples for any coefficients λ1, . . . , λn ∈ R Examples Vector spaces ñ intuition: all vectors that can be constructed from Vector spaces Geometric (1) (n) Geometric ñ If there is no maximal number of linearly independent interpretation u~ ,..., u~ using the basic vector operations interpretation Formal definition Formal definition vectors, the vector space is infinite-dimensional Techniques Techniques (dim V = ∞) Linear algebra ñ u~(1),..., u~(n) are linearly independent iff Linear algebra Linear combination Linear combination Basis & coordinates Basis & coordinates ñ An example is dim C[a, b] = ∞ (easy to show) Linear subspaces (1) (2) (n) ~ Linear subspaces Matrix algebra λ1u~ + λ2u~ + · · · + λnu~ = 0 Matrix algebra Practice with R Practice with R Volume & determinant Volume & determinant ñ Every finite-dimensional vector space V is isomorphic Linear maps implies λ = λ = · · · = λ = 0 Linear maps n Equation systems 1 2 n Equation systems to the Euclidean space R (with n = dim V) Coordinate Coordinate transformations transformations We will focus on Rn from now on Hyperplanes ñ otherwise, they are linearly dependent Hyperplanes You did it! ñ intuition: if vectors are linearly dependent, the neutral You did it! element 0~ can be constructed in different ways

Basis & coordinates Basis & coordinates

Linear Algebra Linear Algebra (1) (n) in a Nutshell: ñ A set of vectors b~ ,..., b~ ∈ V is called a basis of V in a Nutshell: ñ The unique coefficients x1, . . . , xn are called the Vector Spaces Vector Spaces iff every u~ ∈ V can be written as a linear combination coordinates of u~ wrt. the basis B Í b~(1),..., b~(n): S. Evert S. Evert

Introduction (1) (2) (n) Introduction   u~ = x1b~ + x2b~ + · · · + xnb~ x1 What is a vector? What is a vector?   Examples Examples x2  Vector spaces Vector spaces u~ ≡   Î x~ with unique coefficients x1, . . . , xn B  .  Geometric Geometric  .  interpretation ñ Number of vectors in a basis = dim V interpretation   Formal definition Formal definition x Techniques Techniques n

Linear algebra Linear algebra Linear combination ñ For every n-dimensional vector space V, Linear combination Basis & coordinates a set of n vectors b~(1),..., b~(n) ∈ V is a basis of V Basis & coordinates Linear subspaces Linear subspaces ñ x~ ∈ Rn is the of u~ ∈ V wrt. B Matrix algebra iff the vectors are linearly independent Matrix algebra Practice with R Practice with R n Volume & determinant Can you prove this? Volume & determinant V is isomorphic to R by virtue of this correspondence Linear maps Linear maps Equation systems Equation systems Coordinate Coordinate transformations transformations Hyperplanes Hyperplanes

You did it! You did it! Basis & coordinates Basis & coordinates

Linear Algebra Linear Algebra in a Nutshell: ñ The components (u1, u2, . . . , un) of a number vector in a Nutshell: 2 Vector Spaces Vector Spaces ñ u~ = (4, 5) ∈ R u~ ∈ Rn correspond to its natural coordinates S. Evert S. Evert ñ basis B of R2: x2   Introduction Introduction u1 (1) u=(4,5) What is a vector?   What is a vector? b~ = (2, 1) 6 Examples u2  Examples u~ = (u , u , . . . , u ) ≡   (2) 5 Vector spaces 1 2 n E  .  Vector spaces b~ = (−1, 1) Geometric  .  Geometric interpretation interpretation   4 Formal definition un Formal definition " # Techniques Techniques 3 ñ ≡ 3 Linear algebra Linear algebra u~ B (1) (n) n 2 Linear combination according to the e~ ,..., e~ of R : Linear combination Basis & coordinates Basis & coordinates ñ standard basis: 2 Linear subspaces Linear subspaces e(2) Matrix algebra (1) Matrix algebra 1 Practice with R e~ = (1, 0,..., 0) Practice with R (1) Volume & determinant Volume & determinant e~ = (1, 0) (2) Linear maps e~ = (0, 1,..., 0) Linear maps Equation systems Equation systems (2) x (2) b (1)1 2 3 4 5 6 1 Coordinate Coordinate e~ = (0, 1) e transformations . transformations b(1) Hyperplanes . Hyperplanes " # You did it! (n) You did it! 4 e~ = (0, 0,..., 1) ñ u~ ≡ E 5

Linear subspaces Linear combinations &

Linear Algebra Linear Algebra 3 in a Nutshell: ñ The set of all linear combinations of vectors in a Nutshell: ñ Example: linear subspace U ⊆ R spanned by vectors Vector Spaces Vector Spaces b~(1),..., b~(k) ∈ V is called the span b~(1) = (6, 0, 2), b~(2) = (0, 3, 3) and b~(3) = (3, 1, 2) S. Evert S. Evert ñ = (why?)   n o dim U 2 Introduction ~(1) ~(k) ~(1) ~(k) Introduction sp b ,..., b Í λ1b + · · · + λkb | λi ∈ R What is a vector? What is a vector? Examples   Examples Vector spaces ñ sp b~(1),..., b~(k) is a linear subspace of V Vector spaces Geometric Geometric x interpretation ñ a linear subspace is a subset of V that is closed under interpretation 3 Formal definition Formal definition Techniques vector addition and scalar multiplication Techniques 6   Linear algebra ~(1) ~(k) ~(1) ~(k) Linear algebra 5 Linear combination ñ b ,..., b form a basis of sp b ,..., b Linear combination Basis & coordinates Basis & coordinates 4 Linear subspaces iff they are linearly independent Linear subspaces x2 Matrix algebra Matrix algebra n 3 Practice with R Can you prove that every linear subspace of R has a basis? Practice with R Volume & determinant Volume & determinant 2 Linear maps ñ The of vectors b~(1),..., b~(k) is the dimension of Linear maps Equation systems Equation systems 1 Coordinate Coordinate transformations their span, corresponding to the largest number of transformations Hyperplanes Hyperplanes x linearly independent vectors among them 1 2 3 4 5 6 1 You did it! You did it! Matrices = arrays of numbers Matrices = arrays of numbers

Linear Algebra Linear Algebra in a Nutshell: ñ vector u~ ∈ Rn = list of real numbers (coordinates) in a Nutshell: ñ rank (A) = rank of the list of column vectors Vector Spaces Vector Spaces

S. Evert ñ list of k vectors = rectangular array of real numbers, S. Evert ñ Column matrices are a convention in linear algebra called a k × n (or n × k) matrix ñ Introduction Introduction Some applications build matrices from row vectors, or What is a vector? ñ ∈ 3 What is a vector? Examples e.g. vectors u,~ v~ R Examples interpret rows and columns as vectors (e.g. word space) Vector spaces     Vector spaces Geometric 3 2 Geometric interpretation interpretation ñ Row rank and column rank of a matrix A are always the Formal definition u~ ≡ 0 , v~ ≡ 2 Formal definition Techniques     Techniques same (this is not trivial!) Linear algebra 2 1 Linear algebra Linear combination Linear combination Basis & coordinates Basis & coordinates Linear subspaces form the columns of a matrix A: Linear subspaces Matrix algebra Matrix algebra Practice with R   Practice with R Volume & determinant . .     Volume & determinant Linear maps . . 3 2 a11 a12 Linear maps Equation systems   Equation systems       Coordinate A = u~ v~ = 0 2 = a21 a22 Coordinate transformations   transformations Hyperplanes . . 2 1 a a Hyperplanes . . 31 32 You did it! You did it!

Matrices = arrays of numbers Matrix algebra

Linear Algebra Linear Algebra in a Nutshell: ñ Matrices are a versatile instrument and a convenient in a Nutshell: ñ Concise notation of linear equation system by defining Vector Spaces Vector Spaces way to express linear operations on sets of numbers product of matrix and vector in a suitable way S. Evert S. Evert ñ Such rectangular arrays of numbers come up in many Introduction Introduction a11x1 + a12x2 + · · · + a1nxn = b1 What is a vector? other situations, e.g. the coefficients of a linear What is a vector? Examples Examples a x + a x + · · · + a x = b system of equations: 21 1 22 2 2n n 2 Vector spaces Vector spaces . Geometric Geometric . interpretation interpretation . Formal definition a11x1 + a12x2 + · · · + a1nxn = b1 Formal definition Techniques Techniques ak1x1 + ak2x2 + · · · + aknxn = bk Linear algebra a21x1 + a22x2 + · · · + a2nxn = b2 Linear algebra Linear combination Linear combination Basis & coordinates . Basis & coordinates .       Linear subspaces . Linear subspaces a11 ··· a1n x1 b1 Matrix algebra Matrix algebra Practice with R Practice with R  . .   .   .  a 1x1 + a 2x2 + · · · + a xn = b . . · . = . Volume & determinant k k kn k Volume & determinant ¯  . .   .   .  Linear maps Linear maps       Equation systems Equation systems ak1 ··· akn xn bk Coordinate   Coordinate transformations a ··· a transformations Hyperplanes 11 1n Hyperplanes  . .  You did it! ¯ A =  . .  You did it!  . .  ¯ A · x~ = b~ ak1 ··· akn Matrix algebra

Linear Algebra Linear Algebra in a Nutshell: ñ The set of all real-valued k × n matrices is a in a Nutshell: Vector Spaces Vector Spaces   (k · n)-dimensional vector space over R: c1j S. Evert S. Evert     ñ A + B is defined by element-wise addition  .   .  Introduction ñ Introduction   λA is defined by element-wise s-multiplication aij  = bi1 ··· bin ·   What is a vector? What is a vector?     . ñ  .  Examples these operations satisfy all vector space axioms Examples  .    Vector spaces Vector spaces cnj Geometric Geometric interpretation ñ Additional operation: matrix multiplication interpretation Formal definition Formal definition Techniques ñ two equation systems: z~ = B · y~ and y~ = C · x~ Techniques A = B · C Linear algebra ñ by inserting the expressions for y~ into the first Linear algebra × × × Linear combination Linear combination (k m) (k n) (n m) Basis & coordinates system, we can express z~ directly in terms of x~ Basis & coordinates Linear subspaces Linear subspaces Matrix algebra (and use this e.g. to solve the equations for x~) Matrix algebra Practice with R ñ the result is a linear equation system z~ = A · x~ Practice with R ñ B and C must be compatible Volume & determinant Volume & determinant Linear maps define matrix multiplication so that A = B · C Linear maps Equation systems Equation systems Coordinate Coordinate A · x~ corresponds to matrix multiplication of A with a transformations transformations Hyperplanes Hyperplanes single-column matrix (containing the vector x~) You did it! You did it! ñ convention: vector = column matrix

Matrix multiplication Transposition

Linear Algebra Linear Algebra in a Nutshell: ñ Properties of matrix multiplication (§ algebra): in a Nutshell: ñ The AT of a matrix A swaps the rows and Vector Spaces Vector Spaces ñ A(BC) = (AB)C Î ABC columns: S. Evert S. Evert T ñ + 0 = + 0   A(B B ) AB AB a1 b1 " # Introduction ñ (A + A0)B = AB + A0B Introduction   a1 a2 a3 What is a vector? What is a vector? a2 b2 = ñ   Examples (λA)B = A(λB) = λ(AB) Î λAB Examples b1 b2 b3 a3 b3 Vector spaces ñ A · 0 = 0, 0 · B = 0 Vector spaces Geometric ñ Geometric interpretation A · I = A, I · B = B interpretation Formal definition Formal definition ñ properties of the transpose: Techniques where A, B and C are compatible matrices Techniques ñ (A + B)T = AT + BT Linear algebra Linear algebra T T T Linear combination Linear combination ñ (λA) = λ(A ) Î λA ñ Basis & coordinates 0 is a of arbitrary dimensions Basis & coordinates ñ T T T Linear subspaces Linear subspaces (A · B) = B · A Matrix algebra ñ I is a square identity matrix of arbitrary dimensions: Matrix algebra (note that A and B are swapped!) Practice with R Practice with R T  Volume & determinant Volume & determinant ñ rank A = rank (A) Linear maps Linear maps   ñ T Equation systems 1 Equation systems I = I Coordinate Coordinate transformations  .  transformations T I Í  ..  ñ A is called symmetric iff A = A Hyperplanes   Hyperplanes ñ You did it! 1 You did it! symmetric matrices have many special properties that will become important later (e.g. eigenvalues) Vectors and matrices Matrix algebra playground

Linear Algebra Linear Algebra in a Nutshell: ñ A coordinate vector x~ ∈ Rn can be identified with a in a Nutshell: ñ Matrix algebra is a powerful and convenient tool in Vector Spaces Vector Spaces n × 1 matrix (i.e. a single-column matrix): numerical mathematics and computational science S. Evert S. Evert   ñ Specialised (and highly optimised) libraries are Introduction x1 Introduction What is a vector? h iT What is a vector? available for various programming languages Examples  .  Examples x~ =  .  = x1 ··· xn Vector spaces   Vector spaces ñ Some numerical programming environments are even Geometric xn Geometric interpretation interpretation based entirely on matrix algebra (Matlab) Formal definition Formal definition Techniques Techniques ñ Most statistical software packages support matrices Linear algebra ñ Multiplication of a matrix A containing the vectors Linear algebra Linear combination Linear combination (1) (k) Basis & coordinates a~ ,..., a~ with a vector of coefficients λ1, . . . , λk Basis & coordinates Linear subspaces Linear subspaces (1) (k) ñ Look at some examples using the Matrix algebra yields a linear combination of a~ ,..., a~ : Matrix algebra Practice with R Practice with R Volume & determinant Volume & determinant open-source statistical environment R Linear maps   Linear maps Equation systems λ1 Equation systems (http://www.r-project.org/) Coordinate  .  (1) (k) Coordinate transformations A ·  .  = λ1a~ + · · · + λka~ transformations ñ An excellent playground (and research Hyperplanes   Hyperplanes tool) for computational linguists, You did it! λk You did it! cognitive scientists, . . .

Matrix algebra in R Matrix algebra in R

Linear Algebra Linear Algebra in a Nutshell: Vectors in R: in a Nutshell: Matrix of column vectors: Vector Spaces Vector Spaces

S. Evert ñ u1 <- c(3, 0, 2) S. Evert ñ B <- cbind(u1, u2) ñ ñ print(B) Introduction u2 <- c(0, 2, 2) Introduction What is a vector? What is a vector? u1 u2 Examples ñ v <- 1:6 Examples [1,] 3 0 Vector spaces ñ print(v) Vector spaces Geometric Geometric [2,] 0 2 interpretation [1] 1 2 3 4 5 6 interpretation [3,] 2 2 Formal definition Formal definition Techniques Techniques Linear algebra Linear algebra Matrix of row vectors: Linear combination Defining matrices: Linear combination Basis & coordinates Basis & coordinates Linear subspaces ñ A <- matrix(v, nrow=3) Linear subspaces ñ C <- rbind(u1, u2) Matrix algebra Matrix algebra Practice with R ñ print(A) Practice with R ñ print(C) Volume & determinant Volume & determinant Linear maps [,1] [,2] Linear maps [,1] [,2] [,3] Equation systems Equation systems Coordinate [1,] 1 4 Coordinate u1 3 0 2 transformations [2,] 2 5 transformations u2 0 2 2 Hyperplanes Hyperplanes [3,] 3 6 You did it! You did it! Matrix algebra in R Matrix algebra in R

Linear Algebra Linear Algebra in a Nutshell: Matrix multiplication: in a Nutshell: Transpose of matrix: Vector Spaces Vector Spaces ñ A% %C ñ t(A) S. Evert * S. Evert [,1] [,2] [,3] [,1] [,2] [,3] Introduction [1,] 3 8 10 Introduction [1,] 1 2 3 What is a vector? What is a vector? Examples [2,] 6 10 14 Examples [2,] 4 5 6 Vector spaces [3,] 9 12 18 Vector spaces Geometric Geometric interpretation ñ NB: does not perform matrix multiplication interpretation Transposition of vectors: Formal definition * Formal definition Techniques Techniques ñ t(u1) (row vector) Linear algebra Also for multiplication of matrix with vector: Linear algebra Linear combination Linear combination [,1] [,2] [,3] Basis & coordinates ñ Basis & coordinates [1,] 3 0 2 Linear subspaces C%*% c(1,1,0) Linear subspaces Matrix algebra Matrix algebra Practice with R [,1] Practice with R Volume & determinant u1 3 Volume & determinant ñ t(t(u1)) (explicit column vector) Linear maps Linear maps Equation systems u2 2 Equation systems [,1] Coordinate Coordinate transformations result of multiplication is a column vector transformations [1,] 3 Hyperplanes Hyperplanes [2,] 0 You did it! ñ plain vectors understood as columns in matrix op’s You did it! [3,] 2

Matrix algebra in R Measuring & volume

Linear Algebra Linear Algebra in a Nutshell: Rank of a matrix: in a Nutshell: ñ A set of n linearly independent vectors Vector Spaces Vector Spaces ñ qr(A)$rank v~(1),..., v~(n) ∈ Rn spans a parallelotope S. Evert S. Evert 2 Introduction Introduction What is a vector? ñ la.rank <- function (A) qr(A)$rank What is a vector? Examples Examples

Vector spaces ñ la.rank(A) Vector spaces Geometric Geometric interpretation interpretation Formal definition Formal definition Techniques Column rank = row rank: Techniques parallelogram Linear algebra ñ la.rank(A) == la.rank(t(A)) Linear algebra Linear combination Linear combination Basis & coordinates [1] TRUE Basis & coordinates ñ Measure volume of such a parallelotope by the Linear subspaces Linear subspaces  (1) (n) Matrix algebra Matrix algebra determinant Det v~ ,..., v~ with properties: Practice with R T Practice with R Volume & determinant A · A is symmetric (can you prove this?): Volume & determinant  (k)   (k)  Linear maps Linear maps Det . . . , λv~ ,... = λ · Det ..., v~ ,... Equation systems ñ t(A) % %A Equation systems Coordinate * Coordinate transformations transformations Hyperplanes Hyperplanes   (j) (k) + (j) You did it! You did it! Det ..., v~ ,..., v~ λv~ ,...   = Det ..., v~(j),..., v~(k),... The determinant function The determinant of a matrix

Linear Algebra Linear Algebra in a Nutshell: ñ The function Det (·) is uniquely determined by these in a Nutshell: ñ The determinant of a n × n matrix A is defined as the Vector Spaces Vector Spaces two requirements (for a given vector space Rn), up to a determinant of its column vectors a~(1),..., a~(n) S. Evert S. Evert constant (which determines the unit of measurement)   Introduction Introduction det A Í Det a~(1),..., a~(n) What is a vector? ñ This constant is usually chosen in such a way that What is a vector? Examples   Examples Det e~(1),..., e~(n) = 1 for the standard basis of Rn Vector spaces Vector spaces ñ rank (A) = n ⇐⇒ det A ≠ 0 Geometric (this is the volume of an n-dimensional unit cube) Geometric interpretation interpretation T Formal definition Formal definition ñ further property: det A = det A Techniques Techniques ñ When v~(1),..., v~(n) are linearly dependent, the volume Linear algebra   Linear algebra ñ Calculating in R: Linear combination of the parallelotope is zero, i.e. Det v~(1),..., v~(n) = 0 Linear combination Basis & coordinates Basis & coordinates ñ D <- A % %C (D is a square matrix) Linear subspaces   Linear subspaces * Matrix algebra ñ v~(1),..., v~(n) is a basis of Rn iff Det v~(1),..., v~(n) ≠ 0 Matrix algebra ñ det(D) — almost zero! Practice with R Practice with R Volume & determinant Volume & determinant Linear maps Linear maps Equation systems Equation systems Coordinate Coordinate transformations transformations Hyperplanes Hyperplanes

You did it! You did it!

Linear maps Matrix representation of a

Linear Algebra Linear Algebra ñ (1) (n) n in a Nutshell: A linear map is a homomorphism between two vector in a Nutshell: ñ For a vector u~ = x1e~ + · · · + xne~ ∈ R , we have Vector Spaces spaces V and W , i.e. a function f : V → W that is Vector Spaces S. Evert S. Evert (1) (n) compatible with addition and s-multiplication: v~ = f (u)~ = f x1e~ + · · · + xne~ Introduction 1. f (u~ + v)~ = f (u)~ + f (v)~ Introduction (1) (n) What is a vector? What is a vector? = x1 · f e~ + · · · + xn · f e~ Examples 2. f (λu)~ = λ · f (u)~ Examples Vector spaces ñ Obviously, f is uniquely determined by the images Vector spaces and hence the coordinate vector y~ of v~ is given by Geometric Geometric interpretation ~(1) ~(n) ~(1) ~(n) interpretation Formal definition f b , . . . , f b of any basis b ,..., b of V Formal definition Techniques Techniques yj = x1 · aj1 + x2 · aj2 + · · · + xn · ajn ñ Using natural coordinates, a linear map f : Rn → Rk Linear algebra Linear algebra Linear combination can therefore be described by the vectors Linear combination Basis & coordinates Basis & coordinates Linear subspaces Linear subspaces Matrix algebra     Matrix algebra ñ This corresponds to matrix multiplication Practice with R a11 a1n Practice with R Volume & determinant     Volume & determinant a21 a2n       Linear maps (1)   (n)   Linear maps y1 a11 ··· a1n x1 Equation systems f e~ ≡   , . . . , f e~ ≡   Equation systems E  .  E  .  . . . . Coordinate  .   .  Coordinate  .  =  . .  ·  .  transformations     transformations  .   . .   .  Hyperplanes Hyperplanes       ak1 akn ··· You did it! You did it! yk ak1 akn xn

¯ v~ = f (u)~ ⇐⇒ y~ = Ax~ Image & Rank & composition

Linear Algebra Linear Algebra   in a Nutshell: ñ The image of a linear map f : Rn → Rk is the subspace in a Nutshell: ñ We have dim Im (f ) + dim Ker (f ) = n Vector Spaces Vector Spaces of all values v~ ∈ Rk that f can assume: S. Evert S. Evert ñ f is injective iff every v~ ∈ Im (f ) has a unique    preimage v~ = f (u)~ , i.e. iff Ker (f ) = 0~ or Introduction Im (f ) Í sp f e~(1), . . . , f e~(n) Introduction What is a vector? What is a vector? rank (f ) = n Examples Examples Vector spaces  Vector spaces Geometric ñ The rank of f is defined by rank (f ) Í dim Im (f ) Geometric ñ The composition of linear maps corresponds interpretation interpretation Formal definition Formal definition to matrix multiplication: Techniques ñ rank (f ) = rank (A) for the matrix representation A Techniques ñ n → k × Linear algebra k Linear algebra f : R R given by a k n matrix A ñ f is surjective (onto) iff Im (f ) = R , i.e. rank (f ) = k k m Linear combination Linear combination ñ g : R → R given by a m × k matrix B Basis & coordinates Basis & coordinates n m Linear subspaces Linear subspaces ¯ the composition g ◦ f : R → R is given n Matrix algebra ñ The kernel of f is the subspace of all x~ ∈ R that are Matrix algebra by the matrix product B · A Practice with R k Practice with R Volume & determinant mapped to 0~ ∈ R : Volume & determinant ñ recall that (g ◦ f )(u)~ Í g(f (u))~ Linear maps Linear maps Equation systems n o Equation systems Coordinate n ~ Coordinate transformations Ker (f ) Í x~ ∈ R f (x)~ = 0 transformations Hyperplanes Hyperplanes

You did it! You did it!

The inverse matrix Linear equation systems

Linear Algebra n n Linear Algebra in a Nutshell: ñ A linear map f : R → R is called an endomorphism in a Nutshell: ñ Recall that a of equations can be written Vector Spaces Vector Spaces ñ can be represented as a A in compact matrix notation: S. Evert S. Evert

Introduction Introduction ñ A · x~ = b~ What is a vector? f surjective ⇐⇒ rank (f ) = n ⇐⇒ f injective What is a vector? Examples    Examples ñ rank (f ) = rank f e~(1) , . . . , f e~(n) = n Vector spaces Vector spaces ñ Obviously, A describes a linear map f : Rn → Rk, and Geometric ⇐⇒ rank (A) = n ⇐⇒ det A ≠ 0 Geometric interpretation interpretation the linear system of equations can be written f (x)~ = b~ Formal definition Formal definition Techniques ¯ f bijective (one-to-one) ⇐⇒ det A ≠ 0 Techniques ñ This linear system can be solved iff b~ ∈ Im (f ), i.e. iff b~ Linear algebra Linear algebra Linear combination Linear combination is a linear combination of the column vectors of A Basis & coordinates ñ If f is bijective, there exists an inverse function Basis & coordinates Linear subspaces −1 n n Linear subspaces ñ The solution is given by the coefficients x1, . . . , xn Matrix algebra f : R → R , which is also a linear map and satisfies Matrix algebra Practice with R −1 −1 Practice with R of this linear combination Volume & determinant f (f (u))~ = u~ and f (f (v))~ = v~ Volume & determinant Linear maps Linear maps −1 −1 Equation systems ñ f is given by the inverse matrix A of A, Equation systems Coordinate Coordinate transformations −1 · = · −1 = transformations Hyperplanes which satisfies A A A A I Hyperplanes

You did it! You did it! Linear equation systems Linear equation systems

Linear Algebra Linear Algebra in a Nutshell: ñ The linear system has a solution for arbitrary b~ ∈ Rk in a Nutshell: Solving equation systems in R: Vector Spaces Vector Spaces iff f is surjective, i.e. iff rank (A) = k S. Evert S. Evert ñ A <- rbind(c(1,3), c(2,-1)) ñ Solutions of the linear system are unique iff f is ñ Introduction Introduction b <- c(5,3) What is a vector? injective, i.e. iff rank (A) = n (in this case, What is a vector? Examples Examples ñ la.rank(A) (test that A is invertible) the column vectors are linearly independent) Vector spaces Vector spaces Geometric Geometric −1 interpretation interpretation ñ A.inv <- solve(A) (inverse matrix A ) Formal definition ñ If k = n (i.e. A is a square matrix), the linear map f is Formal definition Techniques Techniques ñ print(round(A.inv, digits=3)) an endomorphism. Consequently, the linear system Linear algebra Linear algebra [,1] [,2] Linear combination has a unique solution for arbitrary b~ iff det A ≠ 0 Linear combination Basis & coordinates Basis & coordinates [1,] 0.143 0.429 Linear subspaces Linear subspaces Matrix algebra ñ In this case, the solution can be computed with the Matrix algebra [2,] 0.286 -0.143 Practice with R −1 −1 Practice with R Volume & determinant inverse function f or the inverse matrix A : Volume & determinant Linear maps Linear maps ñ Equation systems Equation systems A.inv %*% b Coordinate −1 ~ −1 ~ Coordinate transformations x~ = f (b) = A b transformations [,1] Hyperplanes Hyperplanes [1,] 2 You did it! practically, A−1 is often determined by solving the You did it! [2,] 1 corresponding linear system of equations ñ solve(A, b) (calculates A−1 · b~ directly)

Coordinate transformations Coordinate transformations

Linear Algebra Linear Algebra in a Nutshell: ñ We want to transform between coordinates wrt. in a Nutshell: ñ The basis can be represented by a matrix B whose Vector Spaces Vector Spaces a basis b~(1),..., b~(n) and standard coordinates in Rn columns are the standard coordinates of b~(1),..., b~(n) S. Evert S. Evert ñ Given a vector u~ ∈ Rn with standard coordinates Introduction Introduction x2 What is a vector? What is a vector? u~ ≡E x~ and B-coordinates u~ ≡B y~, we have Examples Examples u=(4,5) 6 Vector spaces Vector spaces ~(1) ~(n) Geometric Geometric u~ = y1b + · · · + ynb interpretation 5 interpretation Formal definition Formal definition Techniques Techniques

Linear algebra 4 Linear algebra Linear combination Linear combination ñ In standard coordinates, this equation corresponds to Basis & coordinates 3 Basis & coordinates Linear subspaces Linear subspaces matrix multiplication: Matrix algebra Matrix algebra Practice with R 2 Practice with R Volume & determinant e(2) Volume & determinant Linear maps Linear maps x~ = By~ Equation systems 1 Equation systems Coordinate Coordinate transformations transformations Hyperplanes Hyperplanes ¯ The matrix B transforms from B-coordinates into x b(2) 1 You did it! e(1)1 2 3 4 5 6 You did it! standard coordinates b(1) Coordinate transformations Coordinate transformations: an example

Linear Algebra Linear Algebra in a Nutshell: ñ In order to transform from standard coordinates into in a Nutshell: Vector Spaces Vector Spaces x B-coordinates, i.e. from x~ to y~, we have to solve the 2 S. Evert S. Evert linear system x~ = By~ u=(4,5) Introduction Introduction 6 (i) What is a vector? ñ Since the b~ are linearly independent, B is regular and What is a vector? Examples Examples the inverse B−1 exists, so that 5 Vector spaces Vector spaces Geometric Geometric interpretation −1 interpretation 4 Formal definition y~ = B x~ Formal definition Techniques Techniques 3 Linear algebra −1 Linear algebra Linear combination ¯ The inverse matrix B transforms from standard Linear combination Basis & coordinates Basis & coordinates Linear subspaces coordinates into B-coordinates Linear subspaces 2 Matrix algebra Matrix algebra (2) −1 −1 e Practice with R ñ Recall that BB = B B = I Practice with R Volume & determinant Volume & determinant 1 Linear maps ñ Linear maps Equation systems Transformation from B-coordinates (u~ ≡B y~) to Equation systems Coordinate Coordinate transformations C-coordinates (u~ ≡C z~): transformations (2) x Hyperplanes Hyperplanes b 1 e(1)1 2 3 4 5 6 You did it! You did it! (1) z~ = C−1By~ b

Coordinate transformations: an example Transforming linear maps

Linear Algebra Linear Algebra in a Nutshell: ñ basis b~(1) = (2, 1), b~(2) = (−1, 1) with matrix in a Nutshell: ñ Endomorphism f : Rn → Rn is given by matrix A Vector Spaces Vector Spaces with respect to the standard basis E S. Evert " #  1 1  S. Evert 2 −1 3 3 B = ,B−1 =   ñ v~ = f (u)~ for u~ ∈ Rn Introduction 1 1 − 1 2 Introduction What is a vector? 3 3 What is a vector? Examples Examples ñ Standard coordinates u~ ≡E x~ and v~ ≡E y~ Vector spaces Vector spaces Geometric ñ vector u~ = (4, 5) with standard and B-coordinates Geometric ñ 0 0 interpretation interpretation Coordinates wrt. some basis B: u~ ≡B x~ , v~ ≡E y~ Formal definition " # " # Formal definition Techniques 4 3 Techniques 0 0 −1 Linear algebra u~ ≡E , u~ ≡C Linear algebra ñ We have y~ = Ax~, as well as x~ = Bx~ and y~ = B y~ Linear combination 5 2 Linear combination Basis & coordinates Basis & coordinates 0 − 0 0 0 Linear subspaces Linear subspaces § y~ = B 1ABx~ Î A x~ Matrix algebra Matrix algebra Practice with R ñ Check that these equalities hold: Practice with R Volume & determinant Volume & determinant Linear maps   Linear maps Equation systems " # " #" # " # 1 1 " # Equation systems ¯ In B-coordinates, f is given by the matrix Coordinate 4 2 −1 3 3 3 3 4 Coordinate transformations = = transformations ,  1 2  Hyperplanes 5 1 1 2 2 − 5 Hyperplanes 0 −1 3 3 A = B AB You did it! You did it!

ñ Now repeat the calculations in R! Hyperplanes You did it!

Linear Algebra Linear Algebra in a Nutshell: ñ A one-dimensional linear subspace U of Rn in a Nutshell: Vector Spaces Vector Spaces is a through the origin S. Evert S. Evert ñ A two-dimensional linear subspace U of Rn Introduction Introduction What is a vector? is a plane through the origin What is a vector? Examples Examples n Vector spaces ñ A (n − 1)-dimensional linear subspace U of R Vector spaces Geometric Geometric interpretation is called a hyperplane through the origin interpretation Congratulations! Formal definition Formal definition Techniques Techniques Linear algebra ñ To represent arbitrary lines, planes, . . . Linear algebra Linear combination Linear combination Now you know the basic Basis & coordinates one can add a support vector v~ to U Basis & coordinates Linear subspaces Linear subspaces concepts and methods Matrix algebra ñ Γ Í v~ + U is a line, plane, . . . through the point v~ Matrix algebra Practice with R Practice with R of linear algebra. Volume & determinant Volume & determinant Linear maps Linear maps Equation systems ñ A hyperplane Γ is of particular importance because it Equation systems Coordinate Coordinate transformations divides Rn into two separate components transformations Hyperplanes Hyperplanes You did it! ¯ hyperplanes are used to implement linear classifiers You did it!