Math 22 – Linear Algebra and its applications

- Lecture 20 -

Instructor: Bjoern Muetzel GENERAL INFORMATION ▪ Office hours: Tu 1-3 pm, Th, Sun 2-4 pm in KH 229 Tutorial: Tu, Th, Sun 7-9 pm in KH 105 Come tomorrow to practice for the exam.

▪ Homework 6: due today at 4 pm outside KH 008. Please divide into the parts A, C and D. Exercise 1 b) is optional.

▪ Midterm 2: Friday Nov 1 at 4 pm in Carpenter 013 - Topics: Chapter 2.1 – 4.7 (included) - about 8-9 questions - Practice exam 2 solutions available 6 Orthogonality and Least Squares

6.1 INNER PRODUCT, LENGTH, AND ORTHOGONALITY Summary: If u and v are vectors in ℝ푛 then the inner product uTv = u∙v allows us to measure angles, lengths and distances.

u v 휃 GEOMETRIC INTERPRETATION

Inner product in ℝퟐ 2 1 Given two vectors u = and 푣 = how can we measure the length 1 −1 of these vectors and the angle between them?

INNER PRODUCT ▪ If u and v are vectors in ℝ푛, then we can regard u and v as 푛 × 1 matrices. The transpose uT is a 1 × 푛 matrix, and the matrix product uTv is a 1 × 1 matrix, which we write as a real number without brackets.

푢1 푣1 푢2 푣2 푛 ▪ Definition: If 푢 = ⋮ and 푣 = ⋮ are vectors in ℝ , then 푢푛 푣푛

the inner product or dot product of u and v is 푣1 푇 푣2 푢 푣 = 푢 ∙ 푣 = [푢1, 푢2, … , 푢푛] ⋮ = 푢1푣1 + 푢2푣2 + ⋯ + 푢푛푣푛. 푣푛 ▪ Theorem 1: Let u, v, and w be vectors in ℝ푛, and let c in ℝ be a scalar. Then a. 푢 ∙ 푣 = 푣 ∙ 푢 b. 푢 + 푣 ∙ 푤 = 푢 ∙ 푤 + 푣 ∙ 푤 c. 푐푢 ∙ 푣 = 푐 푢 ∙ 푣 = 푢 ∙ (푐푣) d. 푢 ∙ 푢 ≥ 0, and 푢 ∙ 푢 = 0 if and only if 푢 = 0.

▪ Consequence: (Linearity)

(푐1푢1+…+푐푝푢푝) ∙ 푤 = 푐1 푢1 ∙ 푤 + ⋯ + 푐푝 푢푝 ∙ 푤 .

Proof: 1.) a. and d. can be easily checked. 2. ) b. and c. are true as the dot product is a matrix multiplication which is linear. By a. it is linear from “both sides”. 3.) The consequence follows from b. and c. LENGTH 푛 ▪ If v is in ℝ , with entries v1, …, vn, then the square root of 푣 ∙ 푣 is well-defined as 푣 ∙ 푣 ≥ 0. We define:

▪ Definition: The length (or norm) 푣 of v is the nonnegative real number defined by 2 2 2 2 푣 = 푣 ∙ 푣 = 푣1 + 푣2 + ⋯ + 푣푛 , hence 푣 = 푣 ∙ 푣.

Example: Length in ℝ2 푎 푣 = 푏 Definition: A vector 푣 in ℝ푛, that has length 푣 = 푣 ∙ 푣 = 1 is called a unit vector.

Note: If a vector 푣 in ℝ푛 is multiplied by a scalar c in ℝ, then 푐푣 = 푐푣 ∙ 푐푣 = 푐2(푣 ∙ 푣) = 푐 ∙ 푣 . 푣 푣 푢 = 푣

Definition: If we divide a vector 푣 ≠ 0 by its length 푣 then we ퟏ obtain a unit vector 풖 = ∙ 풗 . The process of creating 푢 from 푣 is 풗 called normalizing 푣, and we say that 풖 is in the same direction as 풗. LENGTH 1 ▪ Example: Let 푣 = 2 be a vector in ℝ3. 2 1.) Calculate 푣 . 2.) Find a unit vector 푢 in the same direction as 푣. What is 푢 2 ? 3.) Draw a coordinate system with 푣, 푢 and the unit sphere 푆 = {푥 in ℝ3, 푥 = 1} in ℝ3. ANGLE Definition: Let 푢 and 푣 be vectors in ℝ푛. Then the angle 휽 = ∡ 풖, 풗 between 푢 and 푣 is given by 푢∙푣 푢∙푣 cos θ = or θ = 푎푟푐푐표푠 . 푢 푣 푢 푣 3 −2 Example: For 푢 = and 푣 = , compute cos(∡ 푢, 푣 ). Estimate 4 2 the angle ∡ 푢, 푣 with a calculator and with a sketch of 푢 and 푣 in ℝ2. Does the angle change if we normalize 푢 and 푣 ? DISTANCE

Definition: For 푢 and 푣 in ℝ푛, the distance between 풖 and 풗, written as dist(푢,푣), is the length of the vector 푢-v. That is,

dist 푢, 푣 = 푢 − 푣 Example: Distance in ℝ2 5 3 Example: Compute the distance between 푢 = and 푣 = . 1 2 Check your calculation with a sketch of 푢 and 푣 in ℝ2. ORTHOGONALITY

▪ Note: Two nonzero vectors 푢 and 푣 in ℝ푛 are perpendicular, i.e. 휋 ∡ 푢, 푣 = or cos ∡ 푢, 푣 = 0 if and only if 푢 ∙ 푣 = 0. 2

▪ Definition: Two vectors 푢 and 푣 in ℝ푛 are orthogonal to each other if 풖 ∙ 풗 = ퟎ .

Example: The zero vector 0 is orthogonal to every vector in ℝ푛.

▪ Theorem 2: Two vectors 푢 and 푣 in ℝ푛 are orthogonal if and only if 푢 + 푣 2 = 푢 2 + 푣 2. Proof: Definition: (Orthogonal Complements) 1.) If a vector z is orthogonal to every vector in a subspace W of ℝ푛, then z is said to be orthogonal to W. 2.) The set of all vectors z that are orthogonal to W is called the orthogonal complement of W and is denoted by 푊⊥. 푊⊥ is read as “W perpendicular” or simply “W perp”.

Note: 1.) A vector x is in 푊⊥ if and only if x is orthogonal to every vector in a set that spans W. 2.) 푊⊥ is a subspace of ℝ푛.

Proof: Idea: Use linearity of the inner product for 1.). Then check the subspace criteria for 2.) See HW 7. ORTHOGONALITY Theorem 3: Let A be an 푚 × 푛 matrix. The orthogonal complement of the row space of A is the null space of A, and the orthogonal complement of the column space of A is the null space of AT:

푅표푤 퐴 ⊥ = 푁푢푙 퐴 and 퐶표푙 퐴 ⊥ = 푁푢푙 퐴푇 .

Proof: See HW 7

How can we find the orthogonal complement 푾⊥ of a subspace W?

푛 Note: Given a subspace W= Span{푎1, 푎2,…, 푎푚} in ℝ . Let A be the 푛 × 푚 matrix A = [푎1,푎2,…, 푎푚] . Theorem 3 states that 푊⊥ = 퐶표푙 퐴 ⊥ = 푁푢푙 퐴푇. 3 3 Example: Let W = Span{ 4 , −1 }. 1 0 Find 푊⊥, then sketch W and 푊⊥ in ℝ3. Hint: Use Theorem 3.

Slide 6.1- 18