Recitation 4

Carles Domingo Fall 2020 Norms

Definition (Norm)

A norm k · k on V is a function k · k : V → R≥0 that verifies 1. Positive definiteness: if kvk = 0 =⇒ v = 0. 2. Homogeneity: kαvk = |α| × kvk 3. Triangle inequality: ku + vk ≤ kuk + kvk

A norm defines a distance d(x, y) = kx − yk on the vector space V . Definition (Distance) A distance d on a set S (not necessarily a vector space) is a function d : S × S → R. 1. Positive definiteness: if d(x, y) ≥ 0 for all x, y ∈ S and d(x, y) ≥ 0 i x = y. 2. Symmetry: d(x, y) = d(y, x) for all x, y ∈ S. 3. Triangle inequality: d(x, y) ≤ d(x, z) + d(z, y) 2/14 Inner Products

Definition (Inner product) Let V be a vector space. An inner product on V is a function h·, ·i from V × V to R that verifies the following points: 1. Symmetry: hu, vi = hv, ui for all u, v ∈ V . 2. Linearity: hu + v, wi = hu, wi + hv, wi and hαv, wi = αhv, wi for all u, v, w ∈ V and α ∈ R. 3. Positive definiteness: hv, vi ≥ 0 with equality if and only if v = 0.

Important: An inner product defines a norm kuk = phu, ui. Most used inner product: the Euclidean inner product: > hu, vi =√u v. The corresponding norm is > √ Pn 2 kuk = u u = i=1 ui . Given an inner product, we can define the angle between two vectors: cos(θ) = hu,vi . kukkvk 3/14 Questions: Norms & Inner Products

3 1. Which of the following functions are inner products for x, y ∈ R ? i. f(x, y) = x1y2 + x2y3 + x3y1 2 2 2 2 2 2 ii. f(x, y) = x1y1 + x2y2 + x1y1 iii. f(x, y) = x1y1 + x3y3

4/14 Questions: Norms & Inner Products

3 1. Which of the following functions are inner products for x, y ∈ R ? i. f(x, y) = x1y2 + x2y3 + x3y1 2 2 2 2 2 2 ii. f(x, y) = x1y1 + x2y2 + x1y1 iii. f(x, y) = x1y1 + x3y3

4/14 Questions: Norms & Inner Products

3 1. Which of the following functions are inner products for x, y ∈ R ? i. f(x, y) = x1y2 + x2y3 + x3y1 2 2 2 2 2 2 ii. f(x, y) = x1y1 + x2y2 + x1y1 iii. f(x, y) = x1y1 + x3y3

4/14 Questions: Norms & Inner Products

m×n n 2. For A ∈ R and x ∈ R , prove that v u m n uX X 2 kAxk ≤ kxkt Ai,j i=1 j=1

5/14 Questions: Norms & Inner Products

m×n n 2. For A ∈ R and x ∈ R , prove that v u m n uX X 2 kAxk ≤ kxkt Ai,j i=1 j=1

5/14 Questions: Orthogonality

Recall from the lecture: 1. Two vectors u, v are orthogonal if hu, vi = 0. 2. If U is a subspace of of a vector space V with inner product h, i, the orthogonal projection PU : V → U is defined as x 7→ arg minu∈U kx − uk. Exercises:

1. Let v1, ..., vk be a list of orthogonal vectors. Show that v1, ..., vk are linearly independent. n 2. Let U be the subspace of R with orthonormal basis u1, ..., uk. i. Prove that the orthogonal projection of v ∈ Rn onto U can be Pk expressed as PU (v) = i=0hv, uiiui. (Use the fact that the orthonormal basis for a subspace of R can be extended to obtain an orthonormal basis for R). ii. Prove that kPU (v)k ≤ kvk. iii. Prove that v − PU (v) is orthogonal to PU (v).

6/14 Questions: Orthogonality

1. Let v1, ..., vk be a list of orthogonal vectors. Show that v1, ..., vk are linearly independent.

7/14 Questions: Orthogonality

1. Let v1, ..., vk be a list of orthogonal vectors. Show that v1, ..., vk are linearly independent.

7/14 Questions: Orthogonality

n 2. Let U be the subspace of R with orthonormal basis u1, ..., uk. n i. Prove that the orthogonal projection of v ∈ R onto U can be Pk expressed as PU (v) = i=0hv, uiiui. (Use the fact that the orthonormal basis for a subspace of R can be extended to obtain an orthonormal basis for R). ii. Prove that kPU (v)k ≤ kvk.