Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
Multivariable Calculus Review
Multivariable Calculus Review Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
Multi-Variable Calculus
Point-Set Topology
Compactness
The Weierstrass Extreme Value Theorem
Operator and Matrix Norms
Mean Value Theorem
Multivariable Calculus Review n I ν(x) ≥ 0 ∀ x ∈ R with equality iff x = 0. n I ν(αx) = |α|ν(x) ∀ x ∈ R α ∈ R n I ν(x + y) ≤ ν(x) + ν(y) ∀ x, y ∈ R We usually denote ν(x) by kxk. Norms are convex functions.
lp norms
1 Pn p p kxkp := ( i=1 |xi | ) , 1 ≤ p < ∞ kxk∞ = maxi=1,...,n |xi |
Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
Multi-Variable Calculus
Norms: n n A function ν : R → R is a vector norm on R if
Multivariable Calculus Review n I ν(αx) = |α|ν(x) ∀ x ∈ R α ∈ R n I ν(x + y) ≤ ν(x) + ν(y) ∀ x, y ∈ R We usually denote ν(x) by kxk. Norms are convex functions.
lp norms
1 Pn p p kxkp := ( i=1 |xi | ) , 1 ≤ p < ∞ kxk∞ = maxi=1,...,n |xi |
Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
Multi-Variable Calculus
Norms: n n A function ν : R → R is a vector norm on R if n I ν(x) ≥ 0 ∀ x ∈ R with equality iff x = 0.
Multivariable Calculus Review We usually denote ν(x) by kxk. Norms are convex functions.
lp norms
1 Pn p p kxkp := ( i=1 |xi | ) , 1 ≤ p < ∞ kxk∞ = maxi=1,...,n |xi |
Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
Multi-Variable Calculus
Norms: n n A function ν : R → R is a vector norm on R if n I ν(x) ≥ 0 ∀ x ∈ R with equality iff x = 0. n I ν(αx) = |α|ν(x) ∀ x ∈ R α ∈ R n I ν(x + y) ≤ ν(x) + ν(y) ∀ x, y ∈ R
Multivariable Calculus Review lp norms
1 Pn p p kxkp := ( i=1 |xi | ) , 1 ≤ p < ∞ kxk∞ = maxi=1,...,n |xi |
Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
Multi-Variable Calculus
Norms: n n A function ν : R → R is a vector norm on R if n I ν(x) ≥ 0 ∀ x ∈ R with equality iff x = 0. n I ν(αx) = |α|ν(x) ∀ x ∈ R α ∈ R n I ν(x + y) ≤ ν(x) + ν(y) ∀ x, y ∈ R We usually denote ν(x) by kxk. Norms are convex functions.
Multivariable Calculus Review Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
Multi-Variable Calculus
Norms: n n A function ν : R → R is a vector norm on R if n I ν(x) ≥ 0 ∀ x ∈ R with equality iff x = 0. n I ν(αx) = |α|ν(x) ∀ x ∈ R α ∈ R n I ν(x + y) ≤ ν(x) + ν(y) ∀ x, y ∈ R We usually denote ν(x) by kxk. Norms are convex functions.
lp norms
1 Pn p p kxkp := ( i=1 |xi | ) , 1 ≤ p < ∞ kxk∞ = maxi=1,...,n |xi |
Multivariable Calculus Review I D is open if for every x ∈ D there exists > 0 such that x + B ⊂ D where x + B = {x + u : u ∈ B} n and B is the unit ball of some given norm on R .
I D is closed if every point x satisfying (x + B) ∩ D 6= ∅ for all > 0, must be a point in D.
I D is bounded if there exists β > 0 such that kxk ≤ β for all x ∈ D.
Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
Elementary Topological Properties of Sets
n Let D ⊂ R .
Multivariable Calculus Review I D is closed if every point x satisfying (x + B) ∩ D 6= ∅ for all > 0, must be a point in D.
I D is bounded if there exists β > 0 such that kxk ≤ β for all x ∈ D.
Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
Elementary Topological Properties of Sets
n Let D ⊂ R . I D is open if for every x ∈ D there exists > 0 such that x + B ⊂ D where x + B = {x + u : u ∈ B} n and B is the unit ball of some given norm on R .
Multivariable Calculus Review I D is bounded if there exists β > 0 such that kxk ≤ β for all x ∈ D.
Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
Elementary Topological Properties of Sets
n Let D ⊂ R . I D is open if for every x ∈ D there exists > 0 such that x + B ⊂ D where x + B = {x + u : u ∈ B} n and B is the unit ball of some given norm on R .
I D is closed if every point x satisfying (x + B) ∩ D 6= ∅ for all > 0, must be a point in D.
Multivariable Calculus Review Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
Elementary Topological Properties of Sets
n Let D ⊂ R . I D is open if for every x ∈ D there exists > 0 such that x + B ⊂ D where x + B = {x + u : u ∈ B} n and B is the unit ball of some given norm on R .
I D is closed if every point x satisfying (x + B) ∩ D 6= ∅ for all > 0, must be a point in D.
I D is bounded if there exists β > 0 such that kxk ≤ β for all x ∈ D.
Multivariable Calculus Review I x ∈ D is said to be a cluster point of D if
(x + B) ∩ D 6= ∅
for every > 0.
I D is compact if it is closed and bounded.
Theorem: [Weierstrass Compactness Theorem] n A set D ⊂ R is compact if and only if every infinite subset of D has a cluster point and all such cluster points lie in D.
Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
Elementary Topological Properties of Sets: Compactness
n Let D ⊂ R .
Multivariable Calculus Review I D is compact if it is closed and bounded.
Theorem: [Weierstrass Compactness Theorem] n A set D ⊂ R is compact if and only if every infinite subset of D has a cluster point and all such cluster points lie in D.
Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
Elementary Topological Properties of Sets: Compactness
n Let D ⊂ R .
I x ∈ D is said to be a cluster point of D if
(x + B) ∩ D 6= ∅
for every > 0.
Multivariable Calculus Review Theorem: [Weierstrass Compactness Theorem] n A set D ⊂ R is compact if and only if every infinite subset of D has a cluster point and all such cluster points lie in D.
Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
Elementary Topological Properties of Sets: Compactness
n Let D ⊂ R .
I x ∈ D is said to be a cluster point of D if
(x + B) ∩ D 6= ∅
for every > 0.
I D is compact if it is closed and bounded.
Multivariable Calculus Review Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
Elementary Topological Properties of Sets: Compactness
n Let D ⊂ R .
I x ∈ D is said to be a cluster point of D if
(x + B) ∩ D 6= ∅
for every > 0.
I D is compact if it is closed and bounded.
Theorem: [Weierstrass Compactness Theorem] n A set D ⊂ R is compact if and only if every infinite subset of D has a cluster point and all such cluster points lie in D.
Multivariable Calculus Review n F is continuous on a set D ⊂ R if F is continuous at every point of D.
Theorem: [Weierstrass Extreme Value Theorem] Every continuous function on a compact set attains its extreme values on that set.
Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
Continuity and The Weierstrass Extreme Value Theorem
n m The mapping F : R → R is continuous at the point x if
lim kF (x) − F (x)k = 0. kx−xk→0
Multivariable Calculus Review Theorem: [Weierstrass Extreme Value Theorem] Every continuous function on a compact set attains its extreme values on that set.
Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
Continuity and The Weierstrass Extreme Value Theorem
n m The mapping F : R → R is continuous at the point x if
lim kF (x) − F (x)k = 0. kx−xk→0
n F is continuous on a set D ⊂ R if F is continuous at every point of D.
Multivariable Calculus Review Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
Continuity and The Weierstrass Extreme Value Theorem
n m The mapping F : R → R is continuous at the point x if
lim kF (x) − F (x)k = 0. kx−xk→0
n F is continuous on a set D ⊂ R if F is continuous at every point of D.
Theorem: [Weierstrass Extreme Value Theorem] Every continuous function on a compact set attains its extreme values on that set.
Multivariable Calculus Review Examples: kAk2 = kAk(2,2) = max{kAxk2 : kxk2 ≤ 1}
kAk∞ = kAk(∞,∞) = max{kAxk∞ : kxk∞ ≤ 1} Pn = max j=1 |aij | (max row sum) 1≤i≤m
kAk1 = kAk(1,1) = max{kAxk1 : kxk1 ≤ 1} Pm = max i=1 |aij | (max column sum) 1≤j≤n
Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
Operator Norms
m×n A ∈ R kAk(p,q) = max{kAxkp : kxkq ≤ 1}
Multivariable Calculus Review kAk2 = kAk(2,2) = max{kAxk2 : kxk2 ≤ 1}
kAk∞ = kAk(∞,∞) = max{kAxk∞ : kxk∞ ≤ 1} Pn = max j=1 |aij | (max row sum) 1≤i≤m
kAk1 = kAk(1,1) = max{kAxk1 : kxk1 ≤ 1} Pm = max i=1 |aij | (max column sum) 1≤j≤n
Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
Operator Norms
m×n A ∈ R kAk(p,q) = max{kAxkp : kxkq ≤ 1}
Examples:
Multivariable Calculus Review kAk∞ = kAk(∞,∞) = max{kAxk∞ : kxk∞ ≤ 1} Pn = max j=1 |aij | (max row sum) 1≤i≤m
kAk1 = kAk(1,1) = max{kAxk1 : kxk1 ≤ 1} Pm = max i=1 |aij | (max column sum) 1≤j≤n
Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
Operator Norms
m×n A ∈ R kAk(p,q) = max{kAxkp : kxkq ≤ 1}
Examples: kAk2 = kAk(2,2) = max{kAxk2 : kxk2 ≤ 1}
Multivariable Calculus Review kAk1 = kAk(1,1) = max{kAxk1 : kxk1 ≤ 1} Pm = max i=1 |aij | (max column sum) 1≤j≤n
Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
Operator Norms
m×n A ∈ R kAk(p,q) = max{kAxkp : kxkq ≤ 1}
Examples: kAk2 = kAk(2,2) = max{kAxk2 : kxk2 ≤ 1}
kAk∞ = kAk(∞,∞) = max{kAxk∞ : kxk∞ ≤ 1} Pn = max j=1 |aij | (max row sum) 1≤i≤m
Multivariable Calculus Review Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
Operator Norms
m×n A ∈ R kAk(p,q) = max{kAxkp : kxkq ≤ 1}
Examples: kAk2 = kAk(2,2) = max{kAxk2 : kxk2 ≤ 1}
kAk∞ = kAk(∞,∞) = max{kAxk∞ : kxk∞ ≤ 1} Pn = max j=1 |aij | (max row sum) 1≤i≤m
kAk1 = kAk(1,1) = max{kAxk1 : kxk1 ≤ 1} Pm = max i=1 |aij | (max column sum) 1≤j≤n
Multivariable Calculus Review I kAxkp ≤ kAk(p,q)kxkq.
I kAk ≥ 0 with equality ⇔ kAxk = 0 ∀ x or A ≡ 0.
I kαAk = max{kαAxk : kxk ≤ 1} = max{|α| kAxk : kαk ≤ 1} = |α| kAk
I kA + Bk = max{kAx + Bxk : kxk ≤ 1} ≤ max{kAxk + kBxk A ≤ 1} = max{kAx1k + kBx2k : x1 = x2, kx1k ≤ 1, kx2k ≤ 1} ≤ max{kAx1k + kBx2k : kx1k ≤ 1, kx2k ≤ 1} = kAk + kBk
Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
Operator Norms
Properties:
Multivariable Calculus Review I kAk ≥ 0 with equality ⇔ kAxk = 0 ∀ x or A ≡ 0.
I kαAk = max{kαAxk : kxk ≤ 1} = max{|α| kAxk : kαk ≤ 1} = |α| kAk
I kA + Bk = max{kAx + Bxk : kxk ≤ 1} ≤ max{kAxk + kBxk A ≤ 1} = max{kAx1k + kBx2k : x1 = x2, kx1k ≤ 1, kx2k ≤ 1} ≤ max{kAx1k + kBx2k : kx1k ≤ 1, kx2k ≤ 1} = kAk + kBk
Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
Operator Norms
Properties:
I kAxkp ≤ kAk(p,q)kxkq.
Multivariable Calculus Review I kαAk = max{kαAxk : kxk ≤ 1} = max{|α| kAxk : kαk ≤ 1} = |α| kAk
I kA + Bk = max{kAx + Bxk : kxk ≤ 1} ≤ max{kAxk + kBxk A ≤ 1} = max{kAx1k + kBx2k : x1 = x2, kx1k ≤ 1, kx2k ≤ 1} ≤ max{kAx1k + kBx2k : kx1k ≤ 1, kx2k ≤ 1} = kAk + kBk
Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
Operator Norms
Properties:
I kAxkp ≤ kAk(p,q)kxkq.
I kAk ≥ 0 with equality ⇔ kAxk = 0 ∀ x or A ≡ 0.
Multivariable Calculus Review I kA + Bk = max{kAx + Bxk : kxk ≤ 1} ≤ max{kAxk + kBxk A ≤ 1} = max{kAx1k + kBx2k : x1 = x2, kx1k ≤ 1, kx2k ≤ 1} ≤ max{kAx1k + kBx2k : kx1k ≤ 1, kx2k ≤ 1} = kAk + kBk
Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
Operator Norms
Properties:
I kAxkp ≤ kAk(p,q)kxkq.
I kAk ≥ 0 with equality ⇔ kAxk = 0 ∀ x or A ≡ 0.
I kαAk = max{kαAxk : kxk ≤ 1} = max{|α| kAxk : kαk ≤ 1} = |α| kAk
Multivariable Calculus Review Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
Operator Norms
Properties:
I kAxkp ≤ kAk(p,q)kxkq.
I kAk ≥ 0 with equality ⇔ kAxk = 0 ∀ x or A ≡ 0.
I kαAk = max{kαAxk : kxk ≤ 1} = max{|α| kAxk : kαk ≤ 1} = |α| kAk
I kA + Bk = max{kAx + Bxk : kxk ≤ 1} ≤ max{kAxk + kBxk A ≤ 1} = max{kAx1k + kBx2k : x1 = x2, kx1k ≤ 1, kx2k ≤ 1} ≤ max{kAx1k + kBx2k : kx1k ≤ 1, kx2k ≤ 1} = kAk + kBk
Multivariable Calculus Review m×n (mn) Identify R with R by simply stacking the columns of a matrix one on top of the other to create a very long vector in (mn) R . m×n (mn) This mapping takes a matrix in R to a vector in R by stacking columns. It is called vec (or sometimes cvec). m×n Using vec we can define an inner product on R (called the Frobenius inner prodiuct) by writting
T T hA, BiF = vec (A) vec (B) = trace(A B) .
2 Note kAkF = hA, AiF .
Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
Frobenius Norm and Inner Product
m×n T 1/2 A ∈ R kAkF = trace(A A)
Multivariable Calculus Review m×n (mn) This mapping takes a matrix in R to a vector in R by stacking columns. It is called vec (or sometimes cvec). m×n Using vec we can define an inner product on R (called the Frobenius inner prodiuct) by writting
T T hA, BiF = vec (A) vec (B) = trace(A B) .
2 Note kAkF = hA, AiF .
Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
Frobenius Norm and Inner Product
m×n T 1/2 A ∈ R kAkF = trace(A A) m×n (mn) Identify R with R by simply stacking the columns of a matrix one on top of the other to create a very long vector in (mn) R .
Multivariable Calculus Review m×n Using vec we can define an inner product on R (called the Frobenius inner prodiuct) by writting
T T hA, BiF = vec (A) vec (B) = trace(A B) .
2 Note kAkF = hA, AiF .
Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
Frobenius Norm and Inner Product
m×n T 1/2 A ∈ R kAkF = trace(A A) m×n (mn) Identify R with R by simply stacking the columns of a matrix one on top of the other to create a very long vector in (mn) R . m×n (mn) This mapping takes a matrix in R to a vector in R by stacking columns. It is called vec (or sometimes cvec).
Multivariable Calculus Review 2 Note kAkF = hA, AiF .
Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
Frobenius Norm and Inner Product
m×n T 1/2 A ∈ R kAkF = trace(A A) m×n (mn) Identify R with R by simply stacking the columns of a matrix one on top of the other to create a very long vector in (mn) R . m×n (mn) This mapping takes a matrix in R to a vector in R by stacking columns. It is called vec (or sometimes cvec). m×n Using vec we can define an inner product on R (called the Frobenius inner prodiuct) by writting
T T hA, BiF = vec (A) vec (B) = trace(A B) .
Multivariable Calculus Review Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
Frobenius Norm and Inner Product
m×n T 1/2 A ∈ R kAkF = trace(A A) m×n (mn) Identify R with R by simply stacking the columns of a matrix one on top of the other to create a very long vector in (mn) R . m×n (mn) This mapping takes a matrix in R to a vector in R by stacking columns. It is called vec (or sometimes cvec). m×n Using vec we can define an inner product on R (called the Frobenius inner prodiuct) by writting
T T hA, BiF = vec (A) vec (B) = trace(A B) .
2 Note kAkF = hA, AiF .
Multivariable Calculus Review If the limit F (x + td) − F (x) lim =: F 0(x; d) t↓0 t exists, it is called the directional derivative of F at x in the direction h. If this limit exists for all d ∈ Rn and is linear in the d argument, 0 0 0 F (x; αd1 + βd2) = αF (x; d1) + βF (x; d2), then F is said to be differentiable at x, and denote the associated linear operator by F 0(x).
Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
Differentiation
n m Let F : R → R , and let Fi denote the ith component functioon of F : F1(x) F2(x) F (x) = , . . Fm(x) n m where each Fi is a mapping from R to R .
Multivariable Calculus Review If this limit exists for all d ∈ Rn and is linear in the d argument, 0 0 0 F (x; αd1 + βd2) = αF (x; d1) + βF (x; d2), then F is said to be differentiable at x, and denote the associated linear operator by F 0(x).
Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
Differentiation
n m Let F : R → R , and let Fi denote the ith component functioon of F : F1(x) F2(x) F (x) = , . . Fm(x) n m where each Fi is a mapping from R to R . If the limit F (x + td) − F (x) lim =: F 0(x; d) t↓0 t exists, it is called the directional derivative of F at x in the direction h.
Multivariable Calculus Review Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
Differentiation
n m Let F : R → R , and let Fi denote the ith component functioon of F : F1(x) F2(x) F (x) = , . . Fm(x) n m where each Fi is a mapping from R to R . If the limit F (x + td) − F (x) lim =: F 0(x; d) t↓0 t exists, it is called the directional derivative of F at x in the direction h. If this limit exists for all d ∈ Rn and is linear in the d argument, 0 0 0 F (x; αd1 + βd2) = αF (x; d1) + βF (x; d2), then F is said to be differentiable at x, and denote the associated linear operator by F 0(x).
Multivariable Calculus Review or equivalently,
F (y) = F (x) + F 0(x)(y − x) + o(ky − xk),
where o(t) is a function satisfying
o(t) lim = 0 . t→0 t
Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
Differentiation
One can show that if F 0(x) exists, then
kF (y) − (F (x) + F 0(x)(y − x))k lim = 0, ky−xk→0 ky − xk
Multivariable Calculus Review where o(t) is a function satisfying
o(t) lim = 0 . t→0 t
Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
Differentiation
One can show that if F 0(x) exists, then
kF (y) − (F (x) + F 0(x)(y − x))k lim = 0, ky−xk→0 ky − xk
or equivalently,
F (y) = F (x) + F 0(x)(y − x) + o(ky − xk),
Multivariable Calculus Review Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
Differentiation
One can show that if F 0(x) exists, then
kF (y) − (F (x) + F 0(x)(y − x))k lim = 0, ky−xk→0 ky − xk
or equivalently,
F (y) = F (x) + F 0(x)(y − x) + o(ky − xk),
where o(t) is a function satisfying
o(t) lim = 0 . t→0 t
Multivariable Calculus Review (i) If F 0(x) exists, it is unique. (ii) If F 0(x) exists, then F 0(x; d) exists for all d and F 0(x; d) = F 0(x)d. (iii) If F 0(x) exists, then F is continuous at x. (iv) (Matrix Representation) Suppose F 0(x) is continuous at x, Then
∂F1 ∂F1 ∂F1 ∂x ∂x ··· ∂x T 1 2 n ∇F1(¯x) ∂F2 ∂F2 ··· ∂F2 T ∂x1 ∂x2 ∂xn ∇F2(¯x) F 0(x) = ∇F (x) = = , . . . . ∇F (¯x)T ∂Fn ······ ∂Fm m ∂x1 ∂xn
T where each partial derivative is evaluated at x = (x 1,..., x n) .
Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
Differentiation Facts
Multivariable Calculus Review (ii) If F 0(x) exists, then F 0(x; d) exists for all d and F 0(x; d) = F 0(x)d. (iii) If F 0(x) exists, then F is continuous at x. (iv) (Matrix Representation) Suppose F 0(x) is continuous at x, Then
∂F1 ∂F1 ∂F1 ∂x ∂x ··· ∂x T 1 2 n ∇F1(¯x) ∂F2 ∂F2 ··· ∂F2 T ∂x1 ∂x2 ∂xn ∇F2(¯x) F 0(x) = ∇F (x) = = , . . . . ∇F (¯x)T ∂Fn ······ ∂Fm m ∂x1 ∂xn
T where each partial derivative is evaluated at x = (x 1,..., x n) .
Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
Differentiation Facts
(i) If F 0(x) exists, it is unique.
Multivariable Calculus Review (iii) If F 0(x) exists, then F is continuous at x. (iv) (Matrix Representation) Suppose F 0(x) is continuous at x, Then
∂F1 ∂F1 ∂F1 ∂x ∂x ··· ∂x T 1 2 n ∇F1(¯x) ∂F2 ∂F2 ··· ∂F2 T ∂x1 ∂x2 ∂xn ∇F2(¯x) F 0(x) = ∇F (x) = = , . . . . ∇F (¯x)T ∂Fn ······ ∂Fm m ∂x1 ∂xn
T where each partial derivative is evaluated at x = (x 1,..., x n) .
Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
Differentiation Facts
(i) If F 0(x) exists, it is unique. (ii) If F 0(x) exists, then F 0(x; d) exists for all d and F 0(x; d) = F 0(x)d.
Multivariable Calculus Review (iv) (Matrix Representation) Suppose F 0(x) is continuous at x, Then
∂F1 ∂F1 ∂F1 ∂x ∂x ··· ∂x T 1 2 n ∇F1(¯x) ∂F2 ∂F2 ··· ∂F2 T ∂x1 ∂x2 ∂xn ∇F2(¯x) F 0(x) = ∇F (x) = = , . . . . ∇F (¯x)T ∂Fn ······ ∂Fm m ∂x1 ∂xn
T where each partial derivative is evaluated at x = (x 1,..., x n) .
Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
Differentiation Facts
(i) If F 0(x) exists, it is unique. (ii) If F 0(x) exists, then F 0(x; d) exists for all d and F 0(x; d) = F 0(x)d. (iii) If F 0(x) exists, then F is continuous at x.
Multivariable Calculus Review Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
Differentiation Facts
(i) If F 0(x) exists, it is unique. (ii) If F 0(x) exists, then F 0(x; d) exists for all d and F 0(x; d) = F 0(x)d. (iii) If F 0(x) exists, then F is continuous at x. (iv) (Matrix Representation) Suppose F 0(x) is continuous at x, Then
∂F1 ∂F1 ∂F1 ∂x ∂x ··· ∂x T 1 2 n ∇F1(¯x) ∂F2 ∂F2 ··· ∂F2 T ∂x1 ∂x2 ∂xn ∇F2(¯x) F 0(x) = ∇F (x) = = , . . . . ∇F (¯x)T ∂Fn ······ ∂Fm m ∂x1 ∂xn
T where each partial derivative is evaluated at x = (x 1,..., x n) .
Multivariable Calculus Review k n Let G : B ⊂ R → R be differentiable on the open set B.
If F (A) ⊂ B, then the composite function G ◦ F is differentiable on A and (G ◦ F )0(x) = G 0(F (x)) ◦ F 0(x).
Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
Chain Rule
m k Let F : A ⊂ R → R be differentiable on the open set A.
Multivariable Calculus Review If F (A) ⊂ B, then the composite function G ◦ F is differentiable on A and (G ◦ F )0(x) = G 0(F (x)) ◦ F 0(x).
Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
Chain Rule
m k Let F : A ⊂ R → R be differentiable on the open set A.
k n Let G : B ⊂ R → R be differentiable on the open set B.
Multivariable Calculus Review Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
Chain Rule
m k Let F : A ⊂ R → R be differentiable on the open set A.
k n Let G : B ⊂ R → R be differentiable on the open set B.
If F (A) ⊂ B, then the composite function G ◦ F is differentiable on A and (G ◦ F )0(x) = G 0(F (x)) ◦ F 0(x).
Multivariable Calculus Review (a) If f : R → R is differentiable, then for every x, y ∈ R there exists z between x and y such that f (y) = f (x) + f 0(z)(y − x).
n (b) If f : R → R is differentiable, then for every x, y ∈ R there is a z ∈ [x, y] such that f (y) = f (x) + ∇f (z)T (y − x).
n m (c) If F : R → R continuously differentiable, then for every x, y ∈ R " # 0 kF (y) − F (x)kq ≤ sup kF (z)k(p,q) kx − ykp. z∈[x,y]
Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
The Mean Value Theorem
Multivariable Calculus Review n (b) If f : R → R is differentiable, then for every x, y ∈ R there is a z ∈ [x, y] such that f (y) = f (x) + ∇f (z)T (y − x).
n m (c) If F : R → R continuously differentiable, then for every x, y ∈ R " # 0 kF (y) − F (x)kq ≤ sup kF (z)k(p,q) kx − ykp. z∈[x,y]
Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
The Mean Value Theorem
(a) If f : R → R is differentiable, then for every x, y ∈ R there exists z between x and y such that f (y) = f (x) + f 0(z)(y − x).
Multivariable Calculus Review n m (c) If F : R → R continuously differentiable, then for every x, y ∈ R " # 0 kF (y) − F (x)kq ≤ sup kF (z)k(p,q) kx − ykp. z∈[x,y]
Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
The Mean Value Theorem
(a) If f : R → R is differentiable, then for every x, y ∈ R there exists z between x and y such that f (y) = f (x) + f 0(z)(y − x).
n (b) If f : R → R is differentiable, then for every x, y ∈ R there is a z ∈ [x, y] such that f (y) = f (x) + ∇f (z)T (y − x).
Multivariable Calculus Review Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
The Mean Value Theorem
(a) If f : R → R is differentiable, then for every x, y ∈ R there exists z between x and y such that f (y) = f (x) + f 0(z)(y − x).
n (b) If f : R → R is differentiable, then for every x, y ∈ R there is a z ∈ [x, y] such that f (y) = f (x) + ∇f (z)T (y − x).
n m (c) If F : R → R continuously differentiable, then for every x, y ∈ R " # 0 kF (y) − F (x)kq ≤ sup kF (z)k(p,q) kx − ykp. z∈[x,y]
Multivariable Calculus Review Proof: Set ϕ(t) = f (x + t(y − x)). Chain rule ⇒ ϕ0(t) = ∇f (x + t(y − x))T (y − x). In particular, ϕ is differentiable. By the 1-D MVT there exists t ∈ (0, 1) such that ϕ(1) = ϕ(0) + ϕ0(t)(1 − 0), or equivalently, f (y) = f (x) + ∇f (z)T (y − x) where z = x + t(y − x).
Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
The Mean Value Theorem: Proof
n n If f : R → R is differentiable, then for every x, y ∈ R there exists z ∈ [x, y] such that f (y) = f (x) + ∇f (z)(y − x).
Multivariable Calculus Review Chain rule ⇒ ϕ0(t) = ∇f (x + t(y − x))T (y − x). In particular, ϕ is differentiable. By the 1-D MVT there exists t ∈ (0, 1) such that ϕ(1) = ϕ(0) + ϕ0(t)(1 − 0), or equivalently, f (y) = f (x) + ∇f (z)T (y − x) where z = x + t(y − x).
Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
The Mean Value Theorem: Proof
n n If f : R → R is differentiable, then for every x, y ∈ R there exists z ∈ [x, y] such that f (y) = f (x) + ∇f (z)(y − x). Proof: Set ϕ(t) = f (x + t(y − x)).
Multivariable Calculus Review In particular, ϕ is differentiable. By the 1-D MVT there exists t ∈ (0, 1) such that ϕ(1) = ϕ(0) + ϕ0(t)(1 − 0), or equivalently, f (y) = f (x) + ∇f (z)T (y − x) where z = x + t(y − x).
Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
The Mean Value Theorem: Proof
n n If f : R → R is differentiable, then for every x, y ∈ R there exists z ∈ [x, y] such that f (y) = f (x) + ∇f (z)(y − x). Proof: Set ϕ(t) = f (x + t(y − x)). Chain rule ⇒ ϕ0(t) = ∇f (x + t(y − x))T (y − x).
Multivariable Calculus Review By the 1-D MVT there exists t ∈ (0, 1) such that ϕ(1) = ϕ(0) + ϕ0(t)(1 − 0), or equivalently, f (y) = f (x) + ∇f (z)T (y − x) where z = x + t(y − x).
Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
The Mean Value Theorem: Proof
n n If f : R → R is differentiable, then for every x, y ∈ R there exists z ∈ [x, y] such that f (y) = f (x) + ∇f (z)(y − x). Proof: Set ϕ(t) = f (x + t(y − x)). Chain rule ⇒ ϕ0(t) = ∇f (x + t(y − x))T (y − x). In particular, ϕ is differentiable.
Multivariable Calculus Review or equivalently, f (y) = f (x) + ∇f (z)T (y − x) where z = x + t(y − x).
Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
The Mean Value Theorem: Proof
n n If f : R → R is differentiable, then for every x, y ∈ R there exists z ∈ [x, y] such that f (y) = f (x) + ∇f (z)(y − x). Proof: Set ϕ(t) = f (x + t(y − x)). Chain rule ⇒ ϕ0(t) = ∇f (x + t(y − x))T (y − x). In particular, ϕ is differentiable. By the 1-D MVT there exists t ∈ (0, 1) such that ϕ(1) = ϕ(0) + ϕ0(t)(1 − 0),
Multivariable Calculus Review Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
The Mean Value Theorem: Proof
n n If f : R → R is differentiable, then for every x, y ∈ R there exists z ∈ [x, y] such that f (y) = f (x) + ∇f (z)(y − x). Proof: Set ϕ(t) = f (x + t(y − x)). Chain rule ⇒ ϕ0(t) = ∇f (x + t(y − x))T (y − x). In particular, ϕ is differentiable. By the 1-D MVT there exists t ∈ (0, 1) such that ϕ(1) = ϕ(0) + ϕ0(t)(1 − 0), or equivalently, f (y) = f (x) + ∇f (z)T (y − x) where z = x + t(y − x).
Multivariable Calculus Review This is an n × n matrix: If ∇f exists and is continuous at x, then it is given as the matrix of second partials of f at x:
∂2f ∇2f (x) = (x) . ∂xi ∂xj
Moreover, ∂f = ∂f for all i, j = 1,..., n. The matrix ∂xi ∂xj ∂xj ∂xi ∇2f (x) is called the Hessian of f at x. It is a symmetric matrix.
Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
n n n Let f : R → R so that ∇f : R → R . The second derivative of f is just the derivative of ∇f as a n n mapping from R to R : ∇[∇f (x)] = ∇2f (x).
Multivariable Calculus Review If ∇f exists and is continuous at x, then it is given as the matrix of second partials of f at x:
∂2f ∇2f (x) = (x) . ∂xi ∂xj
Moreover, ∂f = ∂f for all i, j = 1,..., n. The matrix ∂xi ∂xj ∂xj ∂xi ∇2f (x) is called the Hessian of f at x. It is a symmetric matrix.
Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
The Second Derivative
n n n Let f : R → R so that ∇f : R → R . The second derivative of f is just the derivative of ∇f as a n n mapping from R to R : ∇[∇f (x)] = ∇2f (x).
This is an n × n matrix:
Multivariable Calculus Review Moreover, ∂f = ∂f for all i, j = 1,..., n. The matrix ∂xi ∂xj ∂xj ∂xi ∇2f (x) is called the Hessian of f at x. It is a symmetric matrix.
Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
The Second Derivative
n n n Let f : R → R so that ∇f : R → R . The second derivative of f is just the derivative of ∇f as a n n mapping from R to R : ∇[∇f (x)] = ∇2f (x).
This is an n × n matrix: If ∇f exists and is continuous at x, then it is given as the matrix of second partials of f at x:
∂2f ∇2f (x) = (x) . ∂xi ∂xj
Multivariable Calculus Review The matrix ∇2f (x) is called the Hessian of f at x. It is a symmetric matrix.
Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
The Second Derivative
n n n Let f : R → R so that ∇f : R → R . The second derivative of f is just the derivative of ∇f as a n n mapping from R to R : ∇[∇f (x)] = ∇2f (x).
This is an n × n matrix: If ∇f exists and is continuous at x, then it is given as the matrix of second partials of f at x:
∂2f ∇2f (x) = (x) . ∂xi ∂xj
Moreover, ∂f = ∂f for all i, j = 1,..., n. ∂xi ∂xj ∂xj ∂xi
Multivariable Calculus Review Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
The Second Derivative
n n n Let f : R → R so that ∇f : R → R . The second derivative of f is just the derivative of ∇f as a n n mapping from R to R : ∇[∇f (x)] = ∇2f (x).
This is an n × n matrix: If ∇f exists and is continuous at x, then it is given as the matrix of second partials of f at x:
∂2f ∇2f (x) = (x) . ∂xi ∂xj
Moreover, ∂f = ∂f for all i, j = 1,..., n. The matrix ∂xi ∂xj ∂xj ∂xi ∇2f (x) is called the Hessian of f at x. It is a symmetric matrix.
Multivariable Calculus Review Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
Second-Order Taylor Theorem
n If f : R → R is twice continuously differentiable on an open set containing [x, y], then there is a z ∈ [x, y] such that
1 f (y) = f (x) + ∇f (x)T (y − x) + (y − x)T ∇2f (z)(y − x). 2
shown that
0 1 2 2 |f (y) − (f (x) + f (x)(y − x))| ≤ kx − ykp sup k∇ f (z)k(p,q), 2 z∈[x,y]
for any choice of p and q from [1, ∞].
Multivariable Calculus Review