Maxima, Minima, and Saddle Points

MATH2111 Higher Several Variable Calculus Maxima, Minima and Saddle Points

Dr. Jonathan Kress

School of Mathematics and Statistics University of New South Wales

Semester 1, 2016 [updated: March 21, 2016]

JM Kress (UNSW Maths & Stats) MATH2111 Analysis Semester 1, 2016 1 / 32

Maxima, minima and saddle points

Denition n Suppose f :Ω ⊂ R → R. Then a ∈ Ω is an absolute or global maximum of f if f (a) ≥ f (x) for all x ∈ Ω. a ∈ Ω is an absolute or global minimum of f if f (a) ≤ f (x) for all x ∈ Ω. a ∈ Ω is a local maximum of f if there is an open set A containing a such that f (a) ≥ f (x) for all x ∈ Ω ∩ A. a ∈ Ω is a local minimum of f if there is an open set A containing a such that f (a) ≤ f (x) for all x ∈ Ω ∩ A. a ∈ Ω is a stationary point of f if f is dierentiable at a and ∇f (a) = 0. a ∈ Ω is a saddle point of f if it is a stationary point of f but is neither a local maximum nor minimum point of f .

JM Kress (UNSW Maths & Stats) MATH2111 Analysis Semester 1, 2016 2 / 32 Maxima, minima and saddle points

Theorem n Suppose f :Ω ⊂ R → R. Then local and maxima and minima can only occur at a ∈ Ω where a satises one of the following: (1) a is a stationary point, (2) a lies on the boundary of Ω or (3) f is not dierentiable at a.

Denition Points satisfying at least one of (1), (2) or (3) in the theorem above are called critical points.

JM Kress (UNSW Maths & Stats) MATH2111 Analysis Semester 1, 2016 3 / 32

Maxima, minima and saddle points

2 Consider Ω, the region of R bounded by x = 0, y = 0 and y = x + 3. Find the maximum and minimum values Ý of f :Ω → R, given by, ¿ f (x, y) = x 3 − y 3 − 3xy. Ω f is continuous and dierentiable on Ω which is compact.

Hence f (Ω) is compact and so maximum and minimum

Ü −¿ values exist and are attained on Ω. Since f is dierentiable everywhere, the extrema must occur at (1) stationary points f or (2) boundary points of Ω. Stationary points of f occur when ∇f = 0 ⇔ (3x 2 − 3y, −3y 2 − 3x) = (0, 0) ⇔ y = x 2 and x = −y 2 ⇒ y = x 2 ⇒ x 4 + x = 0 ⇒ (x 3 + 1)x = 0. Hence the only stationary points of f are (0, 0) and (−1, 1). Also note that f (0, 0) = 0 and f (−1, 1) = 1. JM Kress (UNSW Maths & Stats) MATH2111 Analysis Semester 1, 2016 4 / 32

Maxima, minima and saddle points Ý

Divide the boundary into 3 pieces. First consider B1. ¿ B

B1 = {(0, t): 0 ≤ t ≤ 3}, ¿ Ω

B ½ B2 = {(t, 0): −3 ≤ t ≤ 0},

B3 = {(t, t + 3): −3 ≤ t ≤ 0}.

−¿

f Ø (¼, ) On B1

f (0, t) = 03 − t3 − 0 = −t3

Ø ¿ for t ∈ [0, 3].

So the max on B1 is at (0, 0) where f (0, 0) = 0 and the −¾7 min is at (0, 3) where f (0, 3) = −27.

JM Kress (UNSW Maths & Stats) MATH2111 Analysis Semester 1, 2016 5 / 32

Maxima, minima and saddle points Ý

Next consider B2. ¿ B

B1 = {(0, t): 0 ≤ t ≤ 3}, ¿ Ω

B ½ B2 = {(t, 0): −3 ≤ t ≤ 0},

B3 = {(t, t + 3): −3 ≤ t ≤ 0}.

−¿

f ¼ (Ø , ) On B2

f (t, 0) = t3 − 03 − 0 = t3

Ø −¿ for t ∈ [−3, 0].

So the max on B2 is at (0, 0) where f (0, 0) = 0 and the −¾7 min is at (−3, 0) where f (−3, 0) = −27.

JM Kress (UNSW Maths & Stats) MATH2111 Analysis Semester 1, 2016 6 / 32

Maxima, minima and saddle points Ý

Lastly consider B3. ¿ B

B1 = {(0, t): 0 ≤ t ≤ 3}, ¿ Ω

B ½ B2 = {(t, 0): −3 ≤ t ≤ 0},

B3 = {(t, t + 3): −3 ≤ t ≤ 0}.

−¿

f Ø ¿ (Ø , + ) On B3

3 3 2 Ø −¿ f (t, t + 3) = t − (t + 3) − 3t(t + 3) = −3(4t + 12t + 9)

for t ∈ [−3, 0]. Now, g(t) = f (t, t + 3) has a stationary point when

−¾7 3 8t 12 0 t + = ⇒ = −2.

Extreme values can occur on B3 at the end points (already considered) or the stationary point where

3 3 f 0 JM Kress (UNSW Maths & Stats) MATH2111 Analysis −2, 2 = . Semester 1, 2016 7 / 32

Maxima, minima and saddle points

So we have a number of candidate points for the extreme

values of f . Ý ¿

f (−1, 1) = 1 B

¿ Ω B f (0, 0) = 0 ½ f (0, 3) = −27

f 3 0 27 Ü

(− , ) = − −¿

B ¾ f (−1.5, 1.5) = 0

Hence the maximum of f on Ω is 1 and the minimum value of f on Ω is −27.

JM Kress (UNSW Maths & Stats) MATH2111 Analysis Semester 1, 2016 8 / 32 Classication of stationary points

2 The following functions f : R → R have a stationary point at (0, 0). Is it a local maximum, minimum or saddle point? (i) f (x, y) = x 2 + y 2 (ii) f (x, y) = −x 2 − y 2 (iii) f (x, y) = x 2 − y 2 (iv) f (x, y) = xy (v) f (x, y) = x 2 + y 4 (vi) f (x, y) = x 2 − y 4 (vii) f (x, y) = x 2 − 6xy + y 2 (viii) f (x, y) = 3x 2 − 2xy + 3y 2

JM Kress (UNSW Maths & Stats) MATH2111 Analysis Semester 1, 2016 9 / 32

Classication of stationary points

(i) f (x, y) = x 2 + y 2 (ii) f (x, y) = −x 2 − y 2

Local minimum at (0, 0). Local maximum at (0, 0).

JM Kress (UNSW Maths & Stats) MATH2111 Analysis Semester 1, 2016 10 / 32 Classication of stationary points

(iii) f (x, y) = x 2 − y 2 For all > 0,

2 0 B 0 0 with f 0 2, ∈ (( , ), ) 2, = 4 and

2 0 B 0 0 with f 0 , 2 ∈ (( , ), ) , 2 = − 4 . So, f 0 f 0 0 f 0 , 2 < ( , ) < 2, . That is, (0, 0) is a stationary point that is neither Along y = 0, f (x, 0) = x 2 and a local max nor min and hence is a saddle point. (0, 0) is a local minimum. Along x = 0, f (0, y) = −y 2 and (0, 0) is a local maximum.

JM Kress (UNSW Maths & Stats) MATH2111 Analysis Semester 1, 2016 11 / 32

Classication of stationary points

(iv) f (x, y) = xy Along y = x,

f (x, x) = x 2

which has a local minimum at (0, 0). Along y = −x,

f (x, −x) = −x 2

which has a local maximum at (0, 0). So (0, 0) is neither a local maximum nor local minimum. Hence f has a saddle point at (0, 0). Note that 1 f x y x y 2 x y 2 ( , ) = 4 ( + ) − ( − ) .

JM Kress (UNSW Maths & Stats) MATH2111 Analysis Semester 1, 2016 12 / 32 Classication of stationary points

(v) f (x, y) = x 2 + y 4 (iv) f (x, y) = x 2 − y 4

Local minimum at (0, 0). Saddle point at (0, 0).

JM Kress (UNSW Maths & Stats) MATH2111 Analysis Semester 1, 2016 13 / 32

Classication of stationary points

(vii) f (x, y) = x 2 − 6xy + y 2 (viii) f (x, y) = 3x 2 − 2xy + 3y 2

Saddle point at (0, 0). Local minimum at (0, 0).

JM Kress (UNSW Maths & Stats) MATH2111 Analysis Semester 1, 2016 14 / 32 Classication of stationary points

(vii) Let 1 1 −1 P = √ f (x, y) = x 2 − 6xy + y 2 2 1 1 1 −3 x and then = x y . −3 1 y 1 1 1 P−1 = PT = √ . 2 −1 1 Let 1 −3 H = . So −3 1 −2 0 PT HP = D = . H has eigenvalues and eigenvectors 0 4 Now make a change of variables 1 λ1 = −2, v1 = , 1 x X = P . 1 y Y 4 v − λ2 = , 2 = 1 .

So we can orthoganally diagonalise H.

JM Kress (UNSW Maths & Stats) MATH2111 Analysis Semester 1, 2016 15 / 32

Classication of stationary points

x X P x y XY PT y = Y . ⇒ = . So, x X f x y x y H XY PT HP ( , ) = y = Y 2 0 X XY − = 0 4 Y = −2X 2 + 4Y 2 Note that 1 √ x y ! X T x 1 1 1 x 2 ( + ) = P = √ = 1 . Y y 1 1 y √ y x 2 − 2 ( − ) So, 1 2 1 2 f (x, y) = −2 √ (x + y) + 4 √ (y − x) = −(x + y)2 + 2(x − y)2. 2 2

JM Kress (UNSW Maths & Stats) MATH2111 Analysis Semester 1, 2016 16 / 32 Classication of stationary points

(viii)

3 −1 x f (x, y) = x y −1 3 y The eigenvalues and eigenvectors of

3 −1 H = −1 3 are 1 2 v λ1 = , 1 = 1 , 1 4 v − λ2 = , 2 = 1 .

Diagonalising and rotating the coordinates leads to

f (x, y) = 2X 2 + 4Y 2 = (x + y)2 + 2(x − y)2.

JM Kress (UNSW Maths & Stats) MATH2111 Analysis Semester 1, 2016 17 / 32

Classication of stationary points

The `Taylor series' of f at a stationary point (a, b) is ( (((( f (x, y) = f (a, b) + ∇f (a, b) ·((x,(y) − (a, b) ((((  ∂2f ∂2f  1 x2 (a, b) y x (a, b) x a x a y b ∂ ∂ ∂ − + − −  2 2  2! ∂ f a b ∂ f a b y − b ∂x∂y ( , ) ∂y 2 ( , ) + ··· terms involving higher powers of (x − a) and (y − b) since ∇f (a, b) = (0, 0).

For (x, y) close to (a, b) the nature of the stationary point will be determined by the eigenvalues of the matrix

 ∂2f ∂2f  2 (a, b) (a, b) H ∂x ∂y∂x =  2 2  . ∂ f a b ∂ f a b ∂x∂y ( , ) ∂y 2 ( , )

JM Kress (UNSW Maths & Stats) MATH2111 Analysis Semester 1, 2016 18 / 32 Classication of stationary points

2 2 Suppose f :Ω ⊂ R → R is C and has a stationary point at (a, b), that is, ∇f (a, b) = 0. So Taylor's theorem says that

f (x, y) = f (a, b) + R1,(a,b)(x, y) where the remainder term is given by 1 x − a R1 a b (x, y) = x − a y − b H ,( , ) 2! y − b

 ∂2f ∂2f  2 (c, d) (c, d) where H ∂x ∂y∂x =  2 2  ∂ f c d ∂ f c d ∂x∂y ( , ) ∂y 2 ( , ) for some point (c, d) between (a, b) and (x, y).

Can the eigenvalues of H be used to determine whether f has a local max, min or saddle point at (a, b)? H is made of partial derivatives evaluated at an unknown point (c, d). Can we determine the nature of the stationary point using partial derivatives calculated at (a, b)? Yes, on a suciently small ball. Why? JM Kress (UNSW Maths & Stats) MATH2111 Analysis Semester 1, 2016 19 / 32

Maxima, minima and saddle points

Denition n For f :Ω ⊂ R → R the Hessian of f at a is the n × n matrix  2f 2f 2f  ∂ a ∂ a ∂ a 2 ( ) ( ) ··· ( )  ∂x1 ∂x2∂x1 ∂xn∂x1   2f 2f 2f   ∂ a ∂ a ∂ a   ( ) 2 ( ) ··· ( )  ∂x1∂x2 ∂x ∂xn∂x2  H(f , a) =  2  .  . . .. .   . . . .     2f 2f 2f   ∂ a ∂ a ∂ a  ( ) ( ) ··· 2 ( ) ∂x1∂xn ∂x2∂xn ∂xn

JM Kress (UNSW Maths & Stats) MATH2111 Analysis Semester 1, 2016 20 / 32 Classication of stationary points

The signs of the eigenvalues of

 ∂2f ∂2f  2 (a, b) (a, b) H f a b ∂x ∂y∂x ( , ( , )) =  2 2  ∂ f a b ∂ f a b ∂x∂y ( . ) ∂y 2 ( , ) can be determined from the signs of the trace1 and determinant of H(f , (a, b)). Tr H(f , (a, b)) = sum of eigenvalues and det H(f , (a, b)) = product of eigenvalues. These are continuous functions of the entries in the matrix which are continuous by the assumption that f is C 2. Hence there must be a open ball around (a, b) on which the trace and determinant (and hence the eigenvalues) of the Hessian have the same signs as those of the Hessian at (a, b).

1The trace of a matrix is the sum of its diagonal entries. JM Kress (UNSW Maths & Stats) MATH2111 Analysis Semester 1, 2016 21 / 32

e-values-2x2-matrix.canvas

JM Kress (UNSW Maths & Stats) MATH2111 Analysis Semester 1, 2016 22 / 32 Maxima, minima and saddle points

Find the eigenvalues of the Hessian of f at (0, 0) for each of the functions we considered last lecture.  2f 2f  ∂ 0 0 ∂ 0 0 2 ( , ) ( , ) ∂x ∂x2∂x1 H(f , (0, 0)) =  1  .  2f 2f   ∂ 0 0 ∂ 0 0  ( , ) 2 ( , ) ∂x1∂x2 ∂x2

(i) f (x, y) = x 2 + y 2 (ii) f (x, y) = −x 2 − y 2

2 0 −2 0 H(f , (0, 0)) = . H(f , (0, 0)) = . 0 2 0 −2

Eigenvalues are 2, 2. Eigenvalues are −2, −2.

JM Kress (UNSW Maths & Stats) MATH2111 Analysis Semester 1, 2016 23 / 32

Maxima, minima and saddle points

(iii) f x y x 2 y 2 ( , ) = − (vi) f (x, y) = x 2 − y 4 2 0 2 0 H(f , (0, 0)) = . H f 0 0 0 −2 ( , ( , )) = 0 0 . Eigenvalues are 2 2. , − Eigenvalues are 2, 0. (iv) f x y xy ( , ) = (vii) f (x, y) = x 2 − 6xy + y 2 0 1 H f 0 0 2 −6 ( , ( , )) = 1 0 . H(f , (0, 0)) = . −6 2 Eigenvalues are 1 1. , − Eigenvalues are −4, 8. (v) f x y x 2 y 4 ( , ) = + (viii) f (x, y) = 3x 2 − 2xy + 3y 2 2 0 H f 0 0 6 −2 ( , ( , )) = 0 0 . H(f , (0, 0)) = . −2 6 Eigenvalues are 2 0. , Eigenvalues are 4, 8. JM Kress (UNSW Maths & Stats) MATH2111 Analysis Semester 1, 2016 24 / 32 Classication of stationary points

Denition An n × n matrix H is positive denite ⇔ all eigenvalues are > 0 positive semidenite ⇔ all eigenvalues are ≥ 0 negative denite ⇔ all eigenvalues are < 0 negative semidenite ⇔ all eigenvalues are ≤ 0

Theorem (Alternative test Sylvester's criterion)

If Hk is the upper left k × k submatrix of H and 4k = det Hk then H is

positive denite ⇔ 4k > 0 for all k positive semidenite ⇒ 4k ≥ 0 for all k negative denite ⇔ 4k < 0 for all odd k and 4k > 0 for all even k negative semidenite ⇒ 4k ≤ 0 for all odd k and 4k ≥ 0 for all even k

JM Kress (UNSW Maths & Stats) MATH2111 Analysis Semester 1, 2016 25 / 32

Classication of stationary points

Theorem n 2 Suppose f :Ω ⊂ R → R is C and ∇f (a) = 0 at an interior point a of Ω. Then H(f , a) is positive denite ⇒ f has a local minimum at a. H(f , a) is negative denite ⇒ f has a local maximum at a. f has a local minimum at a ⇒ H(f , a) is positive semidenite. f has a local maximum at a ⇒ H(f , a) is negative semidenite.

JM Kress (UNSW Maths & Stats) MATH2111 Analysis Semester 1, 2016 26 / 32 Classication of stationary points

2 For f : R → R with a stationary point at (a, b), 2 ∂2f ∂2f ∂2f ∂2f 41 = (a, b) and 42 = (a, b) (a, b) − (a, b) . ∂x 2 ∂x 2 ∂y 2 ∂x∂y Then

41 > 0 and 42 > 0 (two positive eigenvalues) ⇒ (a, b) is a local minimum.

41 < 0 and 42 > 0 (two negative eigenvalues) ⇒ (a, b) is a local maximum.

local minimum at (a, b) ⇒ 41 ≥ 0 and 42 ≥ 0 (no negative eigenvalues).

local maximum at (a, b) ⇒ 41 ≤ 0 and 42 ≥ 0 (no positive eigenvalues). Notes:

42 < 0 ⇒ (a, b) is a saddle point (one positive and one negative eigenvalue). The semidenite case can also be a saddle point.

JM Kress (UNSW Maths & Stats) MATH2111 Analysis Semester 1, 2016 27 / 32

Classication of stationary points

Find and classify the stationary points of

f (x, y) = x 3 + 6x 2 + 3y 2 − 12xy + 9x.

Stationary points occur when ∇f = 0, that is,

(3x 2 + 12x − 12y + 9, 6y − 12x) = (0, 0)

( 2 3x + 12x − 12y + 9 = 0 (1) ⇒ 6y − 12x = 0 (2) (2) ⇒ y = 2x which when substituted into (1) becomes

3(x − 3)(x − 1) = 0.

So x = 1 ⇒ y = 2 or x = 3 ⇒ y = 6.

So f has stationary points at (1, 2) and (3, 6).

JM Kress (UNSW Maths & Stats) MATH2111 Analysis Semester 1, 2016 28 / 32 Classication of stationary points

6x + 12 −12 f (x, y) = x 3 + 6x 2 + 3y 2 − 12xy + 9x. ⇒ H(f , (x, y)) = . −12 6

At (1, 2): At (3, 6):

18 −12 30 −12 H(f , (1, 2)) = H(f , (3, 6)) = −12 6 −12 6

42 = 18 × 6 − (−12) × (−12) 41 = 30 > 0,

= −36 < 0. 42 = 30 × 6 − (−12) × (−12) = 36 > 0. So (1, 2) is a saddle point of f . So (3, 6) is a local minimum point of f .

JM Kress (UNSW Maths & Stats) MATH2111 Analysis Semester 1, 2016 29 / 32

Classication of stationary points

Find and classify the stationary points of

f (x, y, z) = yx 2 + zy 2 + z2 − 2yx − 2zy + y − z.

 2xy − 2y = 0 (1)  ∇f = 0 ⇒ x 2 + 2zy − 2x − 2z + 1 = 0 (2)  2  y + 2z − 2y − 1 = 0 (3) (1) is 2y(x − 1) = 0 so there are two cases y = 0: x = 1: 3 z 1 . 2 z 0 or y 1. ( ) ⇒ = 2 ( ) ⇒ = = √ (2) ⇒ x = 0 or x = 2. For z = 0, (3) ⇒ y = 1 ± 2. So 0 0 1 and 2 0 1 are stationary For y = 1, (3) ⇒ z = 1. ( , , 2 ) ( , , 2 ) √ points. So, (1, 1 ± 2, 0) and (1, 1, 1) are stationary points. 1 1 √ √ f has 5 stationary points: (0, 0, 2 ), (2, 0, 2 ), (1, 1 + 2, 0), (1, 1 − 2, 0), (1, 1, 1).

JM Kress (UNSW Maths & Stats) MATH2111 Analysis Semester 1, 2016 30 / 32 Classication of stationary points

 2y 2x − 2 0  To classify we need H(f , (x, y)) = 2x − 2 2z 2y − 2 . 0 2y − 2 2

 0 2 0  1 − H f 0 0 −2 1 −2 ( , ( , , 2)) =   2 0 0 0 −2 2 H(f , (1, 1, 1)) = 0 2 0 0 0 2 0 −2 41 = 0, 42 = = −4 −2 1 41 = 2

0 −2 0 42 = 4

43 = −2 1 −2 = −8 43 = 8 0 −2 2 1 1 1 is a local minimum point as the 0 0 1 is a saddle point as the Hessian ( , , ) ( , , 2 ) Hessian is positive denite. is neither positive semidenite nor [Eigenvalues are 2, 2, 2.] negative semidenite. [Eigenvalues are 1, −2, 4.]

JM Kress (UNSW Maths & Stats) MATH2111 Analysis Semester 1, 2016 31 / 32

Classication of stationary points

√ √ H(f , (2, 0, 1 )) = H(f , (1, 1 + 2, 0)) = H(f , (1, 1 − 2, 0)) = 2 √ √ 0 2 0  2 2 2 0 0  2 2 2 0 0  + √ − √ 2 1 2 0 0 2 2 0 0 2 2  −   √   √ −  0 −2 2 0 2 2 2 0 −2 2 2 √ √ 41 = 0 41 = 2 + 2 2 41 = 2 − 2 2

42 = −4 42 = 0 42 = 0 √ √ 43 = −8 43 = −16 − 16 2 43 = −16 + 16 2

1 √ √ (2, 0, 2 ) is a saddle (1, 1 + 2, 0) is a saddle (1, 1 − 2, 0) is a saddle point as the Hessian is point as the Hessian is point as the Hessian is neither positive neither positive semidenite neither positive semidenite semidenite nor nor negative semidenite. nor negative semidenite. negative semidenite. [E'values are 2, 4, [E'values are 2, 4, √ − √ − [E'values are 1, −2, 4.] 2 + 2 2.] 2 − 2 2.]

JM Kress (UNSW Maths & Stats) MATH2111 Analysis Semester 1, 2016 32 / 32