Outline Convexity Convexity and optimality The Subdifferential Inequality Convexity and Optimality Conditions Sublinear Functionals Examples of Convex Functions Math 408A Convexity Math 408A Convexity Outline Convexity Convexity and optimality The Subdifferential Inequality Convexity and Optimality Conditions Sublinear Functionals Examples of Convex Functions Convexity Convexity and optimality Directional Derivatives and Convexity The Subdifferential Inequality Convexity and Optimality Conditions Sublinear Functionals Examples of Convex Functions Math 408A Convexity n Definition: A function f : R ! R is said to be convex if for every n two points x1; x2 2 R and λ 2 [0; 1] we have f (λx1 + (1 − λ)x2) ≤ λf (x1) + (1 − λ)f (x2): The function f is said to be strictly convex if for every two distinct n points x1; x2 2 R and λ 2 [0; 1] we have f (λx1 + (1 − λ)x2) < λf (x1) + (1 − λ)f (x2): Outline Convexity Convexity and optimality The Subdifferential Inequality Convexity and Optimality Conditions Sublinear Functionals Examples of Convex Functions Convexity n Definition: A set C ⊂ R is said to be convex if for every x; y 2 C and λ 2 [0; 1] one has (1 − λ)x + λy 2 C : Math 408A Convexity The function f is said to be strictly convex if for every two distinct n points x1; x2 2 R and λ 2 [0; 1] we have f (λx1 + (1 − λ)x2) < λf (x1) + (1 − λ)f (x2): Outline Convexity Convexity and optimality The Subdifferential Inequality Convexity and Optimality Conditions Sublinear Functionals Examples of Convex Functions Convexity n Definition: A set C ⊂ R is said to be convex if for every x; y 2 C and λ 2 [0; 1] one has (1 − λ)x + λy 2 C : n Definition: A function f : R ! R is said to be convex if for every n two points x1; x2 2 R and λ 2 [0; 1] we have f (λx1 + (1 − λ)x2) ≤ λf (x1) + (1 − λ)f (x2): Math 408A Convexity Outline Convexity Convexity and optimality The Subdifferential Inequality Convexity and Optimality Conditions Sublinear Functionals Examples of Convex Functions Convexity n Definition: A set C ⊂ R is said to be convex if for every x; y 2 C and λ 2 [0; 1] one has (1 − λ)x + λy 2 C : n Definition: A function f : R ! R is said to be convex if for every n two points x1; x2 2 R and λ 2 [0; 1] we have f (λx1 + (1 − λ)x2) ≤ λf (x1) + (1 − λ)f (x2): The function f is said to be strictly convex if for every two distinct n points x1; x2 2 R and λ 2 [0; 1] we have f (λx1 + (1 − λ)x2) < λf (x1) + (1 − λ)f (x2): Math 408A Convexity (x 2 , f (x 2 )) (x 1 , f (x 1 )) x λ λ x 1 x 1 + (1 - )x 2 2 That is, the set epi (f ) = f(x; µ): f (x) ≤ µg; called the epi-graph of f , is a convex set. Outline Convexity Convexity and optimality The Subdifferential Inequality Convexity and Optimality Conditions Sublinear Functionals Examples of Convex Functions Convexity The secant line connecting (x1; f (x1)) and (x2; f (x2)) lies above the graph of f . Math 408A Convexity (x 2 , f (x 2 )) (x 1 , f (x 1 )) x λ λ x 1 x 1 + (1 - )x 2 2 That is, the set epi (f ) = f(x; µ): f (x) ≤ µg; called the epi-graph of f , is a convex set. Outline Convexity Convexity and optimality The Subdifferential Inequality Convexity and Optimality Conditions Sublinear Functionals Examples of Convex Functions Convexity The secant line connecting (x1; f (x1)) and (x2; f (x2)) lies above the graph of f . Math 408A Convexity (x 2 , f (x 2 )) (x 1 , f (x 1 )) x λ λ x 1 x 1 + (1 - )x 2 2 Outline Convexity Convexity and optimality The Subdifferential Inequality Convexity and Optimality Conditions Sublinear Functionals Examples of Convex Functions Convexity The secant line connecting (x1; f (x1)) and (x2; f (x2)) lies above the graph of f . That is, the set epi (f ) = f(x; µ): f (x) ≤ µg; called the epi-graph of f , is a convex set. Math 408A Convexity Examples of convex functions are cT x; kxk; xT Hx when H is psd. Outline Convexity Convexity and optimality The Subdifferential Inequality Convexity and Optimality Conditions Sublinear Functionals Examples of Convex Functions Convexity n Definition: A function f : R ! R [ f+1g = R¯ is said to be convex if the set epi (f ) = f(x; µ): f (x) ≤ µg is a convex set. We also define the essential domain of f to be the set dom (f ) = fx : f (x) < +1g : Math 408A Convexity Outline Convexity Convexity and optimality The Subdifferential Inequality Convexity and Optimality Conditions Sublinear Functionals Examples of Convex Functions Convexity n Definition: A function f : R ! R [ f+1g = R¯ is said to be convex if the set epi (f ) = f(x; µ): f (x) ≤ µg is a convex set. We also define the essential domain of f to be the set dom (f ) = fx : f (x) < +1g : Examples of convex functions are cT x; kxk; xT Hx when H is psd. Math 408A Convexity Outline Convexity Convexity and optimality The Subdifferential Inequality Convexity and Optimality Conditions Sublinear Functionals Examples of Convex Functions Convexity and Optimality n n Theorem: Let f : R ! R¯ be convex. If x 2 R is a local minimum for f , then x is a global minimum for f . If, in addition, f is strictly convex, then x¯ is the unique global minimizer. Math 408A Convexity −1 Set λ := (2kx − bxk) < 1 and xλ := x + λ(bx − x). Then kxλ − xk ≤ /2 and f (xλ) ≤ (1 − λ)f (x) + λf (bx) < f (x): This contradicts the choice of and so no such bx exists. Outline Convexity Convexity and optimality The Subdifferential Inequality Convexity and Optimality Conditions Sublinear Functionals Examples of Convex Functions Convexity and Optimality Proof: Suppose f (bx) < f (x). Let > 0 be such that f (x) ≤ f (x) whenever kx − xk ≤ and < 2kx − bxk : Math 408A Convexity Then kxλ − xk ≤ /2 and f (xλ) ≤ (1 − λ)f (x) + λf (bx) < f (x): This contradicts the choice of and so no such bx exists. Outline Convexity Convexity and optimality The Subdifferential Inequality Convexity and Optimality Conditions Sublinear Functionals Examples of Convex Functions Convexity and Optimality Proof: Suppose f (bx) < f (x). Let > 0 be such that f (x) ≤ f (x) whenever kx − xk ≤ and < 2kx − bxk : −1 Set λ := (2kx − bxk) < 1 and xλ := x + λ(bx − x). Math 408A Convexity This contradicts the choice of and so no such bx exists. Outline Convexity Convexity and optimality The Subdifferential Inequality Convexity and Optimality Conditions Sublinear Functionals Examples of Convex Functions Convexity and Optimality Proof: Suppose f (bx) < f (x). Let > 0 be such that f (x) ≤ f (x) whenever kx − xk ≤ and < 2kx − bxk : −1 Set λ := (2kx − bxk) < 1 and xλ := x + λ(bx − x). Then kxλ − xk ≤ /2 and f (xλ) ≤ (1 − λ)f (x) + λf (bx) < f (x): Math 408A Convexity Outline Convexity Convexity and optimality The Subdifferential Inequality Convexity and Optimality Conditions Sublinear Functionals Examples of Convex Functions Convexity and Optimality Proof: Suppose f (bx) < f (x). Let > 0 be such that f (x) ≤ f (x) whenever kx − xk ≤ and < 2kx − bxk : −1 Set λ := (2kx − bxk) < 1 and xλ := x + λ(bx − x). Then kxλ − xk ≤ /2 and f (xλ) ≤ (1 − λ)f (x) + λf (bx) < f (x): This contradicts the choice of and so no such bx exists. Math 408A Convexity n 1. Given x; d 2 R the difference quotient f (x + td) − f (x) t is a non-decreasing function of t on (0; +1). n 0 2. For every x; d 2 R the directional derivative f (x; d) always exists and is given by f (x + td) − f (x) f 0(x; d) := inf : t>0 t Outline Convexity Convexity and optimality The Subdifferential Inequality Convexity and Optimality Conditions Sublinear Functionals Examples of Convex Functions Directional Derivatives and Convexity Directional Derivatives and Convexity n Lemma: Let f : R ! R¯ be convex (not necessarilty differentiable). Math 408A Convexity n 0 2. For every x; d 2 R the directional derivative f (x; d) always exists and is given by f (x + td) − f (x) f 0(x; d) := inf : t>0 t Outline Convexity Convexity and optimality The Subdifferential Inequality Convexity and Optimality Conditions Sublinear Functionals Examples of Convex Functions Directional Derivatives and Convexity Directional Derivatives and Convexity n Lemma: Let f : R ! R¯ be convex (not necessarilty differentiable). n 1. Given x; d 2 R the difference quotient f (x + td) − f (x) t is a non-decreasing function of t on (0; +1). Math 408A Convexity Outline Convexity Convexity and optimality The Subdifferential Inequality Convexity and Optimality Conditions Sublinear Functionals Examples of Convex Functions Directional Derivatives and Convexity Directional Derivatives and Convexity n Lemma: Let f : R ! R¯ be convex (not necessarilty differentiable). n 1. Given x; d 2 R the difference quotient f (x + td) − f (x) t is a non-decreasing function of t on (0; +1). n 0 2. For every x; d 2 R the directional derivative f (x; d) always exists and is given by f (x + td) − f (x) f 0(x; d) := inf : t>0 t Math 408A Convexity Recall that f (x + td) − f (x) f 0(x; d) := lim : t#0 t Now if the difference quotient on the right is non-decreasing in t on (0; +1), then the limit is necessarily given by the infimum of these ratios.