Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals
Topics in Convex Analysis
Tim Hoheisel (McGill University, Montreal)
Spring School on Variational Analysis
Paseky, May 19–25, 2019 Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals
Outline
1 Fundamentals from Convex Analysis Convex sets and functions Subdifferentiation and conjugacy of convex functions Infimal convolution and the Attouch-Brezis´ Theorem Consequences of Attouch-Brezis´
2 Conjugacy of composite functions via K-convexity and inf-convolution K-convexity Composite functions and scalarization Conjugacy results Applications
3 A new class of matrix support functionals The generalized matrix-fractional function The closed convex hull of D(A, B) with applications Applications of the GMF Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals
1. Fundamentals from Convex Analysis Examples
E = Rn , hx, yi := xT y, κ = n E = Rm×n , hA, Bi := tr (A T B), κ = mn
Minkowski addition/multiplication: Let A ⊂ E A + B := {a + b | a ∈ A, b ∈ B } (B ⊂ E) A + x := A + {x} (x ∈ E) Λ · A := {λa | a ∈ A, λ ∈ Λ } (Λ ⊂ R) λA := {λ}· A (λ ∈ R)
Examples: U, V ⊂ E subspaces. Then U + V = span (U ∪ V)
Bε(x) = x + εB
pos S := R+S (conical hull)
Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals
Convex sets and functions The Euclidean setting and Minkowski notation
In what follows E will be a Euclidean space, i.e. a real-vector space equipped with an inner product h·, ·i : E × E → R of dimension κ < ∞. Minkowski addition/multiplication: Let A ⊂ E A + B := {a + b | a ∈ A, b ∈ B } (B ⊂ E) A + x := A + {x} (x ∈ E) Λ · A := {λa | a ∈ A, λ ∈ Λ } (Λ ⊂ R) λA := {λ}· A (λ ∈ R)
Examples: U, V ⊂ E subspaces. Then U + V = span (U ∪ V)
Bε(x) = x + εB
pos S := R+S (conical hull)
Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals
Convex sets and functions The Euclidean setting and Minkowski notation
In what follows E will be a Euclidean space, i.e. a real-vector space equipped with an inner product h·, ·i : E × E → R of dimension κ < ∞. Examples
E = Rn , hx, yi := xT y, κ = n E = Rm×n , hA, Bi := tr (A T B), κ = mn A + x := A + {x} (x ∈ E) Λ · A := {λa | a ∈ A, λ ∈ Λ } (Λ ⊂ R) λA := {λ}· A (λ ∈ R)
Examples: U, V ⊂ E subspaces. Then U + V = span (U ∪ V)
Bε(x) = x + εB
pos S := R+S (conical hull)
Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals
Convex sets and functions The Euclidean setting and Minkowski notation
In what follows E will be a Euclidean space, i.e. a real-vector space equipped with an inner product h·, ·i : E × E → R of dimension κ < ∞. Examples
E = Rn , hx, yi := xT y, κ = n E = Rm×n , hA, Bi := tr (A T B), κ = mn
Minkowski addition/multiplication: Let A ⊂ E A + B := {a + b | a ∈ A, b ∈ B } (B ⊂ E) Λ · A := {λa | a ∈ A, λ ∈ Λ } (Λ ⊂ R) λA := {λ}· A (λ ∈ R)
Examples: U, V ⊂ E subspaces. Then U + V = span (U ∪ V)
Bε(x) = x + εB
pos S := R+S (conical hull)
Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals
Convex sets and functions The Euclidean setting and Minkowski notation
In what follows E will be a Euclidean space, i.e. a real-vector space equipped with an inner product h·, ·i : E × E → R of dimension κ < ∞. Examples
E = Rn , hx, yi := xT y, κ = n E = Rm×n , hA, Bi := tr (A T B), κ = mn
Minkowski addition/multiplication: Let A ⊂ E A + B := {a + b | a ∈ A, b ∈ B } (B ⊂ E) A + x := A + {x} (x ∈ E) λA := {λ}· A (λ ∈ R)
Examples: U, V ⊂ E subspaces. Then U + V = span (U ∪ V)
Bε(x) = x + εB
pos S := R+S (conical hull)
Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals
Convex sets and functions The Euclidean setting and Minkowski notation
In what follows E will be a Euclidean space, i.e. a real-vector space equipped with an inner product h·, ·i : E × E → R of dimension κ < ∞. Examples
E = Rn , hx, yi := xT y, κ = n E = Rm×n , hA, Bi := tr (A T B), κ = mn
Minkowski addition/multiplication: Let A ⊂ E A + B := {a + b | a ∈ A, b ∈ B } (B ⊂ E) A + x := A + {x} (x ∈ E) Λ · A := {λa | a ∈ A, λ ∈ Λ } (Λ ⊂ R) Examples: U, V ⊂ E subspaces. Then U + V = span (U ∪ V)
Bε(x) = x + εB
pos S := R+S (conical hull)
Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals
Convex sets and functions The Euclidean setting and Minkowski notation
In what follows E will be a Euclidean space, i.e. a real-vector space equipped with an inner product h·, ·i : E × E → R of dimension κ < ∞. Examples
E = Rn , hx, yi := xT y, κ = n E = Rm×n , hA, Bi := tr (A T B), κ = mn
Minkowski addition/multiplication: Let A ⊂ E A + B := {a + b | a ∈ A, b ∈ B } (B ⊂ E) A + x := A + {x} (x ∈ E) Λ · A := {λa | a ∈ A, λ ∈ Λ } (Λ ⊂ R) λA := {λ}· A (λ ∈ R) Bε(x) = x + εB
pos S := R+S (conical hull)
Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals
Convex sets and functions The Euclidean setting and Minkowski notation
In what follows E will be a Euclidean space, i.e. a real-vector space equipped with an inner product h·, ·i : E × E → R of dimension κ < ∞. Examples
E = Rn , hx, yi := xT y, κ = n E = Rm×n , hA, Bi := tr (A T B), κ = mn
Minkowski addition/multiplication: Let A ⊂ E A + B := {a + b | a ∈ A, b ∈ B } (B ⊂ E) A + x := A + {x} (x ∈ E) Λ · A := {λa | a ∈ A, λ ∈ Λ } (Λ ⊂ R) λA := {λ}· A (λ ∈ R)
Examples: U, V ⊂ E subspaces. Then U + V = span (U ∪ V) pos S := R+S (conical hull)
Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals
Convex sets and functions The Euclidean setting and Minkowski notation
In what follows E will be a Euclidean space, i.e. a real-vector space equipped with an inner product h·, ·i : E × E → R of dimension κ < ∞. Examples
E = Rn , hx, yi := xT y, κ = n E = Rm×n , hA, Bi := tr (A T B), κ = mn
Minkowski addition/multiplication: Let A ⊂ E A + B := {a + b | a ∈ A, b ∈ B } (B ⊂ E) A + x := A + {x} (x ∈ E) Λ · A := {λa | a ∈ A, λ ∈ Λ } (Λ ⊂ R) λA := {λ}· A (λ ∈ R)
Examples: U, V ⊂ E subspaces. Then U + V = span (U ∪ V)
Bε(x) = x + εB Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals
Convex sets and functions The Euclidean setting and Minkowski notation
In what follows E will be a Euclidean space, i.e. a real-vector space equipped with an inner product h·, ·i : E × E → R of dimension κ < ∞. Examples
E = Rn , hx, yi := xT y, κ = n E = Rm×n , hA, Bi := tr (A T B), κ = mn
Minkowski addition/multiplication: Let A ⊂ E A + B := {a + b | a ∈ A, b ∈ B } (B ⊂ E) A + x := A + {x} (x ∈ E) Λ · A := {λa | a ∈ A, λ ∈ Λ } (Λ ⊂ R) λA := {λ}· A (λ ∈ R)
Examples: U, V ⊂ E subspaces. Then U + V = span (U ∪ V)
Bε(x) = x + εB
pos S := R+S (conical hull) S ⊂ E is said to be convex if λS + (1 − λ)S ⊂ S (λ ∈ (0, 1)); 0 a cone if λS ⊂ S (λ ≥ 0).
Note that K ⊂ E is a convex cone iff K + K ⊂ K. Figure: Convex set/non-convex cone
Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals
Convex sets and functions Convex sets and cones
”The great watershed in optimization is not between linearity and nonlinearity, but convexity and nonconvexity.” (R.T. Rockafellar, *1935) Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals
Convex sets and functions Convex sets and cones
”The great watershed in optimization is not between linearity and nonlinearity, but convexity and nonconvexity.” (R.T. Rockafellar, *1935)
S ⊂ E is said to be convex if λS + (1 − λ)S ⊂ S (λ ∈ (0, 1)); 0 a cone if λS ⊂ S (λ ≥ 0).
Note that K ⊂ E is a convex cone iff K + K ⊂ K. Figure: Convex set/non-convex cone The closed convex hull of S is the smallest closed, convex set containing S, i.e. \ conv S := {C ⊂ E | S ⊂ C, C closed and convex } .
conv S = cl (conv S) n o Pκ+1 ∈ ≥ Pκ+1 conv S = i=1 λi xi xi S, λi 0 (i = 1, . . . , κ + 1), i=1 λi = 1 (Caratheodory’s´ Theorem) conv preserves compactness and boundedness, not necessarily closedness
Example: S := {(0)} ∪ {(a) | a ≥ 0}, 0 1 1 1 1 k − 1 0 ∈ 1/k = k 1 + 1 k 0 conv S. 1 1 But: 1/k → 0 < conv S. 1
Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals
Convex sets and functions The convex hull and the closed convex hull
Definition 1 (Convex hull/closed convex hull). Let S ⊂ E nonempty. Then the convex hull of S is the smallest convex set containing S, i.e. \ conv S := {C ⊂ E | S ⊂ C, C convex } . conv S = cl (conv S) n o Pκ+1 ∈ ≥ Pκ+1 conv S = i=1 λi xi xi S, λi 0 (i = 1, . . . , κ + 1), i=1 λi = 1 (Caratheodory’s´ Theorem) conv preserves compactness and boundedness, not necessarily closedness
Example: S := {(0)} ∪ {(a) | a ≥ 0}, 0 1 1 1 1 k − 1 0 ∈ 1/k = k 1 + 1 k 0 conv S. 1 1 But: 1/k → 0 < conv S. 1
Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals
Convex sets and functions The convex hull and the closed convex hull
Definition 1 (Convex hull/closed convex hull). Let S ⊂ E nonempty. Then the convex hull of S is the smallest convex set containing S, i.e. \ conv S := {C ⊂ E | S ⊂ C, C convex } .
The closed convex hull of S is the smallest closed, convex set containing S, i.e. \ conv S := {C ⊂ E | S ⊂ C, C closed and convex } . n o Pκ+1 ∈ ≥ Pκ+1 conv S = i=1 λi xi xi S, λi 0 (i = 1, . . . , κ + 1), i=1 λi = 1 (Caratheodory’s´ Theorem) conv preserves compactness and boundedness, not necessarily closedness
Example: S := {(0)} ∪ {(a) | a ≥ 0}, 0 1 1 1 1 k − 1 0 ∈ 1/k = k 1 + 1 k 0 conv S. 1 1 But: 1/k → 0 < conv S. 1
Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals
Convex sets and functions The convex hull and the closed convex hull
Definition 1 (Convex hull/closed convex hull). Let S ⊂ E nonempty. Then the convex hull of S is the smallest convex set containing S, i.e. \ conv S := {C ⊂ E | S ⊂ C, C convex } .
The closed convex hull of S is the smallest closed, convex set containing S, i.e. \ conv S := {C ⊂ E | S ⊂ C, C closed and convex } .
conv S = cl (conv S) conv preserves compactness and boundedness, not necessarily closedness
Example: S := {(0)} ∪ {(a) | a ≥ 0}, 0 1 1 1 1 k − 1 0 ∈ 1/k = k 1 + 1 k 0 conv S. 1 1 But: 1/k → 0 < conv S. 1
Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals
Convex sets and functions The convex hull and the closed convex hull
Definition 1 (Convex hull/closed convex hull). Let S ⊂ E nonempty. Then the convex hull of S is the smallest convex set containing S, i.e. \ conv S := {C ⊂ E | S ⊂ C, C convex } .
The closed convex hull of S is the smallest closed, convex set containing S, i.e. \ conv S := {C ⊂ E | S ⊂ C, C closed and convex } .
conv S = cl (conv S) n o Pκ+1 ∈ ≥ Pκ+1 conv S = i=1 λi xi xi S, λi 0 (i = 1, . . . , κ + 1), i=1 λi = 1 (Caratheodory’s´ Theorem) Example: S := {(0)} ∪ {(a) | a ≥ 0}, 0 1 1 1 1 k − 1 0 ∈ 1/k = k 1 + 1 k 0 conv S. 1 1 But: 1/k → 0 < conv S. 1
Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals
Convex sets and functions The convex hull and the closed convex hull
Definition 1 (Convex hull/closed convex hull). Let S ⊂ E nonempty. Then the convex hull of S is the smallest convex set containing S, i.e. \ conv S := {C ⊂ E | S ⊂ C, C convex } .
The closed convex hull of S is the smallest closed, convex set containing S, i.e. \ conv S := {C ⊂ E | S ⊂ C, C closed and convex } .
conv S = cl (conv S) n o Pκ+1 ∈ ≥ Pκ+1 conv S = i=1 λi xi xi S, λi 0 (i = 1, . . . , κ + 1), i=1 λi = 1 (Caratheodory’s´ Theorem) conv preserves compactness and boundedness, not necessarily closedness Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals
Convex sets and functions The convex hull and the closed convex hull
Definition 1 (Convex hull/closed convex hull). Let S ⊂ E nonempty. Then the convex hull of S is the smallest convex set containing S, i.e. \ conv S := {C ⊂ E | S ⊂ C, C convex } .
The closed convex hull of S is the smallest closed, convex set containing S, i.e. \ conv S := {C ⊂ E | S ⊂ C, C closed and convex } .
conv S = cl (conv S) n o Pκ+1 ∈ ≥ Pκ+1 conv S = i=1 λi xi xi S, λi 0 (i = 1, . . . , κ + 1), i=1 λi = 1 (Caratheodory’s´ Theorem) conv preserves compactness and boundedness, not necessarily closedness
Example: S := {(0)} ∪ {(a) | a ≥ 0}, 0 1 1 1 1 k − 1 0 ∈ 1/k = k 1 + 1 k 0 conv S. 1 1 But: 1/k → 0 < conv S. 1 Affine hull: aff M := T {S ∈ E | M ⊂ S, S affine } .
Relative interior/boundary: C ⊂ E convex. n o ri C := x ∈ C ∃ε > 0 : Bε(x) ∩ aff C ⊂ C (relative interior) rbd C := cl C \ ri C (relative boundary) x ∈ ri C ⇔ span C = R+(C − x)
ri C C aff C ri C aff C {x} {x} {x} [x, x0] {λx + (1 − λ)x0 | λ ∈ R} (x, x0) Bε(x) E Bε(x) C Table: Examples for relative interiors
Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals
Convex sets and functions The topology relative to the affine hull
Affine set: A set S = U + x with x ∈ E and a subspace U ⊂ is called affine. This is characterized by αS + (1 − α)S ⊂ S (α ∈ R). Relative interior/boundary: C ⊂ E convex. n o ri C := x ∈ C ∃ε > 0 : Bε(x) ∩ aff C ⊂ C (relative interior) rbd C := cl C \ ri C (relative boundary) x ∈ ri C ⇔ span C = R+(C − x)
ri C C aff C ri C aff C {x} {x} {x} [x, x0] {λx + (1 − λ)x0 | λ ∈ R} (x, x0) Bε(x) E Bε(x) C Table: Examples for relative interiors
Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals
Convex sets and functions The topology relative to the affine hull
Affine set: A set S = U + x with x ∈ E and a subspace U ⊂ is called affine. This is characterized by αS + (1 − α)S ⊂ S (α ∈ R). Affine hull: aff M := T {S ∈ E | M ⊂ S, S affine } . rbd C := cl C \ ri C (relative boundary) x ∈ ri C ⇔ span C = R+(C − x)
ri C C aff C ri C aff C {x} {x} {x} [x, x0] {λx + (1 − λ)x0 | λ ∈ R} (x, x0) Bε(x) E Bε(x) C Table: Examples for relative interiors
Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals
Convex sets and functions The topology relative to the affine hull
Affine set: A set S = U + x with x ∈ E and a subspace U ⊂ is called affine. This is characterized by αS + (1 − α)S ⊂ S (α ∈ R). Affine hull: aff M := T {S ∈ E | M ⊂ S, S affine } .
Relative interior/boundary: C ⊂ E convex. n o ri C := x ∈ C ∃ε > 0 : Bε(x) ∩ aff C ⊂ C (relative interior) ri C C aff C ri C aff C {x} {x} {x} [x, x0] {λx + (1 − λ)x0 | λ ∈ R} (x, x0) Bε(x) E Bε(x) C Table: Examples for relative interiors
Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals
Convex sets and functions The topology relative to the affine hull
Affine set: A set S = U + x with x ∈ E and a subspace U ⊂ is called affine. This is characterized by αS + (1 − α)S ⊂ S (α ∈ R). Affine hull: aff M := T {S ∈ E | M ⊂ S, S affine } .
Relative interior/boundary: C ⊂ E convex. n o ri C := x ∈ C ∃ε > 0 : Bε(x) ∩ aff C ⊂ C (relative interior) rbd C := cl C \ ri C (relative boundary) x ∈ ri C ⇔ span C = R+(C − x) Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals
Convex sets and functions The topology relative to the affine hull
Affine set: A set S = U + x with x ∈ E and a subspace U ⊂ is called affine. This is characterized by αS + (1 − α)S ⊂ S (α ∈ R). Affine hull: aff M := T {S ∈ E | M ⊂ S, S affine } .
Relative interior/boundary: C ⊂ E convex. n o ri C := x ∈ C ∃ε > 0 : Bε(x) ∩ aff C ⊂ C (relative interior) rbd C := cl C \ ri C (relative boundary) x ∈ ri C ⇔ span C = R+(C − x)
ri C C aff C ri C aff C {x} {x} {x} [x, x0] {λx + (1 − λ)x0 | λ ∈ R} (x, x0) Bε(x) E Bε(x) C Table: Examples for relative interiors C∞
C
Figure: The horizon cone of an unbounded, nonconvex set
Proposition 3 (The convex case). Let C ⊂ E be nonempty and convex. Then C∞ = {v | ∀x ∈ cl C, λ ≥ 0 : x + λv ∈ cl C } . In particular, C∞ is (a closed and) convex (cone) if C is convex.
Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals
Convex sets and functions The horizon cone Definition 2 (Horizon cone). For a nonempty set S ⊂ E the set ∞ S := {v ∈ E | ∃{xk ∈ S}, {tk } ↓ 0 : tk xk → v } is called the horizon cone of S. We put ∅∞ := {0}. Proposition 3 (The convex case). Let C ⊂ E be nonempty and convex. Then C∞ = {v | ∀x ∈ cl C, λ ≥ 0 : x + λv ∈ cl C } . In particular, C∞ is (a closed and) convex (cone) if C is convex.
Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals
Convex sets and functions The horizon cone Definition 2 (Horizon cone). For a nonempty set S ⊂ E the set ∞ S := {v ∈ E | ∃{xk ∈ S}, {tk } ↓ 0 : tk xk → v } is called the horizon cone of S. We put ∅∞ := {0}.
C∞
C
Figure: The horizon cone of an unbounded, nonconvex set Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals
Convex sets and functions The horizon cone Definition 2 (Horizon cone). For a nonempty set S ⊂ E the set ∞ S := {v ∈ E | ∃{xk ∈ S}, {tk } ↓ 0 : tk xk → v } is called the horizon cone of S. We put ∅∞ := {0}.
C∞
C
Figure: The horizon cone of an unbounded, nonconvex set
Proposition 3 (The convex case). Let C ⊂ E be nonempty and convex. Then C∞ = {v | ∀x ∈ cl C, λ ≥ 0 : x + λv ∈ cl C } . In particular, C∞ is (a closed and) convex (cone) if C is convex. f(x) epi f n o epi → f is uniquely determined through epi f! Figure: Epigraph of f : R → R f proper :⇔ −∞ < f . +∞ ⇔ 1 dom f , ∅ 1 f convex :⇔ epi f/epi f pos. hom. :⇔ 1 epi f cone ⇔ αf(x) = f(αx)(x ∈ E, α ≥ 0) f sublinear :⇔ 1 epi f cvx. cone ⇔ f(λx + µy) ≤ λf(x) + µf(y)(x, y ∈ E, λ, µ ≥ 0). ⇔ f convex + positively homogeneous Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Convex sets and functions Extended real-valued functions: An epigraphical perspective Let f : E → R := R ∪ {±∞}. n o epi f := (x, α) ∈ E × R f(x) ≤ α (epigraph) 1Only for f : E → R ∪ {+∞} f(x) epi f gph f n o dom f := x ∈ E f(x) < ∞ (domain). x → f is uniquely determined through epi f! Figure: Epigraph of f : R → R f proper :⇔ −∞ < f . +∞ ⇔ 1 dom f , ∅ 1 f convex :⇔ epi f/epi f pos. hom. :⇔ 1 epi f cone ⇔ αf(x) = f(αx)(x ∈ E, α ≥ 0) f sublinear :⇔ 1 epi f cvx. cone ⇔ f(λx + µy) ≤ λf(x) + µf(y)(x, y ∈ E, λ, µ ≥ 0). ⇔ f convex + positively homogeneous Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Convex sets and functions Extended real-valued functions: An epigraphical perspective Let f : E → R := R ∪ {±∞}. n o epi f := (x, α) ∈ E × R f(x) ≤ α (epigraph) n o epi 1Only for f : E → R ∪ {+∞} f proper :⇔ −∞ < f . +∞ ⇔ 1 dom f , ∅ 1 f convex :⇔ epi f/epi f pos. hom. :⇔ 1 epi f cone ⇔ αf(x) = f(αx)(x ∈ E, α ≥ 0) f sublinear :⇔ 1 epi f cvx. cone ⇔ f(λx + µy) ≤ λf(x) + µf(y)(x, y ∈ E, λ, µ ≥ 0). ⇔ f convex + positively homogeneous Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Convex sets and functions Extended real-valued functions: An epigraphical perspective : E → R := R ∪ {±∞} Let f . f(x) n o epi f := (x, α) ∈ E × R f(x) ≤ α (epigraph) epi f n o epi → f is uniquely determined through epi f! Figure: Epigraph of f : R → R 1Only for f : E → R ∪ {+∞} 1 f convex :⇔ epi f/epi f pos. hom. :⇔ 1 epi f cone ⇔ αf(x) = f(αx)(x ∈ E, α ≥ 0) f sublinear :⇔ 1 epi f cvx. cone ⇔ f(λx + µy) ≤ λf(x) + µf(y)(x, y ∈ E, λ, µ ≥ 0). ⇔ f convex + positively homogeneous Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Convex sets and functions Extended real-valued functions: An epigraphical perspective : E → R := R ∪ {±∞} Let f . f(x) n o epi f := (x, α) ∈ E × R f(x) ≤ α (epigraph) epi f n o epi → f is uniquely determined through epi f! Figure: Epigraph of f : R → R f proper :⇔ −∞ < f . +∞ ⇔ 1 dom f , ∅ 1Only for f : E → R ∪ {+∞} f pos. hom. :⇔ 1 epi f cone ⇔ αf(x) = f(αx)(x ∈ E, α ≥ 0) f sublinear :⇔ 1 epi f cvx. cone ⇔ f(λx + µy) ≤ λf(x) + µf(y)(x, y ∈ E, λ, µ ≥ 0). ⇔ f convex + positively homogeneous Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Convex sets and functions Extended real-valued functions: An epigraphical perspective : E → R := R ∪ {±∞} Let f . f(x) n o epi f := (x, α) ∈ E × R f(x) ≤ α (epigraph) epi f n o epi → f is uniquely determined through epi f! Figure: Epigraph of f : R → R f proper :⇔ −∞ < f . +∞ ⇔ 1 dom f , ∅ 1 f convex :⇔ epi f/epi 1Only for f : E → R ∪ {+∞} f sublinear :⇔ 1 epi f cvx. cone ⇔ f(λx + µy) ≤ λf(x) + µf(y)(x, y ∈ E, λ, µ ≥ 0). ⇔ f convex + positively homogeneous Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Convex sets and functions Extended real-valued functions: An epigraphical perspective : E → R := R ∪ {±∞} Let f . f(x) n o epi f := (x, α) ∈ E × R f(x) ≤ α (epigraph) epi f n o epi → f is uniquely determined through epi f! Figure: Epigraph of f : R → R f proper :⇔ −∞ < f . +∞ ⇔ 1 dom f , ∅ 1 f convex :⇔ epi f/epi f pos. hom. :⇔ 1 epi f cone ⇔ αf(x) = f(αx)(x ∈ E, α ≥ 0) 1Only for f : E → R ∪ {+∞} ⇔ f convex + positively homogeneous Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Convex sets and functions Extended real-valued functions: An epigraphical perspective : E → R := R ∪ {±∞} Let f . f(x) n o epi f := (x, α) ∈ E × R f(x) ≤ α (epigraph) epi f n o epi → f is uniquely determined through epi f! Figure: Epigraph of f : R → R f proper :⇔ −∞ < f . +∞ ⇔ 1 dom f , ∅ 1 f convex :⇔ epi f/epi f pos. hom. :⇔ 1 epi f cone ⇔ αf(x) = f(αx)(x ∈ E, α ≥ 0) f sublinear :⇔ 1 epi f cvx. cone ⇔ f(λx + µy) ≤ λf(x) + µf(y)(x, y ∈ E, λ, µ ≥ 0). 1Only for f : E → R ∪ {+∞} Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Convex sets and functions Extended real-valued functions: An epigraphical perspective : E → R := R ∪ {±∞} Let f . f(x) n o epi f := (x, α) ∈ E × R f(x) ≤ α (epigraph) epi f n o epi → f is uniquely determined through epi f! Figure: Epigraph of f : R → R f proper :⇔ −∞ < f . +∞ ⇔ 1 dom f , ∅ 1 f convex :⇔ epi f/epi f pos. hom. :⇔ 1 epi f cone ⇔ αf(x) = f(αx)(x ∈ E, α ≥ 0) f sublinear :⇔ 1 epi f cvx. cone ⇔ f(λx + µy) ≤ λf(x) + µf(y)(x, y ∈ E, λ, µ ≥ 0). ⇔ f convex + positively homogeneous 1Only for f : E → R ∪ {+∞} f(x) Lower semicontinuity: f is said to be lsc (or closed) at x¯ if lim inf f(x) ≥ f(x¯). x→x¯ x x¯ ¯ Closure: cl f : E → R, (cl f)(x) := lim infx→x¯ f(x). Figure: f not lsc at x¯ f(x) Facts: f lsc ⇐⇒ epi f closed ⇐⇒ f = cl f ⇐⇒ lev f r epi f closed (r ∈ R) cl f ≤ f x f proper, lsc and coercive (i.e. limkxk→∞ f(x) = ∞) then: ( argmin f , ∅ and inf f ∈ R 1 x > 0, E Figure: f : x 7→ x E +∞, else. Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Convex sets and functions Lower semicontinuity Let f : E → R and x¯ ∈ E. Lower limit: n o lim infx→x¯ f(x) := inf α ∃ xk → x¯ : f(xk ) → α f(x) x x¯ ¯ Closure: cl f : E → R, (cl f)(x) := lim infx→x¯ f(x). Figure: f not lsc at x¯ f(x) Facts: f lsc ⇐⇒ epi f closed ⇐⇒ f = cl f ⇐⇒ lev f r epi f closed (r ∈ R) cl f ≤ f x f proper, lsc and coercive (i.e. limkxk→∞ f(x) = ∞) then: ( argmin f , ∅ and inf f ∈ R 1 x > 0, E Figure: f : x 7→ x E +∞, else. Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Convex sets and functions Lower semicontinuity Let f : E → R and x¯ ∈ E. Lower limit: n o lim infx→x¯ f(x) := inf α ∃ xk → x¯ : f(xk ) → α Lower semicontinuity: f is said to be lsc (or closed) at x¯ if lim inf f(x) ≥ f(x¯). x→x¯ f(x) Facts: f lsc ⇐⇒ epi f closed ⇐⇒ f = cl f ⇐⇒ lev f r epi f closed (r ∈ R) cl f ≤ f x f proper, lsc and coercive (i.e. limkxk→∞ f(x) = ∞) then: ( argmin f , ∅ and inf f ∈ R 1 x > 0, E Figure: f : x 7→ x E +∞, else. Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Convex sets and functions Lower semicontinuity Let f : E → R and x¯ ∈ E. f(x) Lower limit: n o lim infx→x¯ f(x) := inf α ∃ xk → x¯ : f(xk ) → α Lower semicontinuity: f is said to be lsc (or closed) at x¯ if lim inf f(x) ≥ f(x¯). x→x¯ x x¯ ¯ Closure: cl f : E → R, (cl f)(x) := lim infx→x¯ f(x). Figure: f not lsc at x¯ f(x) epi f cl f ≤ f x f proper, lsc and coercive (i.e. limkxk→∞ f(x) = ∞) then: ( argmin f , ∅ and inf f ∈ R 1 x > 0, E Figure: f : x 7→ x E +∞, else. Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Convex sets and functions Lower semicontinuity Let f : E → R and x¯ ∈ E. f(x) Lower limit: n o lim infx→x¯ f(x) := inf α ∃ xk → x¯ : f(xk ) → α Lower semicontinuity: f is said to be lsc (or closed) at x¯ if lim inf f(x) ≥ f(x¯). x→x¯ x x¯ ¯ Closure: cl f : E → R, (cl f)(x) := lim infx→x¯ f(x). Figure: f not lsc at x¯ Facts: f lsc ⇐⇒ epi f closed ⇐⇒ f = cl f ⇐⇒ levr f closed (r ∈ R) f(x) epi f x f proper, lsc and coercive (i.e. limkxk→∞ f(x) = ∞) then: ( argmin f , ∅ and inf f ∈ R 1 x > 0, E Figure: f : x 7→ x E +∞, else. Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Convex sets and functions Lower semicontinuity Let f : E → R and x¯ ∈ E. f(x) Lower limit: n o lim infx→x¯ f(x) := inf α ∃ xk → x¯ : f(xk ) → α Lower semicontinuity: f is said to be lsc (or closed) at x¯ if lim inf f(x) ≥ f(x¯). x→x¯ x x¯ ¯ Closure: cl f : E → R, (cl f)(x) := lim infx→x¯ f(x). Figure: f not lsc at x¯ Facts: f lsc ⇐⇒ epi f closed ⇐⇒ f = cl f ⇐⇒ levr f closed (r ∈ R) cl f ≤ f f(x) epi f x ( 1 x > 0, Figure: f : x 7→ x +∞, else. Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Convex sets and functions Lower semicontinuity Let f : E → R and x¯ ∈ E. f(x) Lower limit: n o lim infx→x¯ f(x) := inf α ∃ xk → x¯ : f(xk ) → α Lower semicontinuity: f is said to be lsc (or closed) at x¯ if lim inf f(x) ≥ f(x¯). x→x¯ x x¯ ¯ Closure: cl f : E → R, (cl f)(x) := lim infx→x¯ f(x). Figure: f not lsc at x¯ Facts: f lsc ⇐⇒ epi f closed ⇐⇒ f = cl f ⇐⇒ levr f closed (r ∈ R) cl f ≤ f f proper, lsc and coercive (i.e. limkxk→∞ f(x) = ∞) then: argmin f , ∅ and inf f ∈ R E E Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Convex sets and functions Lower semicontinuity Let f : E → R and x¯ ∈ E. f(x) Lower limit: n o lim infx→x¯ f(x) := inf α ∃ xk → x¯ : f(xk ) → α Lower semicontinuity: f is said to be lsc (or closed) at x¯ if lim inf f(x) ≥ f(x¯). x→x¯ x x¯ ¯ Closure: cl f : E → R, (cl f)(x) := lim infx→x¯ f(x). Figure: f not lsc at x¯ f(x) Facts: f lsc ⇐⇒ epi f closed ⇐⇒ f = cl f ⇐⇒ lev f r epi f closed (r ∈ R) cl f ≤ f x f proper, lsc and coercive (i.e. limkxk→∞ f(x) = ∞) then: ( argmin f , ∅ and inf f ∈ R 1 x > 0, E Figure: f : x 7→ x E +∞, else. ◦ F(C) (affine image) ◦ F−1(D) (affine pre-image) ◦ C × D (Cartesian product) ◦ C + C (Minkowski sum) T1 2 ◦ i∈I Ci (Intersection) 2 Functional operations 0 For fi , g : E → R convex and F : E → E affine the following functions are convex: (Affine pre-composition) f := g ◦ F: epi f = T −1(epi g), T :(x, α) 7→ (T(x), α) T (Pointwise supremum) f := supi∈I fi : epi f = i∈I epi fi n o 7→ 1 k − k2 1 k · k2 (Moreau envelope) f := eλg : x infu g(u) + 2λ x u : epi f = epi g + epi 2 . Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Convex sets and functions Convexity preserving operations - new from old 1 Set Operations 0 0 For C, Ci (i ∈ I) ⊂ E, D ⊂ E convex, F : E → E affine the following sets are convex: ◦ F−1(D) (affine pre-image) ◦ C × D (Cartesian product) ◦ C + C (Minkowski sum) T1 2 ◦ i∈I Ci (Intersection) 2 Functional operations 0 For fi , g : E → R convex and F : E → E affine the following functions are convex: (Affine pre-composition) f := g ◦ F: epi f = T −1(epi g), T :(x, α) 7→ (T(x), α) T (Pointwise supremum) f := supi∈I fi : epi f = i∈I epi fi n o 7→ 1 k − k2 1 k · k2 (Moreau envelope) f := eλg : x infu g(u) + 2λ x u : epi f = epi g + epi 2 . Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Convex sets and functions Convexity preserving operations - new from old 1 Set Operations 0 0 For C, Ci (i ∈ I) ⊂ E, D ⊂ E convex, F : E → E affine the following sets are convex: ◦ F(C) (affine image) ◦ C × D (Cartesian product) ◦ C + C (Minkowski sum) T1 2 ◦ i∈I Ci (Intersection) 2 Functional operations 0 For fi , g : E → R convex and F : E → E affine the following functions are convex: (Affine pre-composition) f := g ◦ F: epi f = T −1(epi g), T :(x, α) 7→ (T(x), α) T (Pointwise supremum) f := supi∈I fi : epi f = i∈I epi fi n o 7→ 1 k − k2 1 k · k2 (Moreau envelope) f := eλg : x infu g(u) + 2λ x u : epi f = epi g + epi 2 . Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Convex sets and functions Convexity preserving operations - new from old 1 Set Operations 0 0 For C, Ci (i ∈ I) ⊂ E, D ⊂ E convex, F : E → E affine the following sets are convex: ◦ F(C) (affine image) ◦ F−1(D) (affine pre-image) ◦ C + C (Minkowski sum) T1 2 ◦ i∈I Ci (Intersection) 2 Functional operations 0 For fi , g : E → R convex and F : E → E affine the following functions are convex: (Affine pre-composition) f := g ◦ F: epi f = T −1(epi g), T :(x, α) 7→ (T(x), α) T (Pointwise supremum) f := supi∈I fi : epi f = i∈I epi fi n o 7→ 1 k − k2 1 k · k2 (Moreau envelope) f := eλg : x infu g(u) + 2λ x u : epi f = epi g + epi 2 . Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Convex sets and functions Convexity preserving operations - new from old 1 Set Operations 0 0 For C, Ci (i ∈ I) ⊂ E, D ⊂ E convex, F : E → E affine the following sets are convex: ◦ F(C) (affine image) ◦ F−1(D) (affine pre-image) ◦ C × D (Cartesian product) T ◦ i∈I Ci (Intersection) 2 Functional operations 0 For fi , g : E → R convex and F : E → E affine the following functions are convex: (Affine pre-composition) f := g ◦ F: epi f = T −1(epi g), T :(x, α) 7→ (T(x), α) T (Pointwise supremum) f := supi∈I fi : epi f = i∈I epi fi n o 7→ 1 k − k2 1 k · k2 (Moreau envelope) f := eλg : x infu g(u) + 2λ x u : epi f = epi g + epi 2 . Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Convex sets and functions Convexity preserving operations - new from old 1 Set Operations 0 0 For C, Ci (i ∈ I) ⊂ E, D ⊂ E convex, F : E → E affine the following sets are convex: ◦ F(C) (affine image) ◦ F−1(D) (affine pre-image) ◦ C × D (Cartesian product) ◦ C1 + C2 (Minkowski sum) 2 Functional operations 0 For fi , g : E → R convex and F : E → E affine the following functions are convex: (Affine pre-composition) f := g ◦ F: epi f = T −1(epi g), T :(x, α) 7→ (T(x), α) T (Pointwise supremum) f := supi∈I fi : epi f = i∈I epi fi n o 7→ 1 k − k2 1 k · k2 (Moreau envelope) f := eλg : x infu g(u) + 2λ x u : epi f = epi g + epi 2 . Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Convex sets and functions Convexity preserving operations - new from old 1 Set Operations 0 0 For C, Ci (i ∈ I) ⊂ E, D ⊂ E convex, F : E → E affine the following sets are convex: ◦ F(C) (affine image) ◦ F−1(D) (affine pre-image) ◦ C × D (Cartesian product) ◦ C + C (Minkowski sum) T1 2 ◦ i∈I Ci (Intersection) (Affine pre-composition) f := g ◦ F: epi f = T −1(epi g), T :(x, α) 7→ (T(x), α) T (Pointwise supremum) f := supi∈I fi : epi f = i∈I epi fi n o 7→ 1 k − k2 1 k · k2 (Moreau envelope) f := eλg : x infu g(u) + 2λ x u : epi f = epi g + epi 2 . Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Convex sets and functions Convexity preserving operations - new from old 1 Set Operations 0 0 For C, Ci (i ∈ I) ⊂ E, D ⊂ E convex, F : E → E affine the following sets are convex: ◦ F(C) (affine image) ◦ F−1(D) (affine pre-image) ◦ C × D (Cartesian product) ◦ C + C (Minkowski sum) T1 2 ◦ i∈I Ci (Intersection) 2 Functional operations 0 For fi , g : E → R convex and F : E → E affine the following functions are convex: T (Pointwise supremum) f := supi∈I fi : epi f = i∈I epi fi n o 7→ 1 k − k2 1 k · k2 (Moreau envelope) f := eλg : x infu g(u) + 2λ x u : epi f = epi g + epi 2 . Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Convex sets and functions Convexity preserving operations - new from old 1 Set Operations 0 0 For C, Ci (i ∈ I) ⊂ E, D ⊂ E convex, F : E → E affine the following sets are convex: ◦ F(C) (affine image) ◦ F−1(D) (affine pre-image) ◦ C × D (Cartesian product) ◦ C + C (Minkowski sum) T1 2 ◦ i∈I Ci (Intersection) 2 Functional operations 0 For fi , g : E → R convex and F : E → E affine the following functions are convex: (Affine pre-composition) f := g ◦ F: epi f = T −1(epi g), T :(x, α) 7→ (T(x), α) n o 7→ 1 k − k2 1 k · k2 (Moreau envelope) f := eλg : x infu g(u) + 2λ x u : epi f = epi g + epi 2 . Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Convex sets and functions Convexity preserving operations - new from old 1 Set Operations 0 0 For C, Ci (i ∈ I) ⊂ E, D ⊂ E convex, F : E → E affine the following sets are convex: ◦ F(C) (affine image) ◦ F−1(D) (affine pre-image) ◦ C × D (Cartesian product) ◦ C + C (Minkowski sum) T1 2 ◦ i∈I Ci (Intersection) 2 Functional operations 0 For fi , g : E → R convex and F : E → E affine the following functions are convex: (Affine pre-composition) f := g ◦ F: epi f = T −1(epi g), T :(x, α) 7→ (T(x), α) T (Pointwise supremum) f := supi∈I fi : epi f = i∈I epi fi Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Convex sets and functions Convexity preserving operations - new from old 1 Set Operations 0 0 For C, Ci (i ∈ I) ⊂ E, D ⊂ E convex, F : E → E affine the following sets are convex: ◦ F(C) (affine image) ◦ F−1(D) (affine pre-image) ◦ C × D (Cartesian product) ◦ C + C (Minkowski sum) T1 2 ◦ i∈I Ci (Intersection) 2 Functional operations 0 For fi , g : E → R convex and F : E → E affine the following functions are convex: (Affine pre-composition) f := g ◦ F: epi f = T −1(epi g), T :(x, α) 7→ (T(x), α) T (Pointwise supremum) f := supi∈I fi : epi f = i∈I epi fi n o 7→ 1 k − k2 1 k · k2 (Moreau envelope) f := eλg : x infu g(u) + 2λ x u : epi f = epi g + epi 2 . Slogan: ”The subgradients of f at x¯ are the slopes of affine minorants of f that coincide with f at x”.¯ The subdifferential operator is a set-valued mapping ∂f : E ⇒ E. Set n o dom ∂f := x ∈ E ∂f(x) , ∅ . 0 ∈ ∂f(x) ⇐⇒ x ∈ argminE f (Fermat’s rule) ∂f(x) closed and convex (x ∈ E) ∂f(x) is a singleton ⇐⇒ f differentiable at x ⇐⇒ f continuously differentiable at x ri (dom f) ⊂ dom ∂f ⊂ dom f (f convex). Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Subdifferentiation and conjugacy of convex functions The convex subdifferential Definition 4. Let f : E → R. A vector v ∈ E is called a subgradient of v at x¯ if f(x) ≥ f(x¯) + hv, x − x¯i (x ∈ E). (1) We denote by ∂f(x¯) the set of all subgradients of f at x¯ and call it the (convex) subdifferential of f at x¯. The inequality (1) is referred to as subgradient inequality. The subdifferential operator is a set-valued mapping ∂f : E ⇒ E. Set n o dom ∂f := x ∈ E ∂f(x) , ∅ . 0 ∈ ∂f(x) ⇐⇒ x ∈ argminE f (Fermat’s rule) ∂f(x) closed and convex (x ∈ E) ∂f(x) is a singleton ⇐⇒ f differentiable at x ⇐⇒ f continuously differentiable at x ri (dom f) ⊂ dom ∂f ⊂ dom f (f convex). Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Subdifferentiation and conjugacy of convex functions The convex subdifferential Definition 4. Let f : E → R. A vector v ∈ E is called a subgradient of v at x¯ if f(x) ≥ f(x¯) + hv, x − x¯i (x ∈ E). (1) We denote by ∂f(x¯) the set of all subgradients of f at x¯ and call it the (convex) subdifferential of f at x¯. The inequality (1) is referred to as subgradient inequality. Slogan: ”The subgradients of f at x¯ are the slopes of affine minorants of f that coincide with f at x”.¯ ∂f(x) closed and convex (x ∈ E) ∂f(x) is a singleton ⇐⇒ f differentiable at x ⇐⇒ f continuously differentiable at x ri (dom f) ⊂ dom ∂f ⊂ dom f (f convex). Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Subdifferentiation and conjugacy of convex functions The convex subdifferential Definition 4. Let f : E → R. A vector v ∈ E is called a subgradient of v at x¯ if f(x) ≥ f(x¯) + hv, x − x¯i (x ∈ E). (1) We denote by ∂f(x¯) the set of all subgradients of f at x¯ and call it the (convex) subdifferential of f at x¯. The inequality (1) is referred to as subgradient inequality. Slogan: ”The subgradients of f at x¯ are the slopes of affine minorants of f that coincide with f at x”.¯ The subdifferential operator is a set-valued mapping ∂f : E ⇒ E. Set n o dom ∂f := x ∈ E ∂f(x) , ∅ . 0 ∈ ∂f(x) ⇐⇒ x ∈ argminE f (Fermat’s rule) ∂f(x) is a singleton ⇐⇒ f differentiable at x ⇐⇒ f continuously differentiable at x ri (dom f) ⊂ dom ∂f ⊂ dom f (f convex). Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Subdifferentiation and conjugacy of convex functions The convex subdifferential Definition 4. Let f : E → R. A vector v ∈ E is called a subgradient of v at x¯ if f(x) ≥ f(x¯) + hv, x − x¯i (x ∈ E). (1) We denote by ∂f(x¯) the set of all subgradients of f at x¯ and call it the (convex) subdifferential of f at x¯. The inequality (1) is referred to as subgradient inequality. Slogan: ”The subgradients of f at x¯ are the slopes of affine minorants of f that coincide with f at x”.¯ The subdifferential operator is a set-valued mapping ∂f : E ⇒ E. Set n o dom ∂f := x ∈ E ∂f(x) , ∅ . 0 ∈ ∂f(x) ⇐⇒ x ∈ argminE f (Fermat’s rule) ∂f(x) closed and convex (x ∈ E) ri (dom f) ⊂ dom ∂f ⊂ dom f (f convex). Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Subdifferentiation and conjugacy of convex functions The convex subdifferential Definition 4. Let f : E → R. A vector v ∈ E is called a subgradient of v at x¯ if f(x) ≥ f(x¯) + hv, x − x¯i (x ∈ E). (1) We denote by ∂f(x¯) the set of all subgradients of f at x¯ and call it the (convex) subdifferential of f at x¯. The inequality (1) is referred to as subgradient inequality. Slogan: ”The subgradients of f at x¯ are the slopes of affine minorants of f that coincide with f at x”.¯ The subdifferential operator is a set-valued mapping ∂f : E ⇒ E. Set n o dom ∂f := x ∈ E ∂f(x) , ∅ . 0 ∈ ∂f(x) ⇐⇒ x ∈ argminE f (Fermat’s rule) ∂f(x) closed and convex (x ∈ E) ∂f(x) is a singleton ⇐⇒ f differentiable at x ⇐⇒ f continuously differentiable at x Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Subdifferentiation and conjugacy of convex functions The convex subdifferential Definition 4. Let f : E → R. A vector v ∈ E is called a subgradient of v at x¯ if f(x) ≥ f(x¯) + hv, x − x¯i (x ∈ E). (1) We denote by ∂f(x¯) the set of all subgradients of f at x¯ and call it the (convex) subdifferential of f at x¯. The inequality (1) is referred to as subgradient inequality. Slogan: ”The subgradients of f at x¯ are the slopes of affine minorants of f that coincide with f at x”.¯ The subdifferential operator is a set-valued mapping ∂f : E ⇒ E. Set n o dom ∂f := x ∈ E ∂f(x) , ∅ . 0 ∈ ∂f(x) ⇐⇒ x ∈ argminE f (Fermat’s rule) ∂f(x) closed and convex (x ∈ E) ∂f(x) is a singleton ⇐⇒ f differentiable at x ⇐⇒ f continuously differentiable at x ri (dom f) ⊂ dom ∂f ⊂ dom f (f convex). S n o NS (0) ∂δS (x¯) = v δC (x) ≥ δC (x¯) + hv, x − x¯i (x ∈ E) n o = v ∈ E hv, x − x¯i ≤ 0 (x ∈ S) =: NS (x¯)(x¯ ∈ S) Figure: Normal cone √ (Euclidean norm) k · k := h·, ·i. Then n o x¯ if x¯ , 0, ∂k · k(x¯) = kx¯k B if x¯ = 0. (Empty subdifferential) ( √ − x if x ≥ 0, f : x ∈ R 7→ +∞ else. 1 − √ , x > 0, ∂f(x) = 2 x ∅, else. Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Subdifferentiation and conjugacy of convex functions Examples of subdifferentiation (Indicator function/Normal cone) Let S ⊂ E. Indicator function of S: ( 0, x ∈ S, δ : E → R ∪ {+∞}, δ (x) := S S +∞, else. S NS (0) n o = v ∈ E hv, x − x¯i ≤ 0 (x ∈ S) =: NS (x¯)(x¯ ∈ S) Figure: Normal cone √ (Euclidean norm) k · k := h·, ·i. Then n o x¯ if x¯ , 0, ∂k · k(x¯) = kx¯k B if x¯ = 0. (Empty subdifferential) ( √ − x if x ≥ 0, f : x ∈ R 7→ +∞ else. 1 − √ , x > 0, ∂f(x) = 2 x ∅, else. Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Subdifferentiation and conjugacy of convex functions Examples of subdifferentiation (Indicator function/Normal cone) Let S ⊂ E. Indicator function of S: ( 0, x ∈ S, δ : E → R ∪ {+∞}, δ (x) := S S +∞, else. n o ∂δS (x¯) = v δC (x) ≥ δC (x¯) + hv, x − x¯i (x ∈ E) S NS (0) =: NS (x¯)(x¯ ∈ S) Figure: Normal cone √ (Euclidean norm) k · k := h·, ·i. Then n o x¯ if x¯ , 0, ∂k · k(x¯) = kx¯k B if x¯ = 0. (Empty subdifferential) ( √ − x if x ≥ 0, f : x ∈ R 7→ +∞ else. 1 − √ , x > 0, ∂f(x) = 2 x ∅, else. Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Subdifferentiation and conjugacy of convex functions Examples of subdifferentiation (Indicator function/Normal cone) Let S ⊂ E. Indicator function of S: ( 0, x ∈ S, δ : E → R ∪ {+∞}, δ (x) := S S +∞, else. n o ∂δS (x¯) = v δC (x) ≥ δC (x¯) + hv, x − x¯i (x ∈ E) n o = v ∈ E hv, x − x¯i ≤ 0 (x ∈ S) √ (Euclidean norm) k · k := h·, ·i. Then n o x¯ if x¯ , 0, ∂k · k(x¯) = kx¯k B if x¯ = 0. (Empty subdifferential) ( √ − x if x ≥ 0, f : x ∈ R 7→ +∞ else. 1 − √ , x > 0, ∂f(x) = 2 x ∅, else. Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Subdifferentiation and conjugacy of convex functions Examples of subdifferentiation (Indicator function/Normal cone) Let S ⊂ E. Indicator function of S: ( 0, x ∈ S, S δ : E → R ∪ {+∞}, δ (x) := S S +∞, else. n o NS (0) ∂δS (x¯) = v δC (x) ≥ δC (x¯) + hv, x − x¯i (x ∈ E) n o = v ∈ E hv, x − x¯i ≤ 0 (x ∈ S) =: NS (x¯)(x¯ ∈ S) Figure: Normal cone (Empty subdifferential) ( √ − x if x ≥ 0, f : x ∈ R 7→ +∞ else. 1 − √ , x > 0, ∂f(x) = 2 x ∅, else. Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Subdifferentiation and conjugacy of convex functions Examples of subdifferentiation (Indicator function/Normal cone) Let S ⊂ E. Indicator function of S: ( 0, x ∈ S, S δ : E → R ∪ {+∞}, δ (x) := S S +∞, else. n o NS (0) ∂δS (x¯) = v δC (x) ≥ δC (x¯) + hv, x − x¯i (x ∈ E) n o = v ∈ E hv, x − x¯i ≤ 0 (x ∈ S) =: NS (x¯)(x¯ ∈ S) Figure: Normal cone √ (Euclidean norm) k · k := h·, ·i. Then n o x¯ if x¯ , 0, ∂k · k(x¯) = kx¯k B if x¯ = 0. Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Subdifferentiation and conjugacy of convex functions Examples of subdifferentiation (Indicator function/Normal cone) Let S ⊂ E. Indicator function of S: ( 0, x ∈ S, S δ : E → R ∪ {+∞}, δ (x) := S S +∞, else. n o NS (0) ∂δS (x¯) = v δC (x) ≥ δC (x¯) + hv, x − x¯i (x ∈ E) n o = v ∈ E hv, x − x¯i ≤ 0 (x ∈ S) =: NS (x¯)(x¯ ∈ S) Figure: Normal cone √ (Euclidean norm) k · k := h·, ·i. Then n o x¯ if x¯ , 0, ∂k · k(x¯) = kx¯k B if x¯ = 0. (Empty subdifferential) ( √ − x if x ≥ 0, f : x ∈ R 7→ +∞ else. 1 − √ , x > 0, ∂f(x) = 2 x ∅, else. =⇒ f ∗(v) ≤ β ⇐⇒ sup {hv, xi − f(x)} ≤ β ((v, β) ∈ E × R) x∈E =⇒ f ∗(v) = sup {hv, xi − f(x)} (v ∈ E). (2) x∈E Definition 5 (Fenchel conjugate). Let f : E → R proper. The function f ∗ : E → R defined through (2) is called the (Fenchel) conjugate of f. The function (f ∗∗) := (f ∗)∗ is called the biconjugate of f. n o Define Γ := f : E → R | f convex and proper and Γ0 := {f ∈ Γ | f closed } . f ∗ closed and convex - proper if f . +∞ with an affine minorant ∗∗ f = f proper ⇐⇒ f ∈ Γ0 (Fenchel-Moreau) f ∗ = (cl f)∗ (f ∈ Γ) f(x) + f ∗(y) ≥ hx, yi (x, y ∈ E) (Fenchel-Young Inequality) Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Subdifferentiation and conjugacy of convex functions The Fenchel conjugate For f : E → R ∪ {+∞} let f ∗ : E → R be the function whose epigraph encodes the affine minorants of epi f: ∗ ! n o epi f = (v, β) hv, xi − β ≤ f(x)(x ∈ E) =⇒ f ∗(v) = sup {hv, xi − f(x)} (v ∈ E). (2) x∈E Definition 5 (Fenchel conjugate). Let f : E → R proper. The function f ∗ : E → R defined through (2) is called the (Fenchel) conjugate of f. The function (f ∗∗) := (f ∗)∗ is called the biconjugate of f. n o Define Γ := f : E → R | f convex and proper and Γ0 := {f ∈ Γ | f closed } . f ∗ closed and convex - proper if f . +∞ with an affine minorant ∗∗ f = f proper ⇐⇒ f ∈ Γ0 (Fenchel-Moreau) f ∗ = (cl f)∗ (f ∈ Γ) f(x) + f ∗(y) ≥ hx, yi (x, y ∈ E) (Fenchel-Young Inequality) Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Subdifferentiation and conjugacy of convex functions The Fenchel conjugate For f : E → R ∪ {+∞} let f ∗ : E → R be the function whose epigraph encodes the affine minorants of epi f: ∗ ! n o epi f = (v, β) hv, xi − β ≤ f(x)(x ∈ E) =⇒ f ∗(v) ≤ β ⇐⇒ sup {hv, xi − f(x)} ≤ β ((v, β) ∈ E × R) x∈E Definition 5 (Fenchel conjugate). Let f : E → R proper. The function f ∗ : E → R defined through (2) is called the (Fenchel) conjugate of f. The function (f ∗∗) := (f ∗)∗ is called the biconjugate of f. n o Define Γ := f : E → R | f convex and proper and Γ0 := {f ∈ Γ | f closed } . f ∗ closed and convex - proper if f . +∞ with an affine minorant ∗∗ f = f proper ⇐⇒ f ∈ Γ0 (Fenchel-Moreau) f ∗ = (cl f)∗ (f ∈ Γ) f(x) + f ∗(y) ≥ hx, yi (x, y ∈ E) (Fenchel-Young Inequality) Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Subdifferentiation and conjugacy of convex functions The Fenchel conjugate For f : E → R ∪ {+∞} let f ∗ : E → R be the function whose epigraph encodes the affine minorants of epi f: ∗ ! n o epi f = (v, β) hv, xi − β ≤ f(x)(x ∈ E) =⇒ f ∗(v) ≤ β ⇐⇒ sup {hv, xi − f(x)} ≤ β ((v, β) ∈ E × R) x∈E =⇒ f ∗(v) = sup {hv, xi − f(x)} (v ∈ E). (2) x∈E n o Define Γ := f : E → R | f convex and proper and Γ0 := {f ∈ Γ | f closed } . f ∗ closed and convex - proper if f . +∞ with an affine minorant ∗∗ f = f proper ⇐⇒ f ∈ Γ0 (Fenchel-Moreau) f ∗ = (cl f)∗ (f ∈ Γ) f(x) + f ∗(y) ≥ hx, yi (x, y ∈ E) (Fenchel-Young Inequality) Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Subdifferentiation and conjugacy of convex functions The Fenchel conjugate For f : E → R ∪ {+∞} let f ∗ : E → R be the function whose epigraph encodes the affine minorants of epi f: ∗ ! n o epi f = (v, β) hv, xi − β ≤ f(x)(x ∈ E) =⇒ f ∗(v) ≤ β ⇐⇒ sup {hv, xi − f(x)} ≤ β ((v, β) ∈ E × R) x∈E =⇒ f ∗(v) = sup {hv, xi − f(x)} (v ∈ E). (2) x∈E Definition 5 (Fenchel conjugate). Let f : E → R proper. The function f ∗ : E → R defined through (2) is called the (Fenchel) conjugate of f. The function (f ∗∗) := (f ∗)∗ is called the biconjugate of f. f ∗ closed and convex - proper if f . +∞ with an affine minorant ∗∗ f = f proper ⇐⇒ f ∈ Γ0 (Fenchel-Moreau) f ∗ = (cl f)∗ (f ∈ Γ) f(x) + f ∗(y) ≥ hx, yi (x, y ∈ E) (Fenchel-Young Inequality) Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Subdifferentiation and conjugacy of convex functions The Fenchel conjugate For f : E → R ∪ {+∞} let f ∗ : E → R be the function whose epigraph encodes the affine minorants of epi f: ∗ ! n o epi f = (v, β) hv, xi − β ≤ f(x)(x ∈ E) =⇒ f ∗(v) ≤ β ⇐⇒ sup {hv, xi − f(x)} ≤ β ((v, β) ∈ E × R) x∈E =⇒ f ∗(v) = sup {hv, xi − f(x)} (v ∈ E). (2) x∈E Definition 5 (Fenchel conjugate). Let f : E → R proper. The function f ∗ : E → R defined through (2) is called the (Fenchel) conjugate of f. The function (f ∗∗) := (f ∗)∗ is called the biconjugate of f. n o Define Γ := f : E → R | f convex and proper and Γ0 := {f ∈ Γ | f closed } . ∗∗ f = f proper ⇐⇒ f ∈ Γ0 (Fenchel-Moreau) f ∗ = (cl f)∗ (f ∈ Γ) f(x) + f ∗(y) ≥ hx, yi (x, y ∈ E) (Fenchel-Young Inequality) Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Subdifferentiation and conjugacy of convex functions The Fenchel conjugate For f : E → R ∪ {+∞} let f ∗ : E → R be the function whose epigraph encodes the affine minorants of epi f: ∗ ! n o epi f = (v, β) hv, xi − β ≤ f(x)(x ∈ E) =⇒ f ∗(v) ≤ β ⇐⇒ sup {hv, xi − f(x)} ≤ β ((v, β) ∈ E × R) x∈E =⇒ f ∗(v) = sup {hv, xi − f(x)} (v ∈ E). (2) x∈E Definition 5 (Fenchel conjugate). Let f : E → R proper. The function f ∗ : E → R defined through (2) is called the (Fenchel) conjugate of f. The function (f ∗∗) := (f ∗)∗ is called the biconjugate of f. n o Define Γ := f : E → R | f convex and proper and Γ0 := {f ∈ Γ | f closed } . f ∗ closed and convex - proper if f . +∞ with an affine minorant f ∗ = (cl f)∗ (f ∈ Γ) f(x) + f ∗(y) ≥ hx, yi (x, y ∈ E) (Fenchel-Young Inequality) Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Subdifferentiation and conjugacy of convex functions The Fenchel conjugate For f : E → R ∪ {+∞} let f ∗ : E → R be the function whose epigraph encodes the affine minorants of epi f: ∗ ! n o epi f = (v, β) hv, xi − β ≤ f(x)(x ∈ E) =⇒ f ∗(v) ≤ β ⇐⇒ sup {hv, xi − f(x)} ≤ β ((v, β) ∈ E × R) x∈E =⇒ f ∗(v) = sup {hv, xi − f(x)} (v ∈ E). (2) x∈E Definition 5 (Fenchel conjugate). Let f : E → R proper. The function f ∗ : E → R defined through (2) is called the (Fenchel) conjugate of f. The function (f ∗∗) := (f ∗)∗ is called the biconjugate of f. n o Define Γ := f : E → R | f convex and proper and Γ0 := {f ∈ Γ | f closed } . f ∗ closed and convex - proper if f . +∞ with an affine minorant ∗∗ f = f proper ⇐⇒ f ∈ Γ0 (Fenchel-Moreau) f(x) + f ∗(y) ≥ hx, yi (x, y ∈ E) (Fenchel-Young Inequality) Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Subdifferentiation and conjugacy of convex functions The Fenchel conjugate For f : E → R ∪ {+∞} let f ∗ : E → R be the function whose epigraph encodes the affine minorants of epi f: ∗ ! n o epi f = (v, β) hv, xi − β ≤ f(x)(x ∈ E) =⇒ f ∗(v) ≤ β ⇐⇒ sup {hv, xi − f(x)} ≤ β ((v, β) ∈ E × R) x∈E =⇒ f ∗(v) = sup {hv, xi − f(x)} (v ∈ E). (2) x∈E Definition 5 (Fenchel conjugate). Let f : E → R proper. The function f ∗ : E → R defined through (2) is called the (Fenchel) conjugate of f. The function (f ∗∗) := (f ∗)∗ is called the biconjugate of f. n o Define Γ := f : E → R | f convex and proper and Γ0 := {f ∈ Γ | f closed } . f ∗ closed and convex - proper if f . +∞ with an affine minorant ∗∗ f = f proper ⇐⇒ f ∈ Γ0 (Fenchel-Moreau) f ∗ = (cl f)∗ (f ∈ Γ) Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Subdifferentiation and conjugacy of convex functions The Fenchel conjugate For f : E → R ∪ {+∞} let f ∗ : E → R be the function whose epigraph encodes the affine minorants of epi f: ∗ ! n o epi f = (v, β) hv, xi − β ≤ f(x)(x ∈ E) =⇒ f ∗(v) ≤ β ⇐⇒ sup {hv, xi − f(x)} ≤ β ((v, β) ∈ E × R) x∈E =⇒ f ∗(v) = sup {hv, xi − f(x)} (v ∈ E). (2) x∈E Definition 5 (Fenchel conjugate). Let f : E → R proper. The function f ∗ : E → R defined through (2) is called the (Fenchel) conjugate of f. The function (f ∗∗) := (f ∗)∗ is called the biconjugate of f. n o Define Γ := f : E → R | f convex and proper and Γ0 := {f ∈ Γ | f closed } . f ∗ closed and convex - proper if f . +∞ with an affine minorant ∗∗ f = f proper ⇐⇒ f ∈ Γ0 (Fenchel-Moreau) f ∗ = (cl f)∗ (f ∈ Γ) f(x) + f ∗(y) ≥ hx, yi (x, y ∈ E) (Fenchel-Young Inequality) Proof. Notice that y ∈ ∂f(x) ⇐⇒ f(z) ≥ f(x) + hy, z − xi (z ∈ E) ⇐⇒ hy, xi − f(x) ≥ sup{hy, zi − f(z)} z ⇐⇒ f(x) + f ∗(y) ≤ hx, yi Fenchel−Young ⇐⇒ f(x) + f ∗(y) = hx, yi , ∗ ∗∗ Applying the same reasoning to f and noticing that f = f if f ∈ Γ0, gives the missing equivalence. Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Subdifferentiation and conjugacy of convex functions Interplay of conjugation and subdifferentiation Theorem 6 (Subdifferential and conjugate function). Let f ∈ Γ0. TFAE: i) y ∈ ∂f(x); ii) f(x) + f ∗(y) = hx, yi; iii) x ∈ ∂f ∗(y). In particular, ∂f ∗ = (∂f)−1. ⇐⇒ hy, xi − f(x) ≥ sup{hy, zi − f(z)} z ⇐⇒ f(x) + f ∗(y) ≤ hx, yi Fenchel−Young ⇐⇒ f(x) + f ∗(y) = hx, yi , ∗ ∗∗ Applying the same reasoning to f and noticing that f = f if f ∈ Γ0, gives the missing equivalence. Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Subdifferentiation and conjugacy of convex functions Interplay of conjugation and subdifferentiation Theorem 6 (Subdifferential and conjugate function). Let f ∈ Γ0. TFAE: i) y ∈ ∂f(x); ii) f(x) + f ∗(y) = hx, yi; iii) x ∈ ∂f ∗(y). In particular, ∂f ∗ = (∂f)−1. Proof. Notice that y ∈ ∂f(x) ⇐⇒ f(z) ≥ f(x) + hy, z − xi (z ∈ E) ⇐⇒ f(x) + f ∗(y) ≤ hx, yi Fenchel−Young ⇐⇒ f(x) + f ∗(y) = hx, yi , ∗ ∗∗ Applying the same reasoning to f and noticing that f = f if f ∈ Γ0, gives the missing equivalence. Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Subdifferentiation and conjugacy of convex functions Interplay of conjugation and subdifferentiation Theorem 6 (Subdifferential and conjugate function). Let f ∈ Γ0. TFAE: i) y ∈ ∂f(x); ii) f(x) + f ∗(y) = hx, yi; iii) x ∈ ∂f ∗(y). In particular, ∂f ∗ = (∂f)−1. Proof. Notice that y ∈ ∂f(x) ⇐⇒ f(z) ≥ f(x) + hy, z − xi (z ∈ E) ⇐⇒ hy, xi − f(x) ≥ sup{hy, zi − f(z)} z Fenchel−Young ⇐⇒ f(x) + f ∗(y) = hx, yi , ∗ ∗∗ Applying the same reasoning to f and noticing that f = f if f ∈ Γ0, gives the missing equivalence. Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Subdifferentiation and conjugacy of convex functions Interplay of conjugation and subdifferentiation Theorem 6 (Subdifferential and conjugate function). Let f ∈ Γ0. TFAE: i) y ∈ ∂f(x); ii) f(x) + f ∗(y) = hx, yi; iii) x ∈ ∂f ∗(y). In particular, ∂f ∗ = (∂f)−1. Proof. Notice that y ∈ ∂f(x) ⇐⇒ f(z) ≥ f(x) + hy, z − xi (z ∈ E) ⇐⇒ hy, xi − f(x) ≥ sup{hy, zi − f(z)} z ⇐⇒ f(x) + f ∗(y) ≤ hx, yi ∗ ∗∗ Applying the same reasoning to f and noticing that f = f if f ∈ Γ0, gives the missing equivalence. Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Subdifferentiation and conjugacy of convex functions Interplay of conjugation and subdifferentiation Theorem 6 (Subdifferential and conjugate function). Let f ∈ Γ0. TFAE: i) y ∈ ∂f(x); ii) f(x) + f ∗(y) = hx, yi; iii) x ∈ ∂f ∗(y). In particular, ∂f ∗ = (∂f)−1. Proof. Notice that y ∈ ∂f(x) ⇐⇒ f(z) ≥ f(x) + hy, z − xi (z ∈ E) ⇐⇒ hy, xi − f(x) ≥ sup{hy, zi − f(z)} z ⇐⇒ f(x) + f ∗(y) ≤ hx, yi Fenchel−Young ⇐⇒ f(x) + f ∗(y) = hx, yi , Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Subdifferentiation and conjugacy of convex functions Interplay of conjugation and subdifferentiation Theorem 6 (Subdifferential and conjugate function). Let f ∈ Γ0. TFAE: i) y ∈ ∂f(x); ii) f(x) + f ∗(y) = hx, yi; iii) x ∈ ∂f ∗(y). In particular, ∂f ∗ = (∂f)−1. Proof. Notice that y ∈ ∂f(x) ⇐⇒ f(z) ≥ f(x) + hy, z − xi (z ∈ E) ⇐⇒ hy, xi − f(x) ≥ sup{hy, zi − f(z)} z ⇐⇒ f(x) + f ∗(y) ≤ hx, yi Fenchel−Young ⇐⇒ f(x) + f ∗(y) = hx, yi , ∗ ∗∗ Applying the same reasoning to f and noticing that f = f if f ∈ Γ0, gives the missing equivalence. σS is finite-valued if and only if S is bounded (and nonempty) σS = σconv S = σconv S = σcl S ∗ σS = δconv S n o ∂σS (x) = z ∈ conv S x ∈ Nconv S (z) T epi σS = s∈S epi hs, ·i is a nonempty, closed, convex cone, i.e. σS is proper, closed and sublinear. Here’s the complete picture: Theorem 7 (Hormander).¨ A function f : E → R is proper, closed and sublinear if and only if it is a support function. Proof. Blackboard/Notes. Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Subdifferentiation and conjugacy of convex functions Support functions: A special case of conjugacy The support function σS of S ⊂ E (nonempty) is defined by ∗ σS : E → R ∪ {+∞}, σS (z) := δS (z) = sup hx, zi . x∈S ∗ σS = δconv S n o ∂σS (x) = z ∈ conv S x ∈ Nconv S (z) T epi σS = s∈S epi hs, ·i is a nonempty, closed, convex cone, i.e. σS is proper, closed and sublinear. Here’s the complete picture: Theorem 7 (Hormander).¨ A function f : E → R is proper, closed and sublinear if and only if it is a support function. Proof. Blackboard/Notes. Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Subdifferentiation and conjugacy of convex functions Support functions: A special case of conjugacy The support function σS of S ⊂ E (nonempty) is defined by ∗ σS : E → R ∪ {+∞}, σS (z) := δS (z) = sup hx, zi . x∈S σS is finite-valued if and only if S is bounded (and nonempty) σS = σconv S = σconv S = σcl S n o ∂σS (x) = z ∈ conv S x ∈ Nconv S (z) T epi σS = s∈S epi hs, ·i is a nonempty, closed, convex cone, i.e. σS is proper, closed and sublinear. Here’s the complete picture: Theorem 7 (Hormander).¨ A function f : E → R is proper, closed and sublinear if and only if it is a support function. Proof. Blackboard/Notes. Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Subdifferentiation and conjugacy of convex functions Support functions: A special case of conjugacy The support function σS of S ⊂ E (nonempty) is defined by ∗ σS : E → R ∪ {+∞}, σS (z) := δS (z) = sup hx, zi . x∈S σS is finite-valued if and only if S is bounded (and nonempty) σS = σconv S = σconv S = σcl S ∗ σS = δconv S T epi σS = s∈S epi hs, ·i is a nonempty, closed, convex cone, i.e. σS is proper, closed and sublinear. Here’s the complete picture: Theorem 7 (Hormander).¨ A function f : E → R is proper, closed and sublinear if and only if it is a support function. Proof. Blackboard/Notes. Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Subdifferentiation and conjugacy of convex functions Support functions: A special case of conjugacy The support function σS of S ⊂ E (nonempty) is defined by ∗ σS : E → R ∪ {+∞}, σS (z) := δS (z) = sup hx, zi . x∈S σS is finite-valued if and only if S is bounded (and nonempty) σS = σconv S = σconv S = σcl S ∗ σS = δconv S n o ∂σS (x) = z ∈ conv S x ∈ Nconv S (z) Here’s the complete picture: Theorem 7 (Hormander).¨ A function f : E → R is proper, closed and sublinear if and only if it is a support function. Proof. Blackboard/Notes. Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Subdifferentiation and conjugacy of convex functions Support functions: A special case of conjugacy The support function σS of S ⊂ E (nonempty) is defined by ∗ σS : E → R ∪ {+∞}, σS (z) := δS (z) = sup hx, zi . x∈S σS is finite-valued if and only if S is bounded (and nonempty) σS = σconv S = σconv S = σcl S ∗ σS = δconv S n o ∂σS (x) = z ∈ conv S x ∈ Nconv S (z) T epi σS = s∈S epi hs, ·i is a nonempty, closed, convex cone, i.e. σS is proper, closed and sublinear. Here’s the complete picture: Theorem 7 (Hormander).¨ A function f : E → R is proper, closed and sublinear if and only if it is a support function. Proof. Blackboard/Notes. Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Subdifferentiation and conjugacy of convex functions Support functions: A special case of conjugacy The support function σS of S ⊂ E (nonempty) is defined by ∗ σS : E → R ∪ {+∞}, σS (z) := δS (z) = sup hx, zi . x∈S σS is finite-valued if and only if S is bounded (and nonempty) σS = σconv S = σconv S = σcl S ∗ σS = δconv S n o ∂σS (x) = z ∈ conv S x ∈ Nconv S (z) T epi σS = s∈S epi hs, ·i is a nonempty, closed, convex cone, i.e. σS is proper, closed and sublinear. Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Subdifferentiation and conjugacy of convex functions Support functions: A special case of conjugacy The support function σS of S ⊂ E (nonempty) is defined by ∗ σS : E → R ∪ {+∞}, σS (z) := δS (z) = sup hx, zi . x∈S σS is finite-valued if and only if S is bounded (and nonempty) σS = σconv S = σconv S = σcl S ∗ σS = δconv S n o ∂σS (x) = z ∈ conv S x ∈ Nconv S (z) T epi σS = s∈S epi hs, ·i is a nonempty, closed, convex cone, i.e. σS is proper, closed and sublinear. Here’s the complete picture: Theorem 7 (Hormander).¨ A function f : E → R is proper, closed and sublinear if and only if it is a support function. Proof. Blackboard/Notes. If C ⊂ E be nonempty, closed and convex with 0 ∈ C, then γC is proper, lsc and sublinear. Definition 9 (Polar sets). Let C ⊂ E. Then its polar set is defined by ◦ n o C := v ∈ E hv, xi ≤ 1 (x ∈ C) . Moreover, we put C◦◦ := (C◦)◦ and call it the bipolar set of C. n o If K is a cone then K ◦ = v ∈ E hv, xi ≤ 0 (x ∈ K) . For C ⊂ E we have C◦◦ = conv (C ∪ {0}). (bipolar theorem) Proposition 10. Let C ⊂ E be closed and convex with 0 ∈ C. Then ∗ ∗ γC = σC◦ ←→ δC◦ and γC◦ = σC ←→ δC . Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Subdifferentiation and conjugacy of convex functions Gauges and polar sets Definition 8 (Gauge function). Let C ⊂ E. The gauge (function) of C is defined by γC : x ∈ E 7→ inf {λ ≥ 0 | x ∈ λC } . Definition 9 (Polar sets). Let C ⊂ E. Then its polar set is defined by ◦ n o C := v ∈ E hv, xi ≤ 1 (x ∈ C) . Moreover, we put C◦◦ := (C◦)◦ and call it the bipolar set of C. n o If K is a cone then K ◦ = v ∈ E hv, xi ≤ 0 (x ∈ K) . For C ⊂ E we have C◦◦ = conv (C ∪ {0}). (bipolar theorem) Proposition 10. Let C ⊂ E be closed and convex with 0 ∈ C. Then ∗ ∗ γC = σC◦ ←→ δC◦ and γC◦ = σC ←→ δC . Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Subdifferentiation and conjugacy of convex functions Gauges and polar sets Definition 8 (Gauge function). Let C ⊂ E. The gauge (function) of C is defined by γC : x ∈ E 7→ inf {λ ≥ 0 | x ∈ λC } . If C ⊂ E be nonempty, closed and convex with 0 ∈ C, then γC is proper, lsc and sublinear. Proposition 10. Let C ⊂ E be closed and convex with 0 ∈ C. Then ∗ ∗ γC = σC◦ ←→ δC◦ and γC◦ = σC ←→ δC . Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Subdifferentiation and conjugacy of convex functions Gauges and polar sets Definition 8 (Gauge function). Let C ⊂ E. The gauge (function) of C is defined by γC : x ∈ E 7→ inf {λ ≥ 0 | x ∈ λC } . If C ⊂ E be nonempty, closed and convex with 0 ∈ C, then γC is proper, lsc and sublinear. Definition 9 (Polar sets). Let C ⊂ E. Then its polar set is defined by ◦ n o C := v ∈ E hv, xi ≤ 1 (x ∈ C) . Moreover, we put C◦◦ := (C◦)◦ and call it the bipolar set of C. n o If K is a cone then K ◦ = v ∈ E hv, xi ≤ 0 (x ∈ K) . For C ⊂ E we have C◦◦ = conv (C ∪ {0}). (bipolar theorem) Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Subdifferentiation and conjugacy of convex functions Gauges and polar sets Definition 8 (Gauge function). Let C ⊂ E. The gauge (function) of C is defined by γC : x ∈ E 7→ inf {λ ≥ 0 | x ∈ λC } . If C ⊂ E be nonempty, closed and convex with 0 ∈ C, then γC is proper, lsc and sublinear. Definition 9 (Polar sets). Let C ⊂ E. Then its polar set is defined by ◦ n o C := v ∈ E hv, xi ≤ 1 (x ∈ C) . Moreover, we put C◦◦ := (C◦)◦ and call it the bipolar set of C. n o If K is a cone then K ◦ = v ∈ E hv, xi ≤ 0 (x ∈ K) . For C ⊂ E we have C◦◦ = conv (C ∪ {0}). (bipolar theorem) Proposition 10. Let C ⊂ E be closed and convex with 0 ∈ C. Then ∗ ∗ γC = σC◦ ←→ δC◦ and γC◦ = σC ←→ δC . Proof. Let L :(x, y, α) 7→ (x, α) and observe that ( ) epi = L(epi <ψ). Hence epi Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Infimal convolution and the Attouch-Brezis´ Theorem Infimal projection Theorem 11 (Infimal projection). Let ψ : E1 × E2 → R ∪ {+∞} be convex. Then the optimal value function p : E1 → R, p(x) := inf ψ(x, y) y∈E2 is convex. n o = (x, α) ∃y : ψ(x, y) < α = L(epi <ψ). Hence epi Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Infimal convolution and the Attouch-Brezis´ Theorem Infimal projection Theorem 11 (Infimal projection). Let ψ : E1 × E2 → R ∪ {+∞} be convex. Then the optimal value function p : E1 → R, p(x) := inf ψ(x, y) y∈E2 is convex. Proof. Let L :(x, y, α) 7→ (x, α) and observe that ( ) epi Hence epi Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Infimal convolution and the Attouch-Brezis´ Theorem Infimal projection Theorem 11 (Infimal projection). Let ψ : E1 × E2 → R ∪ {+∞} be convex. Then the optimal value function p : E1 → R, p(x) := inf ψ(x, y) y∈E2 is convex. Proof. Let L :(x, y, α) 7→ (x, α) and observe that ( ) epi Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Infimal convolution and the Attouch-Brezis´ Theorem Infimal projection Theorem 11 (Infimal projection). Let ψ : E1 × E2 → R ∪ {+∞} be convex. Then the optimal value function p : E1 → R, p(x) := inf ψ(x, y) y∈E2 is convex. Proof. Let L :(x, y, α) 7→ (x, α) and observe that ( ) epi = L(epi <ψ). Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Infimal convolution and the Attouch-Brezis´ Theorem Infimal projection Theorem 11 (Infimal projection). Let ψ : E1 × E2 → R ∪ {+∞} be convex. Then the optimal value function p : E1 → R, p(x) := inf ψ(x, y) y∈E2 is convex. Proof. Let L :(x, y, α) 7→ (x, α) and observe that ( ) epi = L(epi <ψ). Hence epi f, g convex, then f#g convex (as (f#g)(x) = infy h(x, y) with h :(x, y) 7→ f(y) + g(x − y) convex). Example 13 (Distance functions). Let C ⊂ E. Then dC := δC #k · k, i.e. dC (x) = inf kx − uk u∈C is the distance function of C, which is hence convex if C is a convex. Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Infimal convolution and the Attouch-Brezis´ Theorem Infimal convolution - a special case of infimal projection Definition 12 (Infimal convolution). Let f, g : E → R ∪ {+∞}. Then the function f#g : E → R, (f#g)(x) := inf{f(u) + g(x − u)} u∈E is called the infimal convolution of f and g. We call the infimal convolution f#g exact at x ∈ E if argmin{f(u) + g(x − u)} , ∅. u∈E We simply call f#g exact if it is exact at every x ∈ dom f#g. Example 13 (Distance functions). Let C ⊂ E. Then dC := δC #k · k, i.e. dC (x) = inf kx − uk u∈C is the distance function of C, which is hence convex if C is a convex. Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Infimal convolution and the Attouch-Brezis´ Theorem Infimal convolution - a special case of infimal projection Definition 12 (Infimal convolution). Let f, g : E → R ∪ {+∞}. Then the function f#g : E → R, (f#g)(x) := inf{f(u) + g(x − u)} u∈E is called the infimal convolution of f and g. We call the infimal convolution f#g exact at x ∈ E if argmin{f(u) + g(x − u)} , ∅. u∈E We simply call f#g exact if it is exact at every x ∈ dom f#g. We always have: dom f#g = dom f + dom g; f#g = g#f; f, g convex, then f#g convex (as (f#g)(x) = infy h(x, y) with h :(x, y) 7→ f(y) + g(x − y) convex). Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Infimal convolution and the Attouch-Brezis´ Theorem Infimal convolution - a special case of infimal projection Definition 12 (Infimal convolution). Let f, g : E → R ∪ {+∞}. Then the function f#g : E → R, (f#g)(x) := inf{f(u) + g(x − u)} u∈E is called the infimal convolution of f and g. We call the infimal convolution f#g exact at x ∈ E if argmin{f(u) + g(x − u)} , ∅. u∈E We simply call f#g exact if it is exact at every x ∈ dom f#g. We always have: dom f#g = dom f + dom g; f#g = g#f; f, g convex, then f#g convex (as (f#g)(x) = infy h(x, y) with h :(x, y) 7→ f(y) + g(x − y) convex). Example 13 (Distance functions). Let C ⊂ E. Then dC := δC #k · k, i.e. dC (x) = inf kx − uk u∈C is the distance function of C, which is hence convex if C is a convex. Proof. a) For all y ∈ E, we have (f#g)∗(y) = sup hx, yi − inf f(u) + g(x − u) x u = sup hx, yi − f(u) − g(x − u) x,u = sup (hu, yi − f(u)) + (hx − u, yi − g(x − u)) x,u = f ∗(y) + g∗(y). a) f,g∈Γ clear? b) (f ∗#g∗)∗ = f ∗∗ + g∗∗ = 0 f + g ∈ Γ =⇒ cl (f ∗#g∗) = (f ∗#g∗)∗∗ = (f + g)∗. Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Infimal convolution and the Attouch-Brezis´ Theorem Conjugacy of infimal convolution Proposition 14 (Conjugacy of inf-convolution). Let f, g : E → R ∪ {+∞}. Then the following hold: a) (f#g)∗ = f ∗ + g∗; ∗ ∗ ∗ b) If f, g ∈ Γ0 such that dom f ∩ dom g , ∅, then (f + g) = cl (f #g ). = sup hx, yi − f(u) − g(x − u) x,u = sup (hu, yi − f(u)) + (hx − u, yi − g(x − u)) x,u = f ∗(y) + g∗(y). a) f,g∈Γ clear? b) (f ∗#g∗)∗ = f ∗∗ + g∗∗ = 0 f + g ∈ Γ =⇒ cl (f ∗#g∗) = (f ∗#g∗)∗∗ = (f + g)∗. Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Infimal convolution and the Attouch-Brezis´ Theorem Conjugacy of infimal convolution Proposition 14 (Conjugacy of inf-convolution). Let f, g : E → R ∪ {+∞}. Then the following hold: a) (f#g)∗ = f ∗ + g∗; ∗ ∗ ∗ b) If f, g ∈ Γ0 such that dom f ∩ dom g , ∅, then (f + g) = cl (f #g ). Proof. a) For all y ∈ E, we have (f#g)∗(y) = sup hx, yi − inf f(u) + g(x − u) x u = sup (hu, yi − f(u)) + (hx − u, yi − g(x − u)) x,u = f ∗(y) + g∗(y). a) f,g∈Γ clear? b) (f ∗#g∗)∗ = f ∗∗ + g∗∗ = 0 f + g ∈ Γ =⇒ cl (f ∗#g∗) = (f ∗#g∗)∗∗ = (f + g)∗. Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Infimal convolution and the Attouch-Brezis´ Theorem Conjugacy of infimal convolution Proposition 14 (Conjugacy of inf-convolution). Let f, g : E → R ∪ {+∞}. Then the following hold: a) (f#g)∗ = f ∗ + g∗; ∗ ∗ ∗ b) If f, g ∈ Γ0 such that dom f ∩ dom g , ∅, then (f + g) = cl (f #g ). Proof. a) For all y ∈ E, we have (f#g)∗(y) = sup hx, yi − inf f(u) + g(x − u) x u = sup hx, yi − f(u) − g(x − u) x,u = f ∗(y) + g∗(y). a) f,g∈Γ clear? b) (f ∗#g∗)∗ = f ∗∗ + g∗∗ = 0 f + g ∈ Γ =⇒ cl (f ∗#g∗) = (f ∗#g∗)∗∗ = (f + g)∗. Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Infimal convolution and the Attouch-Brezis´ Theorem Conjugacy of infimal convolution Proposition 14 (Conjugacy of inf-convolution). Let f, g : E → R ∪ {+∞}. Then the following hold: a) (f#g)∗ = f ∗ + g∗; ∗ ∗ ∗ b) If f, g ∈ Γ0 such that dom f ∩ dom g , ∅, then (f + g) = cl (f #g ). Proof. a) For all y ∈ E, we have (f#g)∗(y) = sup hx, yi − inf f(u) + g(x − u) x u = sup hx, yi − f(u) − g(x − u) x,u = sup (hu, yi − f(u)) + (hx − u, yi − g(x − u)) x,u a) f,g∈Γ clear? b) (f ∗#g∗)∗ = f ∗∗ + g∗∗ = 0 f + g ∈ Γ =⇒ cl (f ∗#g∗) = (f ∗#g∗)∗∗ = (f + g)∗. Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Infimal convolution and the Attouch-Brezis´ Theorem Conjugacy of infimal convolution Proposition 14 (Conjugacy of inf-convolution). Let f, g : E → R ∪ {+∞}. Then the following hold: a) (f#g)∗ = f ∗ + g∗; ∗ ∗ ∗ b) If f, g ∈ Γ0 such that dom f ∩ dom g , ∅, then (f + g) = cl (f #g ). Proof. a) For all y ∈ E, we have (f#g)∗(y) = sup hx, yi − inf f(u) + g(x − u) x u = sup hx, yi − f(u) − g(x − u) x,u = sup (hu, yi − f(u)) + (hx − u, yi − g(x − u)) x,u = f ∗(y) + g∗(y). =⇒ cl (f ∗#g∗) = (f ∗#g∗)∗∗ = (f + g)∗. Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Infimal convolution and the Attouch-Brezis´ Theorem Conjugacy of infimal convolution Proposition 14 (Conjugacy of inf-convolution). Let f, g : E → R ∪ {+∞}. Then the following hold: a) (f#g)∗ = f ∗ + g∗; ∗ ∗ ∗ b) If f, g ∈ Γ0 such that dom f ∩ dom g , ∅, then (f + g) = cl (f #g ). Proof. a) For all y ∈ E, we have (f#g)∗(y) = sup hx, yi − inf f(u) + g(x − u) x u = sup hx, yi − f(u) − g(x − u) x,u = sup (hu, yi − f(u)) + (hx − u, yi − g(x − u)) x,u = f ∗(y) + g∗(y). a) f,g∈Γ clear? b) (f ∗#g∗)∗ = f ∗∗ + g∗∗ = 0 f + g ∈ Γ Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Infimal convolution and the Attouch-Brezis´ Theorem Conjugacy of infimal convolution Proposition 14 (Conjugacy of inf-convolution). Let f, g : E → R ∪ {+∞}. Then the following hold: a) (f#g)∗ = f ∗ + g∗; ∗ ∗ ∗ b) If f, g ∈ Γ0 such that dom f ∩ dom g , ∅, then (f + g) = cl (f #g ). Proof. a) For all y ∈ E, we have (f#g)∗(y) = sup hx, yi − inf f(u) + g(x − u) x u = sup hx, yi − f(u) − g(x − u) x,u = sup (hu, yi − f(u)) + (hx − u, yi − g(x − u)) x,u = f ∗(y) + g∗(y). a) f,g∈Γ clear? b) (f ∗#g∗)∗ = f ∗∗ + g∗∗ = 0 f + g ∈ Γ =⇒ cl (f ∗#g∗) = (f ∗#g∗)∗∗ = (f + g)∗. We note that (CQ) is always satisfied under any of the following: int (dom f) ∩ dom g , ∅, dom f = E, and is equivalent to saying that 0 ∈ ri (dom f − dom g). Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Infimal convolution and the Attouch-Brezis´ Theorem Attouch-Brezis´ - drop the closure! Theorem 15 (Attouch-Brezis).´ Let f, g ∈ Γ0 such that ri (dom f) ∩ ri (dom g) , 0 (CQ). Then (f + g)∗ = f ∗#g∗, and the infimal convolution is exact, i.e. the infimum in the infimal convolution is attained on dom f ∗#g∗. Proof. On blackboard. Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Infimal convolution and the Attouch-Brezis´ Theorem Attouch-Brezis´ - drop the closure! Theorem 15 (Attouch-Brezis).´ Let f, g ∈ Γ0 such that ri (dom f) ∩ ri (dom g) , 0 (CQ). Then (f + g)∗ = f ∗#g∗, and the infimal convolution is exact, i.e. the infimum in the infimal convolution is attained on dom f ∗#g∗. Proof. On blackboard. We note that (CQ) is always satisfied under any of the following: int (dom f) ∩ dom g , ∅, dom f = E, and is equivalent to saying that 0 ∈ ri (dom f − dom g). The map Pλf : E → E given by ( ) 1 2 Pλf(x) := argmin f(u) + kx − uk . u 2λ We have Pλf is 1-Lipschitz (in fact, firmly non-expansive) 1,1 eλf ∈ C ∩ Γ0 1 ∇eλf = λ (id − Pλf) x ∈ argmin f ⇐⇒ x ∈ argmin eλf ⇐⇒ x = Pλf(x) (→ proximal point/gradient method) eλf ↑ f (λ ↓ 0) (monotone pointwise convergence) epi eλf → epi f (λ ↓ 0) (epi-convergence) Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Infimal convolution and the Attouch-Brezis´ Theorem Excursion: Moreau envelope and proximal operator 1 Let f ∈ Γ0 and λ > 0. Then 1 e f := f# k · k2 λ 2λ is called the Moreau envelope of f. 1Not in lecture notes! 1,1 eλf ∈ C ∩ Γ0 1 ∇eλf = λ (id − Pλf) x ∈ argmin f ⇐⇒ x ∈ argmin eλf ⇐⇒ x = Pλf(x) (→ proximal point/gradient method) eλf ↑ f (λ ↓ 0) (monotone pointwise convergence) epi eλf → epi f (λ ↓ 0) (epi-convergence) Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Infimal convolution and the Attouch-Brezis´ Theorem Excursion: Moreau envelope and proximal operator 1 Let f ∈ Γ0 and λ > 0. Then 1 e f := f# k · k2 λ 2λ is called the Moreau envelope of f. The map Pλf : E → E given by ( ) 1 2 Pλf(x) := argmin f(u) + kx − uk . u 2λ We have Pλf is 1-Lipschitz (in fact, firmly non-expansive) 1Not in lecture notes! 1 ∇eλf = λ (id − Pλf) x ∈ argmin f ⇐⇒ x ∈ argmin eλf ⇐⇒ x = Pλf(x) (→ proximal point/gradient method) eλf ↑ f (λ ↓ 0) (monotone pointwise convergence) epi eλf → epi f (λ ↓ 0) (epi-convergence) Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Infimal convolution and the Attouch-Brezis´ Theorem Excursion: Moreau envelope and proximal operator 1 Let f ∈ Γ0 and λ > 0. Then 1 e f := f# k · k2 λ 2λ is called the Moreau envelope of f. The map Pλf : E → E given by ( ) 1 2 Pλf(x) := argmin f(u) + kx − uk . u 2λ We have Pλf is 1-Lipschitz (in fact, firmly non-expansive) 1,1 eλf ∈ C ∩ Γ0 1Not in lecture notes! x ∈ argmin f ⇐⇒ x ∈ argmin eλf ⇐⇒ x = Pλf(x) (→ proximal point/gradient method) eλf ↑ f (λ ↓ 0) (monotone pointwise convergence) epi eλf → epi f (λ ↓ 0) (epi-convergence) Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Infimal convolution and the Attouch-Brezis´ Theorem Excursion: Moreau envelope and proximal operator 1 Let f ∈ Γ0 and λ > 0. Then 1 e f := f# k · k2 λ 2λ is called the Moreau envelope of f. The map Pλf : E → E given by ( ) 1 2 Pλf(x) := argmin f(u) + kx − uk . u 2λ We have Pλf is 1-Lipschitz (in fact, firmly non-expansive) 1,1 eλf ∈ C ∩ Γ0 1 ∇eλf = λ (id − Pλf) 1Not in lecture notes! eλf ↑ f (λ ↓ 0) (monotone pointwise convergence) epi eλf → epi f (λ ↓ 0) (epi-convergence) Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Infimal convolution and the Attouch-Brezis´ Theorem Excursion: Moreau envelope and proximal operator 1 Let f ∈ Γ0 and λ > 0. Then 1 e f := f# k · k2 λ 2λ is called the Moreau envelope of f. The map Pλf : E → E given by ( ) 1 2 Pλf(x) := argmin f(u) + kx − uk . u 2λ We have Pλf is 1-Lipschitz (in fact, firmly non-expansive) 1,1 eλf ∈ C ∩ Γ0 1 ∇eλf = λ (id − Pλf) x ∈ argmin f ⇐⇒ x ∈ argmin eλf ⇐⇒ x = Pλf(x) (→ proximal point/gradient method) 1Not in lecture notes! epi eλf → epi f (λ ↓ 0) (epi-convergence) Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Infimal convolution and the Attouch-Brezis´ Theorem Excursion: Moreau envelope and proximal operator 1 Let f ∈ Γ0 and λ > 0. Then 1 e f := f# k · k2 λ 2λ is called the Moreau envelope of f. The map Pλf : E → E given by ( ) 1 2 Pλf(x) := argmin f(u) + kx − uk . u 2λ We have Pλf is 1-Lipschitz (in fact, firmly non-expansive) 1,1 eλf ∈ C ∩ Γ0 1 ∇eλf = λ (id − Pλf) x ∈ argmin f ⇐⇒ x ∈ argmin eλf ⇐⇒ x = Pλf(x) (→ proximal point/gradient method) eλf ↑ f (λ ↓ 0) (monotone pointwise convergence) 1Not in lecture notes! Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Infimal convolution and the Attouch-Brezis´ Theorem Excursion: Moreau envelope and proximal operator 1 Let f ∈ Γ0 and λ > 0. Then 1 e f := f# k · k2 λ 2λ is called the Moreau envelope of f. The map Pλf : E → E given by ( ) 1 2 Pλf(x) := argmin f(u) + kx − uk . u 2λ We have Pλf is 1-Lipschitz (in fact, firmly non-expansive) 1,1 eλf ∈ C ∩ Γ0 1 ∇eλf = λ (id − Pλf) x ∈ argmin f ⇐⇒ x ∈ argmin eλf ⇐⇒ x = Pλf(x) (→ proximal point/gradient method) eλf ↑ f (λ ↓ 0) (monotone pointwise convergence) epi eλf → epi f (λ ↓ 0) (epi-convergence) 1Not in lecture notes! Proposition 16. Let g : E → R be proper and L ∈ L(E, E0) and T ∈ L(E0, E). Then the following hold: a) (Lg)∗ = g∗ ◦ L ∗. b) (g ◦ T)∗ = cl (T ∗g∗) if g ∈ Γ. c) The closure in b) can be dropped and the infimum is attained when finite if g ∈ Γ0 and ri (rge T) ∩ ri (dom g) , ∅. (3) Proof. Notes and Part 2. Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Consequences of Attouch-Brezis´ Conjugacy for convex-linear composites Let f ∈ Γ and L ∈ L(E, E0). Then 0 n o Lf : E → R, (Lf)(y) := inf f(x) L(x) = y is convex2. 2 Show that epi Consequences of Attouch-Brezis´ Conjugacy for convex-linear composites Let f ∈ Γ and L ∈ L(E, E0). Then 0 n o Lf : E → R, (Lf)(y) := inf f(x) L(x) = y is convex2. Proposition 16. Let g : E → R be proper and L ∈ L(E, E0) and T ∈ L(E0, E). Then the following hold: a) (Lg)∗ = g∗ ◦ L ∗. b) (g ◦ T)∗ = cl (T ∗g∗) if g ∈ Γ. c) The closure in b) can be dropped and the infimum is attained when finite if g ∈ Γ0 and ri (rge T) ∩ ri (dom g) , ∅. (3) Proof. Notes and Part 2. 2 Show that epi Consequences of Attouch-Brezis´ Infimal projection revisited Theorem 17 (Infimal projection II). Let ψ ∈ Γ0(E1 × E2) and define p : E1 → R by p(x) := inf ψ(x, v). (4) v Then the following hold: a) p is convex. b) p∗ = ψ∗(·, 0) which is closed and convex. c) The condition dom ψ∗(·, 0) , 0 (5) ∗ is equivalent to having p ∈ Γ0. d) If (5) holds then p ∈ Γ0 and the infimum in its definition is attained when finite. Proof. Blackboard/Notes. Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals 2. Conjugacy of composite functions via K-convexity and inf-convolution Attach to E a largest element +∞• w.r.t. ≤K which satisfies x ≤K +∞• (x ∈ E). • Set E := E ∪ {+∞•}. • For F : E1 → E2 define n o dom F := x ∈ E1 F(x) ∈ E2 (domain), gph F := (x, F(x)) ∈ E1 × E2 | x ∈ dom F (graph), rge F := F(x) ∈ E2 | x ∈ dom F (range). Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals K-convexity Cone-induced ordering Given a cone K ⊂ E, the relation x ≤K y : ⇐⇒ y − x ∈ K (x, y ∈ E) induces an ordering on E which is a partial ordering if K is convex and pointed3. 3i.e. K ∩ (−K) = {0} • Set E := E ∪ {+∞•}. • For F : E1 → E2 define n o dom F := x ∈ E1 F(x) ∈ E2 (domain), gph F := (x, F(x)) ∈ E1 × E2 | x ∈ dom F (graph), rge F := F(x) ∈ E2 | x ∈ dom F (range). Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals K-convexity Cone-induced ordering Given a cone K ⊂ E, the relation x ≤K y : ⇐⇒ y − x ∈ K (x, y ∈ E) induces an ordering on E which is a partial ordering if K is convex and pointed3. Attach to E a largest element +∞• w.r.t. ≤K which satisfies x ≤K +∞• (x ∈ E). 3i.e. K ∩ (−K) = {0} Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals K-convexity Cone-induced ordering Given a cone K ⊂ E, the relation x ≤K y : ⇐⇒ y − x ∈ K (x, y ∈ E) induces an ordering on E which is a partial ordering if K is convex and pointed3. Attach to E a largest element +∞• w.r.t. ≤K which satisfies x ≤K +∞• (x ∈ E). • Set E := E ∪ {+∞•}. • For F : E1 → E2 define n o dom F := x ∈ E1 F(x) ∈ E2 (domain), gph F := (x, F(x)) ∈ E1 × E2 | x ∈ dom F (graph), rge F := F(x) ∈ E2 | x ∈ dom F (range). 3i.e. K ∩ (−K) = {0} F is K-convex ⇐⇒ F(λx + (1 − λ)y) ≤K λF(x) + (1 − λ)F(y)(x, y ∈ E1, λ ∈ [0, 1]) n o FK-convex, then ri (K-epi F) = (x, v) x ∈ ri (dom F), F(x) ri (K) v K ⊂ L cones: FK-convex ⇒ L-convex Examples: m n m • K = R+ and F : R → (R ) with Fi ∈ Γ(i = 1,..., m) K = (x, t) ∈ Rn × R | kxk ≤ t and F : Rn → Rn × R, F(x) = (x, kxk) ( −1 n n n • X , X 0, K = S+ and F : S → (S ) , F(X) = +∞•, else n m×n n T K = S+ and F : R → S , F(X) = XX K arbitrary, F affine. Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals K-convexity K-convexity Definition 18 (K-convexity). • Let K ⊂ E2 be a cone and F : E1 → E2. Then we call F K-convex if n o K-epi F := (x, v) ∈ E1 × E2 F(x) ≤K v (K-epigraph) is convex (in E1 × E2). n o FK-convex, then ri (K-epi F) = (x, v) x ∈ ri (dom F), F(x) ri (K) v K ⊂ L cones: FK-convex ⇒ L-convex Examples: m n m • K = R+ and F : R → (R ) with Fi ∈ Γ(i = 1,..., m) K = (x, t) ∈ Rn × R | kxk ≤ t and F : Rn → Rn × R, F(x) = (x, kxk) ( −1 n n n • X , X 0, K = S+ and F : S → (S ) , F(X) = +∞•, else n m×n n T K = S+ and F : R → S , F(X) = XX K arbitrary, F affine. Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals K-convexity K-convexity Definition 18 (K-convexity). • Let K ⊂ E2 be a cone and F : E1 → E2. Then we call F K-convex if n o K-epi F := (x, v) ∈ E1 × E2 F(x) ≤K v (K-epigraph) is convex (in E1 × E2). F is K-convex ⇐⇒ F(λx + (1 − λ)y) ≤K λF(x) + (1 − λ)F(y)(x, y ∈ E1, λ ∈ [0, 1]) K ⊂ L cones: FK-convex ⇒ L-convex Examples: m n m • K = R+ and F : R → (R ) with Fi ∈ Γ(i = 1,..., m) K = (x, t) ∈ Rn × R | kxk ≤ t and F : Rn → Rn × R, F(x) = (x, kxk) ( −1 n n n • X , X 0, K = S+ and F : S → (S ) , F(X) = +∞•, else n m×n n T K = S+ and F : R → S , F(X) = XX K arbitrary, F affine. Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals K-convexity K-convexity Definition 18 (K-convexity). • Let K ⊂ E2 be a cone and F : E1 → E2. Then we call F K-convex if n o K-epi F := (x, v) ∈ E1 × E2 F(x) ≤K v (K-epigraph) is convex (in E1 × E2). F is K-convex ⇐⇒ F(λx + (1 − λ)y) ≤K λF(x) + (1 − λ)F(y)(x, y ∈ E1, λ ∈ [0, 1]) n o FK-convex, then ri (K-epi F) = (x, v) x ∈ ri (dom F), F(x) ri (K) v Examples: m n m • K = R+ and F : R → (R ) with Fi ∈ Γ(i = 1,..., m) K = (x, t) ∈ Rn × R | kxk ≤ t and F : Rn → Rn × R, F(x) = (x, kxk) ( −1 n n n • X , X 0, K = S+ and F : S → (S ) , F(X) = +∞•, else n m×n n T K = S+ and F : R → S , F(X) = XX K arbitrary, F affine. Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals K-convexity K-convexity Definition 18 (K-convexity). • Let K ⊂ E2 be a cone and F : E1 → E2. Then we call F K-convex if n o K-epi F := (x, v) ∈ E1 × E2 F(x) ≤K v (K-epigraph) is convex (in E1 × E2). F is K-convex ⇐⇒ F(λx + (1 − λ)y) ≤K λF(x) + (1 − λ)F(y)(x, y ∈ E1, λ ∈ [0, 1]) n o FK-convex, then ri (K-epi F) = (x, v) x ∈ ri (dom F), F(x) ri (K) v K ⊂ L cones: FK-convex ⇒ L-convex K = (x, t) ∈ Rn × R | kxk ≤ t and F : Rn → Rn × R, F(x) = (x, kxk) ( −1 n n n • X , X 0, K = S+ and F : S → (S ) , F(X) = +∞•, else n m×n n T K = S+ and F : R → S , F(X) = XX K arbitrary, F affine. Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals K-convexity K-convexity Definition 18 (K-convexity). • Let K ⊂ E2 be a cone and F : E1 → E2. Then we call F K-convex if n o K-epi F := (x, v) ∈ E1 × E2 F(x) ≤K v (K-epigraph) is convex (in E1 × E2). F is K-convex ⇐⇒ F(λx + (1 − λ)y) ≤K λF(x) + (1 − λ)F(y)(x, y ∈ E1, λ ∈ [0, 1]) n o FK-convex, then ri (K-epi F) = (x, v) x ∈ ri (dom F), F(x) ri (K) v K ⊂ L cones: FK-convex ⇒ L-convex Examples: m n m • K = R+ and F : R → (R ) with Fi ∈ Γ(i = 1,..., m) ( −1 n n n • X , X 0, K = S+ and F : S → (S ) , F(X) = +∞•, else n m×n n T K = S+ and F : R → S , F(X) = XX K arbitrary, F affine. Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals K-convexity K-convexity Definition 18 (K-convexity). • Let K ⊂ E2 be a cone and F : E1 → E2. Then we call F K-convex if n o K-epi F := (x, v) ∈ E1 × E2 F(x) ≤K v (K-epigraph) is convex (in E1 × E2). F is K-convex ⇐⇒ F(λx + (1 − λ)y) ≤K λF(x) + (1 − λ)F(y)(x, y ∈ E1, λ ∈ [0, 1]) n o FK-convex, then ri (K-epi F) = (x, v) x ∈ ri (dom F), F(x) ri (K) v K ⊂ L cones: FK-convex ⇒ L-convex Examples: m n m • K = R+ and F : R → (R ) with Fi ∈ Γ(i = 1,..., m) K = (x, t) ∈ Rn × R | kxk ≤ t and F : Rn → Rn × R, F(x) = (x, kxk) n m×n n T K = S+ and F : R → S , F(X) = XX K arbitrary, F affine. Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals K-convexity K-convexity Definition 18 (K-convexity). • Let K ⊂ E2 be a cone and F : E1 → E2. Then we call F K-convex if n o K-epi F := (x, v) ∈ E1 × E2 F(x) ≤K v (K-epigraph) is convex (in E1 × E2). F is K-convex ⇐⇒ F(λx + (1 − λ)y) ≤K λF(x) + (1 − λ)F(y)(x, y ∈ E1, λ ∈ [0, 1]) n o FK-convex, then ri (K-epi F) = (x, v) x ∈ ri (dom F), F(x) ri (K) v K ⊂ L cones: FK-convex ⇒ L-convex Examples: m n m • K = R+ and F : R → (R ) with Fi ∈ Γ(i = 1,..., m) K = (x, t) ∈ Rn × R | kxk ≤ t and F : Rn → Rn × R, F(x) = (x, kxk) ( −1 n n n • X , X 0, K = S+ and F : S → (S ) , F(X) = +∞•, else K arbitrary, F affine. Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals K-convexity K-convexity Definition 18 (K-convexity). • Let K ⊂ E2 be a cone and F : E1 → E2. Then we call F K-convex if n o K-epi F := (x, v) ∈ E1 × E2 F(x) ≤K v (K-epigraph) is convex (in E1 × E2). F is K-convex ⇐⇒ F(λx + (1 − λ)y) ≤K λF(x) + (1 − λ)F(y)(x, y ∈ E1, λ ∈ [0, 1]) n o FK-convex, then ri (K-epi F) = (x, v) x ∈ ri (dom F), F(x) ri (K) v K ⊂ L cones: FK-convex ⇒ L-convex Examples: m n m • K = R+ and F : R → (R ) with Fi ∈ Γ(i = 1,..., m) K = (x, t) ∈ Rn × R | kxk ≤ t and F : Rn → Rn × R, F(x) = (x, kxk) ( −1 n n n • X , X 0, K = S+ and F : S → (S ) , F(X) = +∞•, else n m×n n T K = S+ and F : R → S , F(X) = XX Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals K-convexity K-convexity Definition 18 (K-convexity). • Let K ⊂ E2 be a cone and F : E1 → E2. Then we call F K-convex if n o K-epi F := (x, v) ∈ E1 × E2 F(x) ≤K v (K-epigraph) is convex (in E1 × E2). F is K-convex ⇐⇒ F(λx + (1 − λ)y) ≤K λF(x) + (1 − λ)F(y)(x, y ∈ E1, λ ∈ [0, 1]) n o FK-convex, then ri (K-epi F) = (x, v) x ∈ ri (dom F), F(x) ri (K) v K ⊂ L cones: FK-convex ⇒ L-convex Examples: m n m • K = R+ and F : R → (R ) with Fi ∈ Γ(i = 1,..., m) K = (x, t) ∈ Rn × R | kxk ≤ t and F : Rn → Rn × R, F(x) = (x, kxk) ( −1 n n n • X , X 0, K = S+ and F : S → (S ) , F(X) = +∞•, else n m×n n T K = S+ and F : R → S , F(X) = XX K arbitrary, F affine. Proposition 19. • Let K ⊂ E2 be a convex cone, F : E1 → E2 K-convex and g ∈ Γ(E2) such that rge F ∩ dom g , ∅. If g(F(x)) ≤ g(y) ((x, y) ∈ K-epi F) (6) then the following hold: a) g ◦ F is convex and proper. b) If g is lsc and F is continuous then g ◦ F is lower semicontinuous. Condition (6) holds if g is K-increasing, i.e. x ≤K y =⇒ g(x) ≤ g(y). Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Composite functions and scalarization Convexity of composite functions • For F : E1 → E2 and g : E2 → R ∪ {+∞} we define ( g(F(x)) if x ∈ dom F, (g ◦ F)(x) := +∞ else. Condition (6) holds if g is K-increasing, i.e. x ≤K y =⇒ g(x) ≤ g(y). Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Composite functions and scalarization Convexity of composite functions • For F : E1 → E2 and g : E2 → R ∪ {+∞} we define ( g(F(x)) if x ∈ dom F, (g ◦ F)(x) := +∞ else. Proposition 19. • Let K ⊂ E2 be a convex cone, F : E1 → E2 K-convex and g ∈ Γ(E2) such that rge F ∩ dom g , ∅. If g(F(x)) ≤ g(y) ((x, y) ∈ K-epi F) (6) then the following hold: a) g ◦ F is convex and proper. b) If g is lsc and F is continuous then g ◦ F is lower semicontinuous. Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Composite functions and scalarization Convexity of composite functions • For F : E1 → E2 and g : E2 → R ∪ {+∞} we define ( g(F(x)) if x ∈ dom F, (g ◦ F)(x) := +∞ else. Proposition 19. • Let K ⊂ E2 be a convex cone, F : E1 → E2 K-convex and g ∈ Γ(E2) such that rge F ∩ dom g , ∅. If g(F(x)) ≤ g(y) ((x, y) ∈ K-epi F) (6) then the following hold: a) g ◦ F is convex and proper. b) If g is lsc and F is continuous then g ◦ F is lower semicontinuous. Condition (6) holds if g is K-increasing, i.e. x ≤K y =⇒ g(x) ≤ g(y). For K a closed, convex cone we have: F is K-convex ⇐⇒ hv, Fi is convex (v ∈ −K ◦) ∗ σgph F (u, −v) = hv, Fi (u). σK-epi F (u, v) = σgph F (u, v) + δK◦ (v) Lemma 20 (Pennanen, JCA 1999). • Let f : E1 → E2 with a convex domain and let K ⊂ E2 be the smallest closed convex cone with respect to which F is convex. Then ◦ (−K) = {v ∈ E2 | hv, Fi is convex } . Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Composite functions and scalarization Scalarization Given v ∈ E2 and the linear form hv, ·i : E2 → R, we set hv, Fi := hv, ·i ◦ F, i.e. ( v, F(x) if x ∈ dom F, hv, Fi (x) = +∞ else. Lemma 20 (Pennanen, JCA 1999). • Let f : E1 → E2 with a convex domain and let K ⊂ E2 be the smallest closed convex cone with respect to which F is convex. Then ◦ (−K) = {v ∈ E2 | hv, Fi is convex } . Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Composite functions and scalarization Scalarization Given v ∈ E2 and the linear form hv, ·i : E2 → R, we set hv, Fi := hv, ·i ◦ F, i.e. ( v, F(x) if x ∈ dom F, hv, Fi (x) = +∞ else. For K a closed, convex cone we have: F is K-convex ⇐⇒ hv, Fi is convex (v ∈ −K ◦) ∗ σgph F (u, −v) = hv, Fi (u). σK-epi F (u, v) = σgph F (u, v) + δK◦ (v) Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Composite functions and scalarization Scalarization Given v ∈ E2 and the linear form hv, ·i : E2 → R, we set hv, Fi := hv, ·i ◦ F, i.e. ( v, F(x) if x ∈ dom F, hv, Fi (x) = +∞ else. For K a closed, convex cone we have: F is K-convex ⇐⇒ hv, Fi is convex (v ∈ −K ◦) ∗ σgph F (u, −v) = hv, Fi (u). σK-epi F (u, v) = σgph F (u, v) + δK◦ (v) Lemma 20 (Pennanen, JCA 1999). • Let f : E1 → E2 with a convex domain and let K ⊂ E2 be the smallest closed convex cone with respect to which F is convex. Then ◦ (−K) = {v ∈ E2 | hv, Fi is convex } . Remark: The CQ (7) is trivially satisfied if g is finite-valued. Condition (6) can be replaced by the stronger condition that g be K-increasing. K-epi F is closed if F is continuous. Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Conjugacy results The main result Theorem 21 (Conjugacy for composite function, H./Nguyen ’19, Bot et. al ’11). • Let K ⊂ E2 be a closed convex cone, F : E1 → E2 K-convex such that K-epi F is closed and g0 ∈ Γ(E2) such that (6) is satisfied, i.e. x ≤K y =⇒ g(x) ≤ g(y). Under the CQ F(ri (dom F)) ∩ ri (dom g − K) , ∅ (7) we have (g ◦ F)∗(p) = min g∗(v) + hv, Fi∗ (p) v∈−K◦ ∗ n ∗ ◦ ∗ o with dom (g ◦ F) = p ∈ E1 ∃v ∈ dom g ∩ (−K ): hv, Fi (p) < +∞ . Proof. Blackboard/Notes. Condition (6) can be replaced by the stronger condition that g be K-increasing. K-epi F is closed if F is continuous. Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Conjugacy results The main result Theorem 21 (Conjugacy for composite function, H./Nguyen ’19, Bot et. al ’11). • Let K ⊂ E2 be a closed convex cone, F : E1 → E2 K-convex such that K-epi F is closed and g0 ∈ Γ(E2) such that (6) is satisfied, i.e. x ≤K y =⇒ g(x) ≤ g(y). Under the CQ F(ri (dom F)) ∩ ri (dom g − K) , ∅ (7) we have (g ◦ F)∗(p) = min g∗(v) + hv, Fi∗ (p) v∈−K◦ ∗ n ∗ ◦ ∗ o with dom (g ◦ F) = p ∈ E1 ∃v ∈ dom g ∩ (−K ): hv, Fi (p) < +∞ . Proof. Blackboard/Notes. Remark: The CQ (7) is trivially satisfied if g is finite-valued. K-epi F is closed if F is continuous. Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Conjugacy results The main result Theorem 21 (Conjugacy for composite function, H./Nguyen ’19, Bot et. al ’11). • Let K ⊂ E2 be a closed convex cone, F : E1 → E2 K-convex such that K-epi F is closed and g0 ∈ Γ(E2) such that (6) is satisfied, i.e. x ≤K y =⇒ g(x) ≤ g(y). Under the CQ F(ri (dom F)) ∩ ri (dom g − K) , ∅ (7) we have (g ◦ F)∗(p) = min g∗(v) + hv, Fi∗ (p) v∈−K◦ ∗ n ∗ ◦ ∗ o with dom (g ◦ F) = p ∈ E1 ∃v ∈ dom g ∩ (−K ): hv, Fi (p) < +∞ . Proof. Blackboard/Notes. Remark: The CQ (7) is trivially satisfied if g is finite-valued. Condition (6) can be replaced by the stronger condition that g be K-increasing. Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Conjugacy results The main result Theorem 21 (Conjugacy for composite function, H./Nguyen ’19, Bot et. al ’11). • Let K ⊂ E2 be a closed convex cone, F : E1 → E2 K-convex such that K-epi F is closed and g0 ∈ Γ(E2) such that (6) is satisfied, i.e. x ≤K y =⇒ g(x) ≤ g(y). Under the CQ F(ri (dom F)) ∩ ri (dom g − K) , ∅ (7) we have (g ◦ F)∗(p) = min g∗(v) + hv, Fi∗ (p) v∈−K◦ ∗ n ∗ ◦ ∗ o with dom (g ◦ F) = p ∈ E1 ∃v ∈ dom g ∩ (−K ): hv, Fi (p) < +∞ . Proof. Blackboard/Notes. Remark: The CQ (7) is trivially satisfied if g is finite-valued. Condition (6) can be replaced by the stronger condition that g be K-increasing. K-epi F is closed if F is continuous. Proof. (Sketch) Apply Theorem 21 to g˜ :(s, y) ∈ R × E2 7→ s + g(y), F˜ : x ∈ E1 → (f(x), x) and K˜ := R+ × K. Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Conjugacy results Extension to the additive composite setting Corollary 22 (Conjugate of additive composite functions, H./Nguyen ’19). Under the assumptions of Theorem 21 let f ∈ Γ0 such that F(ri (dom f ∩ dom F)) ∩ ri (dom g − K) , ∅. (8) Then (f + g ◦ F)∗(p) = min g∗(v) + f ∗(y) + hv, Fi∗ (p − y). v∈−K◦, y∈E1 Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Conjugacy results Extension to the additive composite setting Corollary 22 (Conjugate of additive composite functions, H./Nguyen ’19). Under the assumptions of Theorem 21 let f ∈ Γ0 such that F(ri (dom f ∩ dom F)) ∩ ri (dom g − K) , ∅. (8) Then (f + g ◦ F)∗(p) = min g∗(v) + f ∗(y) + hv, Fi∗ (p − y). v∈−K◦, y∈E1 Proof. (Sketch) Apply Theorem 21 to g˜ :(s, y) ∈ R × E2 7→ s + g(y), F˜ : x ∈ E1 → (f(x), x) and K˜ := R+ × K. hzn g = (cone (dom g∗))◦ g is K-increasing for K = −hzn g: Let x K y, i.e. y = x + b for some b ∈ K. Then g(x) = sup {hx, zi − g∗(z)} = sup {hy, zi − hb, zi − g∗(z)} ≤ sup {hy, zi − g∗(z)} = g(y), z∈dom g∗ z∈dom g∗ z∈dom g∗ Corollary 23 (Burke ’91, H./Nguyen ’19). • Let g ∈ Γ0(E2) and let F : E1 → E2 be (−hzn g)-convex with −hzn g-epi F closed such that F(ri (dom F)) ∩ ri (dom g + hzn g) , ∅. Then (g ◦ F)∗(p) = min g∗(v) + hv, Fi∗ (p). v∈E2 Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Conjugacy results The case K = −hzn g ∞ For g ∈ Γ0 its horizon function g is given via epi g∞ = (epi g)∞. The horizon cone of g is n ∞ o∞ hzn g := x g (x) ≤ 0 . g is K-increasing for K = −hzn g: Let x K y, i.e. y = x + b for some b ∈ K. Then g(x) = sup {hx, zi − g∗(z)} = sup {hy, zi − hb, zi − g∗(z)} ≤ sup {hy, zi − g∗(z)} = g(y), z∈dom g∗ z∈dom g∗ z∈dom g∗ Corollary 23 (Burke ’91, H./Nguyen ’19). • Let g ∈ Γ0(E2) and let F : E1 → E2 be (−hzn g)-convex with −hzn g-epi F closed such that F(ri (dom F)) ∩ ri (dom g + hzn g) , ∅. Then (g ◦ F)∗(p) = min g∗(v) + hv, Fi∗ (p). v∈E2 Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Conjugacy results The case K = −hzn g ∞ For g ∈ Γ0 its horizon function g is given via epi g∞ = (epi g)∞. The horizon cone of g is n ∞ o∞ hzn g := x g (x) ≤ 0 . hzn g = (cone (dom g∗))◦ Let x K y, i.e. y = x + b for some b ∈ K. Then g(x) = sup {hx, zi − g∗(z)} = sup {hy, zi − hb, zi − g∗(z)} ≤ sup {hy, zi − g∗(z)} = g(y), z∈dom g∗ z∈dom g∗ z∈dom g∗ Corollary 23 (Burke ’91, H./Nguyen ’19). • Let g ∈ Γ0(E2) and let F : E1 → E2 be (−hzn g)-convex with −hzn g-epi F closed such that F(ri (dom F)) ∩ ri (dom g + hzn g) , ∅. Then (g ◦ F)∗(p) = min g∗(v) + hv, Fi∗ (p). v∈E2 Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Conjugacy results The case K = −hzn g ∞ For g ∈ Γ0 its horizon function g is given via epi g∞ = (epi g)∞. The horizon cone of g is n ∞ o∞ hzn g := x g (x) ≤ 0 . hzn g = (cone (dom g∗))◦ g is K-increasing for K = −hzn g: Corollary 23 (Burke ’91, H./Nguyen ’19). • Let g ∈ Γ0(E2) and let F : E1 → E2 be (−hzn g)-convex with −hzn g-epi F closed such that F(ri (dom F)) ∩ ri (dom g + hzn g) , ∅. Then (g ◦ F)∗(p) = min g∗(v) + hv, Fi∗ (p). v∈E2 Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Conjugacy results The case K = −hzn g ∞ For g ∈ Γ0 its horizon function g is given via epi g∞ = (epi g)∞. The horizon cone of g is n ∞ o∞ hzn g := x g (x) ≤ 0 . hzn g = (cone (dom g∗))◦ g is K-increasing for K = −hzn g: Let x K y, i.e. y = x + b for some b ∈ K. Then g(x) = sup {hx, zi − g∗(z)} = sup {hy, zi − hb, zi − g∗(z)} ≤ sup {hy, zi − g∗(z)} = g(y), z∈dom g∗ z∈dom g∗ z∈dom g∗ Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Conjugacy results The case K = −hzn g ∞ For g ∈ Γ0 its horizon function g is given via epi g∞ = (epi g)∞. The horizon cone of g is n ∞ o∞ hzn g := x g (x) ≤ 0 . hzn g = (cone (dom g∗))◦ g is K-increasing for K = −hzn g: Let x K y, i.e. y = x + b for some b ∈ K. Then g(x) = sup {hx, zi − g∗(z)} = sup {hy, zi − hb, zi − g∗(z)} ≤ sup {hy, zi − g∗(z)} = g(y), z∈dom g∗ z∈dom g∗ z∈dom g∗ Corollary 23 (Burke ’91, H./Nguyen ’19). • Let g ∈ Γ0(E2) and let F : E1 → E2 be (−hzn g)-convex with −hzn g-epi F closed such that F(ri (dom F)) ∩ ri (dom g + hzn g) , ∅. Then (g ◦ F)∗(p) = min g∗(v) + hv, Fi∗ (p). v∈E2 Proof. We notice that F is {0}-convex. Hence we can apply Theorem 21 with K = {0}. Condition (7) then reads rge F ∩ ri (dom g) , ∅, which is our assumption. Hence we obtain ∗ ∗ ∗ ∗ (g ◦ F) (p) = min g (v) + hv, Fi (p) = min g (v) + δ ∗ (p). ◦ {F (v)} v∈−K v∈E2 Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Conjugacy results The linear case Corollary 24 (The linear case). Let g ∈ Γ(E2) and F : E1 → E2 linear such that rge F ∩ ri (dom g) , ∅. Then ∗ n ∗ ∗ o (g ◦ F) (p) = min g (v) F (v) = p v∈E2 with dom (g ◦ F) = (F∗)−1(dom g∗). Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Conjugacy results The linear case Corollary 24 (The linear case). Let g ∈ Γ(E2) and F : E1 → E2 linear such that rge F ∩ ri (dom g) , ∅. Then ∗ n ∗ ∗ o (g ◦ F) (p) = min g (v) F (v) = p v∈E2 with dom (g ◦ F) = (F∗)−1(dom g∗). Proof. We notice that F is {0}-convex. Hence we can apply Theorem 21 with K = {0}. Condition (7) then reads rge F ∩ ri (dom g) , ∅, which is our assumption. Hence we obtain ∗ ∗ ∗ ∗ (g ◦ F) (p) = min g (v) + hv, Fi (p) = min g (v) + δ ∗ (p). ◦ {F (v)} v∈−K v∈E2 min f(x) + (δ−K ◦ F)(x) (10) x∈E1 where f : E1 → R is convex, F : E1 → E2 is K-convex and K ⊂ E2 is a closed, convex cone. The qualification condition (7) turns into a generalized Slater condition rge F ∩ ri (−K) , ∅. (11) Theorem 25 (Strong duality and dual attainment for conic programming). Let f : E1 → R is convex, K ⊂ E2 a closed, convex cone, and let F : E1 → E2 be K-convex with closed K-epigraph. If (11) holds then inf f(x) + (δ ◦ F)(x) = max −f ∗(y) − (δ ◦ F)∗(−y) = max inf f(x) + v, F(x) . −K ◦ −K ◦ x∈E1 v∈−K v∈−K x∈E1 Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Applications Conic programming duality Consider the general conic program min f(x) s.t. F(x) ∈ −K (9) or equivalently The qualification condition (7) turns into a generalized Slater condition rge F ∩ ri (−K) , ∅. (11) Theorem 25 (Strong duality and dual attainment for conic programming). Let f : E1 → R is convex, K ⊂ E2 a closed, convex cone, and let F : E1 → E2 be K-convex with closed K-epigraph. If (11) holds then inf f(x) + (δ ◦ F)(x) = max −f ∗(y) − (δ ◦ F)∗(−y) = max inf f(x) + v, F(x) . −K ◦ −K ◦ x∈E1 v∈−K v∈−K x∈E1 Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Applications Conic programming duality Consider the general conic program min f(x) s.t. F(x) ∈ −K (9) or equivalently min f(x) + (δ−K ◦ F)(x) (10) x∈E1 where f : E1 → R is convex, F : E1 → E2 is K-convex and K ⊂ E2 is a closed, convex cone. Theorem 25 (Strong duality and dual attainment for conic programming). Let f : E1 → R is convex, K ⊂ E2 a closed, convex cone, and let F : E1 → E2 be K-convex with closed K-epigraph. If (11) holds then inf f(x) + (δ ◦ F)(x) = max −f ∗(y) − (δ ◦ F)∗(−y) = max inf f(x) + v, F(x) . −K ◦ −K ◦ x∈E1 v∈−K v∈−K x∈E1 Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Applications Conic programming duality Consider the general conic program min f(x) s.t. F(x) ∈ −K (9) or equivalently min f(x) + (δ−K ◦ F)(x) (10) x∈E1 where f : E1 → R is convex, F : E1 → E2 is K-convex and K ⊂ E2 is a closed, convex cone. The qualification condition (7) turns into a generalized Slater condition rge F ∩ ri (−K) , ∅. (11) Theorem 25 (Strong duality and dual attainment for conic programming). Let f : E1 → R is convex, K ⊂ E2 a closed, convex cone, and let F : E1 → E2 be K-convex with closed K-epigraph. If (11) holds then inf f(x) + (δ ◦ F)(x) = max −f ∗(y) − (δ ◦ F)∗(−y) = max inf f(x) + v, F(x) . −K ◦ −K ◦ x∈E1 v∈−K v∈−K x∈E1 Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Applications Conic programming duality Consider the general conic program min f(x) s.t. F(x) ∈ −K (9) or equivalently min f(x) + (δ−K ◦ F)(x) (10) x∈E1 where f : E1 → R is convex, F : E1 → E2 is K-convex and K ⊂ E2 is a closed, convex cone. The qualification condition (7) turns into a generalized Slater condition rge F ∩ ri (−K) , ∅. (11) Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Applications Conic programming duality Consider the general conic program min f(x) s.t. F(x) ∈ −K (9) or equivalently min f(x) + (δ−K ◦ F)(x) (10) x∈E1 where f : E1 → R is convex, F : E1 → E2 is K-convex and K ⊂ E2 is a closed, convex cone. The qualification condition (7) turns into a generalized Slater condition rge F ∩ ri (−K) , ∅. (11) Theorem 25 (Strong duality and dual attainment for conic programming). Let f : E1 → R is convex, K ⊂ E2 a closed, convex cone, and let F : E1 → E2 be K-convex with closed K-epigraph. If (11) holds then inf f(x) + (δ ◦ F)(x) = max −f ∗(y) − (δ ◦ F)∗(−y) = max inf f(x) + v, F(x) . −K ◦ −K ◦ x∈E1 v∈−K v∈−K x∈E1 Proof. We have f = g ◦ F for Tm (f1(x),..., fm (x)) if x ∈ i=1 dom fi , F : x 7→ and g : y 7→ max xi . +∞• otherwise, i=1,...,m m m m ∗ 4 Then F is R+-convex and g is R+-increasing with dom g = R , and g = δ∆m . Hence m !∗ ∗ ∗ ∗ ∗ X (g ◦ F) (x) = min g (v) + hv, Fi (x) = min δ∆m (v) + hv, Fi (x) = min vi fi (x) v∈Rm v∈Rm v∈∆m + + i=1 Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Applications Conjugate of pointwise maximum of convex functions Proposition 26. For f1,..., fm ∈ Γ0(E) define f := maxi=1,...,m fi . Then f ∈ Γ0(E) with m !∗ ∗ X f (x) = min vi fi (x). v∈∆m i=1 n o 4∆ = ∈ Rm Pm = ≥ ( ) m λ i=1 λi 1, λi 0 i1,..., m m m m ∗ 4 Then F is R+-convex and g is R+-increasing with dom g = R , and g = δ∆m . Hence m !∗ ∗ ∗ ∗ ∗ X (g ◦ F) (x) = min g (v) + hv, Fi (x) = min δ∆m (v) + hv, Fi (x) = min vi fi (x) v∈Rm v∈Rm v∈∆m + + i=1 Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Applications Conjugate of pointwise maximum of convex functions Proposition 26. For f1,..., fm ∈ Γ0(E) define f := maxi=1,...,m fi . Then f ∈ Γ0(E) with m !∗ ∗ X f (x) = min vi fi (x). v∈∆m i=1 Proof. We have f = g ◦ F for Tm (f1(x),..., fm (x)) if x ∈ i=1 dom fi , F : x 7→ and g : y 7→ max xi . +∞• otherwise, i=1,...,m n o 4∆ = ∈ Rm Pm = ≥ ( ) m λ i=1 λi 1, λi 0 i1,..., m Hence m !∗ ∗ ∗ ∗ ∗ X (g ◦ F) (x) = min g (v) + hv, Fi (x) = min δ∆m (v) + hv, Fi (x) = min vi fi (x) v∈Rm v∈Rm v∈∆m + + i=1 Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Applications Conjugate of pointwise maximum of convex functions Proposition 26. For f1,..., fm ∈ Γ0(E) define f := maxi=1,...,m fi . Then f ∈ Γ0(E) with m !∗ ∗ X f (x) = min vi fi (x). v∈∆m i=1 Proof. We have f = g ◦ F for Tm (f1(x),..., fm (x)) if x ∈ i=1 dom fi , F : x 7→ and g : y 7→ max xi . +∞• otherwise, i=1,...,m m m m ∗ 4 Then F is R+-convex and g is R+-increasing with dom g = R , and g = δ∆m . n o 4∆ = ∈ Rm Pm = ≥ ( ) m λ i=1 λi 1, λi 0 i1,..., m m !∗ ∗ ∗ ∗ X = min g (v) + hv, Fi (x) = min δ∆m (v) + hv, Fi (x) = min vi fi (x) v∈Rm v∈Rm v∈∆m + + i=1 Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Applications Conjugate of pointwise maximum of convex functions Proposition 26. For f1,..., fm ∈ Γ0(E) define f := maxi=1,...,m fi . Then f ∈ Γ0(E) with m !∗ ∗ X f (x) = min vi fi (x). v∈∆m i=1 Proof. We have f = g ◦ F for Tm (f1(x),..., fm (x)) if x ∈ i=1 dom fi , F : x 7→ and g : y 7→ max xi . +∞• otherwise, i=1,...,m m m m ∗ 4 Then F is R+-convex and g is R+-increasing with dom g = R , and g = δ∆m . Hence (g ◦ F)∗(x) n o 4∆ = ∈ Rm Pm = ≥ ( ) m λ i=1 λi 1, λi 0 i1,..., m m !∗ ∗ X = min δ∆m (v) + hv, Fi (x) = min vi fi (x) v∈Rm v∈∆m + i=1 Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Applications Conjugate of pointwise maximum of convex functions Proposition 26. For f1,..., fm ∈ Γ0(E) define f := maxi=1,...,m fi . Then f ∈ Γ0(E) with m !∗ ∗ X f (x) = min vi fi (x). v∈∆m i=1 Proof. We have f = g ◦ F for Tm (f1(x),..., fm (x)) if x ∈ i=1 dom fi , F : x 7→ and g : y 7→ max xi . +∞• otherwise, i=1,...,m m m m ∗ 4 Then F is R+-convex and g is R+-increasing with dom g = R , and g = δ∆m . Hence (g ◦ F)∗(x) = min g∗(v) + hv, Fi∗ (x) ∈ m v R+ n o 4∆ = ∈ Rm Pm = ≥ ( ) m λ i=1 λi 1, λi 0 i1,..., m Xm !∗ = min vi fi (x) v∈∆m i=1 Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Applications Conjugate of pointwise maximum of convex functions Proposition 26. For f1,..., fm ∈ Γ0(E) define f := maxi=1,...,m fi . Then f ∈ Γ0(E) with m !∗ ∗ X f (x) = min vi fi (x). v∈∆m i=1 Proof. We have f = g ◦ F for Tm (f1(x),..., fm (x)) if x ∈ i=1 dom fi , F : x 7→ and g : y 7→ max xi . +∞• otherwise, i=1,...,m m m m ∗ 4 Then F is R+-convex and g is R+-increasing with dom g = R , and g = δ∆m . Hence ∗ ∗ ∗ ∗ (g ◦ F) (x) = min g (v) + hv, Fi (x) = min δ∆m (v) + hv, Fi (x) ∈ m ∈ m v R+ v R+ n o 4∆ = ∈ Rm Pm = ≥ ( ) m λ i=1 λi 1, λi 0 i1,..., m Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Applications Conjugate of pointwise maximum of convex functions Proposition 26. For f1,..., fm ∈ Γ0(E) define f := maxi=1,...,m fi . Then f ∈ Γ0(E) with m !∗ ∗ X f (x) = min vi fi (x). v∈∆m i=1 Proof. We have f = g ◦ F for Tm (f1(x),..., fm (x)) if x ∈ i=1 dom fi , F : x 7→ and g : y 7→ max xi . +∞• otherwise, i=1,...,m m m m ∗ 4 Then F is R+-convex and g is R+-increasing with dom g = R , and g = δ∆m . Hence m !∗ ∗ ∗ ∗ ∗ X (g ◦ F) (x) = min g (v) + hv, Fi (x) = min δ∆m (v) + hv, Fi (x) = min vi fi (x) v∈Rm v∈Rm v∈∆m + + i=1 n o 4∆ = ∈ Rm Pm = ≥ ( ) m λ i=1 λi 1, λi 0 i1,..., m Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals 3. A new class of matrix support functionals Approximating the rank function (→ combinatorial) Convex approx. 5 rank X = kσ(X)k0 ∼ kσ(X)k1 =: kXk∗ (nuclear norm) Convex approximation of (12) min kXk∗ s.t. MX = B X∈Rn×m 1 1 T −1 n×m Hsieh/Olsen ’14: kXk∗ = min ∈ n tr (V) + tr (X V X)( X ∈ R ) V S++ 2 2 Smooth approximation of (12) 1 1 min tr (V) + tr (X T V −1X) s.t. MX = B ∈ n×n × n 2 2 (X,V) R S++ Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals The generalized matrix-fractional function Motivation I: Nuclear norm minimization/smoothing Rank minimization (→ Netflix recommender problem) min rank X s.t. MX = B (M ∈ Rp×n , B ∈ Rp×m ) (12) X∈Rn×m 5 σ(X) = (σ1, . . . , σn ) is the vector of positive singular values of X. Convex approximation of (12) min kXk∗ s.t. MX = B X∈Rn×m 1 1 T −1 n×m Hsieh/Olsen ’14: kXk∗ = min ∈ n tr (V) + tr (X V X)( X ∈ R ) V S++ 2 2 Smooth approximation of (12) 1 1 min tr (V) + tr (X T V −1X) s.t. MX = B ∈ n×n × n 2 2 (X,V) R S++ Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals The generalized matrix-fractional function Motivation I: Nuclear norm minimization/smoothing Rank minimization (→ Netflix recommender problem) min rank X s.t. MX = B (M ∈ Rp×n , B ∈ Rp×m ) (12) X∈Rn×m Approximating the rank function (→ combinatorial) Convex approx. 5 rank X = kσ(X)k0 ∼ kσ(X)k1 =: kXk∗ (nuclear norm) 5 σ(X) = (σ1, . . . , σn ) is the vector of positive singular values of X. 1 1 T −1 n×m Hsieh/Olsen ’14: kXk∗ = min ∈ n tr (V) + tr (X V X)( X ∈ R ) V S++ 2 2 Smooth approximation of (12) 1 1 min tr (V) + tr (X T V −1X) s.t. MX = B ∈ n×n × n 2 2 (X,V) R S++ Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals The generalized matrix-fractional function Motivation I: Nuclear norm minimization/smoothing Rank minimization (→ Netflix recommender problem) min rank X s.t. MX = B (M ∈ Rp×n , B ∈ Rp×m ) (12) X∈Rn×m Approximating the rank function (→ combinatorial) Convex approx. 5 rank X = kσ(X)k0 ∼ kσ(X)k1 =: kXk∗ (nuclear norm) Convex approximation of (12) min kXk∗ s.t. MX = B X∈Rn×m 5 σ(X) = (σ1, . . . , σn ) is the vector of positive singular values of X. Smooth approximation of (12) 1 1 min tr (V) + tr (X T V −1X) s.t. MX = B ∈ n×n × n 2 2 (X,V) R S++ Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals The generalized matrix-fractional function Motivation I: Nuclear norm minimization/smoothing Rank minimization (→ Netflix recommender problem) min rank X s.t. MX = B (M ∈ Rp×n , B ∈ Rp×m ) (12) X∈Rn×m Approximating the rank function (→ combinatorial) Convex approx. 5 rank X = kσ(X)k0 ∼ kσ(X)k1 =: kXk∗ (nuclear norm) Convex approximation of (12) min kXk∗ s.t. MX = B X∈Rn×m 1 1 T −1 n×m Hsieh/Olsen ’14: kXk∗ = min ∈ n tr (V) + tr (X V X)( X ∈ R ) V S++ 2 2 5 σ(X) = (σ1, . . . , σn ) is the vector of positive singular values of X. Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals The generalized matrix-fractional function Motivation I: Nuclear norm minimization/smoothing Rank minimization (→ Netflix recommender problem) min rank X s.t. MX = B (M ∈ Rp×n , B ∈ Rp×m ) (12) X∈Rn×m Approximating the rank function (→ combinatorial) Convex approx. 5 rank X = kσ(X)k0 ∼ kσ(X)k1 =: kXk∗ (nuclear norm) Convex approximation of (12) min kXk∗ s.t. MX = B X∈Rn×m 1 1 T −1 n×m Hsieh/Olsen ’14: kXk∗ = min ∈ n tr (V) + tr (X V X)( X ∈ R ) V S++ 2 2 Smooth approximation of (12) 1 1 min tr (V) + tr (X T V −1X) s.t. MX = B ∈ n×n × n 2 2 (X,V) R S++ 5 σ(X) = (σ1, . . . , σn ) is the vector of positive singular values of X. Likelihood function: N ! 1 Y 1 1 T −1 `(µ, Σ) := exp − (yi − µ) Σ (yi − µ) (2π)n/2 (det Σ)1/2 2 i=1 log-likelihood function N 1 XN n log `(µ, Σ) = − log(det Σ) − (y − µ)T Σ−1(y − µ) − log(2π) 2 2 i i 2 i=1 Maximum likelihood estimation max `(µ, Σ) ⇔ min − log `(µ, Σ) (µ,Σ) (µ,Σ) xi :=yi −µ 1 N ⇔ min tr (X T Σ−1X)+ log(det Σ) ∈ n×N × n 2 2 (X,Σ) R S++ Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals The generalized matrix-fractional function Motivation II: Maximum likelihood estimation n Let yi ∈ R (i = 1,..., N) be measurements of n n y ∼ N(µ, Σ) (µ ∈ R , Σ ∈ S++ → unknown) log-likelihood function N 1 XN n log `(µ, Σ) = − log(det Σ) − (y − µ)T Σ−1(y − µ) − log(2π) 2 2 i i 2 i=1 Maximum likelihood estimation max `(µ, Σ) ⇔ min − log `(µ, Σ) (µ,Σ) (µ,Σ) xi :=yi −µ 1 N ⇔ min tr (X T Σ−1X)+ log(det Σ) ∈ n×N × n 2 2 (X,Σ) R S++ Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals The generalized matrix-fractional function Motivation II: Maximum likelihood estimation n Let yi ∈ R (i = 1,..., N) be measurements of n n y ∼ N(µ, Σ) (µ ∈ R , Σ ∈ S++ → unknown) Likelihood function: N ! 1 Y 1 1 T −1 `(µ, Σ) := exp − (yi − µ) Σ (yi − µ) (2π)n/2 (det Σ)1/2 2 i=1 Maximum likelihood estimation max `(µ, Σ) ⇔ min − log `(µ, Σ) (µ,Σ) (µ,Σ) xi :=yi −µ 1 N ⇔ min tr (X T Σ−1X)+ log(det Σ) ∈ n×N × n 2 2 (X,Σ) R S++ Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals The generalized matrix-fractional function Motivation II: Maximum likelihood estimation n Let yi ∈ R (i = 1,..., N) be measurements of n n y ∼ N(µ, Σ) (µ ∈ R , Σ ∈ S++ → unknown) Likelihood function: N ! 1 Y 1 1 T −1 `(µ, Σ) := exp − (yi − µ) Σ (yi − µ) (2π)n/2 (det Σ)1/2 2 i=1 log-likelihood function N 1 XN n log `(µ, Σ) = − log(det Σ) − (y − µ)T Σ−1(y − µ) − log(2π) 2 2 i i 2 i=1 Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals The generalized matrix-fractional function Motivation II: Maximum likelihood estimation n Let yi ∈ R (i = 1,..., N) be measurements of n n y ∼ N(µ, Σ) (µ ∈ R , Σ ∈ S++ → unknown) Likelihood function: N ! 1 Y 1 1 T −1 `(µ, Σ) := exp − (yi − µ) Σ (yi − µ) (2π)n/2 (det Σ)1/2 2 i=1 log-likelihood function N 1 XN n log `(µ, Σ) = − log(det Σ) − (y − µ)T Σ−1(y − µ) − log(2π) 2 2 i i 2 i=1 Maximum likelihood estimation max `(µ, Σ) ⇔ min − log `(µ, Σ) (µ,Σ) (µ,Σ) xi :=yi −µ 1 N ⇔ min tr (X T Σ−1X)+ log(det Σ) ∈ n×N × n 2 2 (X,Σ) R S++ The matrix σ−1 1 . . . † † T † −1 A := VΣ U with Σ := σr , 0 . .. 0 called the (Moore-Penrose) pseudoinverse of A is the unique matrix with the following properties. a) AA †A = A and A †AA † = A † b) (AA †)T = AA † and (A †A)T = A †As Moreover: c) A invertible ⇒ A † = A −1 d) A 0 ⇒ A † 0 Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals The generalized matrix-fractional function The Moore-Penrose pseudoinverse Theorem 27 (Moore-Penrose pseudoinverse). Let A ∈ Rm×n with rank A = r and the singular value decomposition T A = UΣV with Σ = diag (σi ), U, V orthogonal. a) AA †A = A and A †AA † = A † b) (AA †)T = AA † and (A †A)T = A †As Moreover: c) A invertible ⇒ A † = A −1 d) A 0 ⇒ A † 0 Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals The generalized matrix-fractional function The Moore-Penrose pseudoinverse Theorem 27 (Moore-Penrose pseudoinverse). Let A ∈ Rm×n with rank A = r and the singular value decomposition T A = UΣV with Σ = diag (σi ), U, V orthogonal. The matrix σ−1 1 . . . † † T † −1 A := VΣ U with Σ := σr , 0 . .. 0 called the (Moore-Penrose) pseudoinverse of A is the unique matrix with the following properties. Moreover: c) A invertible ⇒ A † = A −1 d) A 0 ⇒ A † 0 Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals The generalized matrix-fractional function The Moore-Penrose pseudoinverse Theorem 27 (Moore-Penrose pseudoinverse). Let A ∈ Rm×n with rank A = r and the singular value decomposition T A = UΣV with Σ = diag (σi ), U, V orthogonal. The matrix σ−1 1 . . . † † T † −1 A := VΣ U with Σ := σr , 0 . .. 0 called the (Moore-Penrose) pseudoinverse of A is the unique matrix with the following properties. a) AA †A = A and A †AA † = A † b) (AA †)T = AA † and (A †A)T = A †As Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals The generalized matrix-fractional function The Moore-Penrose pseudoinverse Theorem 27 (Moore-Penrose pseudoinverse). Let A ∈ Rm×n with rank A = r and the singular value decomposition T A = UΣV with Σ = diag (σi ), U, V orthogonal. The matrix σ−1 1 . . . † † T † −1 A := VΣ U with Σ := σr , 0 . .. 0 called the (Moore-Penrose) pseudoinverse of A is the unique matrix with the following properties. a) AA †A = A and A †AA † = A † b) (AA †)T = AA † and (A †A)T = A †As Moreover: c) A invertible ⇒ A † = A −1 d) A 0 ⇒ A † 0 ( ! ) Schur VX ⇒ epi φ = (X, V, α)| ∃ Y ∈ Sm : 0, V 0, 1 tr (Y) ≤ α X T Y 2 ⇒ φ proper, sublinear and not lsc. ( 1 tr (X T V †X) if V 0, rge X ∈ rge V, ⇒ cl φ :(X, V) ∈ E 7→ 2 +∞ else is proper, lsc and sublinear Hormander’s¨ Theorem ⇒ cl φ is a support function Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals The generalized matrix-fractional function The closure of the matrix-fractional function Put E := Rn×m × Sn . ( 1 tr (X T V −1X) if V 0, φ :(X, V) ∈ E 7→ 2 (matrix-fractional function) +∞ else. ⇒ φ proper, sublinear and not lsc. ( 1 tr (X T V †X) if V 0, rge X ∈ rge V, ⇒ cl φ :(X, V) ∈ E 7→ 2 +∞ else is proper, lsc and sublinear Hormander’s¨ Theorem ⇒ cl φ is a support function Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals The generalized matrix-fractional function The closure of the matrix-fractional function Put E := Rn×m × Sn . ( 1 tr (X T V −1X) if V 0, φ :(X, V) ∈ E 7→ 2 (matrix-fractional function) +∞ else. ( ! ) Schur VX ⇒ epi φ = (X, V, α)| ∃ Y ∈ Sm : 0, V 0, 1 tr (Y) ≤ α X T Y 2 ( 1 tr (X T V †X) if V 0, rge X ∈ rge V, ⇒ cl φ :(X, V) ∈ E 7→ 2 +∞ else is proper, lsc and sublinear Hormander’s¨ Theorem ⇒ cl φ is a support function Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals The generalized matrix-fractional function The closure of the matrix-fractional function Put E := Rn×m × Sn . ( 1 tr (X T V −1X) if V 0, φ :(X, V) ∈ E 7→ 2 (matrix-fractional function) +∞ else. ( ! ) Schur VX ⇒ epi φ = (X, V, α)| ∃ Y ∈ Sm : 0, V 0, 1 tr (Y) ≤ α X T Y 2 ⇒ φ proper, sublinear and not lsc. Hormander’s¨ Theorem ⇒ cl φ is a support function Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals The generalized matrix-fractional function The closure of the matrix-fractional function Put E := Rn×m × Sn . ( 1 tr (X T V −1X) if V 0, φ :(X, V) ∈ E 7→ 2 (matrix-fractional function) +∞ else. ( ! ) Schur VX ⇒ epi φ = (X, V, α)| ∃ Y ∈ Sm : 0, V 0, 1 tr (Y) ≤ α X T Y 2 ⇒ φ proper, sublinear and not lsc. ( 1 tr (X T V †X) if V 0, rge X ∈ rge V, ⇒ cl φ :(X, V) ∈ E 7→ 2 +∞ else is proper, lsc and sublinear Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals The generalized matrix-fractional function The closure of the matrix-fractional function Put E := Rn×m × Sn . ( 1 tr (X T V −1X) if V 0, φ :(X, V) ∈ E 7→ 2 (matrix-fractional function) +∞ else. ( ! ) Schur VX ⇒ epi φ = (X, V, α)| ∃ Y ∈ Sm : 0, V 0, 1 tr (Y) ≤ α X T Y 2 ⇒ φ proper, sublinear and not lsc. ( 1 tr (X T V †X) if V 0, rge X ∈ rge V, ⇒ cl φ :(X, V) ∈ E 7→ 2 +∞ else is proper, lsc and sublinear Hormander’s¨ Theorem ⇒ cl φ is a support function Theorem 28 (Burke, H. ’15). For b ∈ rge A, we have !T ! ( ) 1 x † x T 1 T T − M(V) if x ∈ rge [VA ], V ∈ K , inf u Vu − x u | Au = b = 2 b b A u∈Rn 2 −∞ else. Question: For A ∈ Rp×n , B ∈ Rp×m , is 1 X T † X X ⊂ ∈ K 2 tr (B) M(V) (B) if rge (B) rge M(V), V A , ϕA,B :(X, V) ∈ E 7→ +∞ else a support function? Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals The generalized matrix-fractional function Motivation III: Quadratic programming For A ∈ Rp×n and V ∈ Sn put n o M(V) := VAT and K := V ∈ Sn uT Vu ≥ 0 (u ∈ ker A) . A 0 A Question: For A ∈ Rp×n , B ∈ Rp×m , is 1 X T † X X ⊂ ∈ K 2 tr (B) M(V) (B) if rge (B) rge M(V), V A , ϕA,B :(X, V) ∈ E 7→ +∞ else a support function? Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals The generalized matrix-fractional function Motivation III: Quadratic programming For A ∈ Rp×n and V ∈ Sn put n o M(V) := VAT and K := V ∈ Sn uT Vu ≥ 0 (u ∈ ker A) . A 0 A Theorem 28 (Burke, H. ’15). For b ∈ rge A, we have !T ! ( ) 1 x † x T 1 T T − M(V) if x ∈ rge [VA ], V ∈ K , inf u Vu − x u | Au = b = 2 b b A u∈Rn 2 −∞ else. Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals The generalized matrix-fractional function Motivation III: Quadratic programming For A ∈ Rp×n and V ∈ Sn put n o M(V) := VAT and K := V ∈ Sn uT Vu ≥ 0 (u ∈ ker A) . A 0 A Theorem 28 (Burke, H. ’15). For b ∈ rge A, we have !T ! ( ) 1 x † x T 1 T T − M(V) if x ∈ rge [VA ], V ∈ K , inf u Vu − x u | Au = b = 2 b b A u∈Rn 2 −∞ else. Question: For A ∈ Rp×n , B ∈ Rp×m , is 1 X T † X X ⊂ ∈ K 2 tr (B) M(V) (B) if rge (B) rge M(V), V A , ϕA,B :(X, V) ∈ E 7→ +∞ else a support function? Theorem 29 (Burke, H. ’15). For rge B ⊂ rge A 1 X T † X X ⊂ ∈ K 2 tr (B) M(V) (B) if rge (B) rge M(V), V A , σD(A,B)(X, V) = ((X, V) ∈ E) +∞ else with int (dom σD(A,B)) = (X, V) ∈ E | V ∈ int KA . In particular, ( 1 tr (X T V †X) if V 0, rge X ⊂ rge V, σ (X, V) = 2 = cl φ(X, V). D(0,0) +∞ else Proof. Blackboard/Notes. Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals The generalized matrix-fractional function A new class of matrix support functions Define ( ! ) 1 × × × D(A, B) := Y, − YY T ∈ E Y ∈ Rn m : AY = B (A ∈ Rp n , B ∈ Rp m ). 2 In particular, ( 1 tr (X T V †X) if V 0, rge X ⊂ rge V, σ (X, V) = 2 = cl φ(X, V). D(0,0) +∞ else Proof. Blackboard/Notes. Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals The generalized matrix-fractional function A new class of matrix support functions Define ( ! ) 1 × × × D(A, B) := Y, − YY T ∈ E Y ∈ Rn m : AY = B (A ∈ Rp n , B ∈ Rp m ). 2 Theorem 29 (Burke, H. ’15). For rge B ⊂ rge A 1 X T † X X ⊂ ∈ K 2 tr (B) M(V) (B) if rge (B) rge M(V), V A , σD(A,B)(X, V) = ((X, V) ∈ E) +∞ else with int (dom σD(A,B)) = (X, V) ∈ E | V ∈ int KA . Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals The generalized matrix-fractional function A new class of matrix support functions Define ( ! ) 1 × × × D(A, B) := Y, − YY T ∈ E Y ∈ Rn m : AY = B (A ∈ Rp n , B ∈ Rp m ). 2 Theorem 29 (Burke, H. ’15). For rge B ⊂ rge A 1 X T † X X ⊂ ∈ K 2 tr (B) M(V) (B) if rge (B) rge M(V), V A , σD(A,B)(X, V) = ((X, V) ∈ E) +∞ else with int (dom σD(A,B)) = (X, V) ∈ E | V ∈ int KA . In particular, ( 1 tr (X T V †X) if V 0, rge X ⊂ rge V, σ (X, V) = 2 = cl φ(X, V). D(0,0) +∞ else Proof. Blackboard/Notes. Proposition 30 (Burke, H. ’15). ( ) 1 T 6 conv D(A, B) = (Z(d ⊗ Im ), − ZZ ) (d, Z) ∈ F (A, B) . 2 where d ≥ 0, kdk = 1, κ+1 n×m(κ+1) 7 F (A, B) := (d, Z) ∈ R × R . AZi = di B (i = 1, . . . , κ + 1) Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals The closed convex hull of D(A, B) with applications Closed convex hull of D(A, B) : Caratheodory-based´ description Recall n o ∈ D ∈ ∗ ∂σD(A,B)(X, V) = (Y, W) conv (A, B) (X, V) Nconv D(A,B)(Y, W) and σD(A,B) = δconv D(A,B) where ( ! ) 1 × D(A, B) := Y, − YY T ∈ E Y ∈ Rn m : AY = B . 2 6 m(κ+1) d ⊗ Im = (di Im ) ∈ R 7κ := dim E Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals The closed convex hull of D(A, B) with applications Closed convex hull of D(A, B) : Caratheodory-based´ description Recall n o ∈ D ∈ ∗ ∂σD(A,B)(X, V) = (Y, W) conv (A, B) (X, V) Nconv D(A,B)(Y, W) and σD(A,B) = δconv D(A,B) where ( ! ) 1 × D(A, B) := Y, − YY T ∈ E Y ∈ Rn m : AY = B . 2 Proposition 30 (Burke, H. ’15). ( ) 1 T 6 conv D(A, B) = (Z(d ⊗ Im ), − ZZ ) (d, Z) ∈ F (A, B) . 2 where d ≥ 0, kdk = 1, κ+1 n×m(κ+1) 7 F (A, B) := (d, Z) ∈ R × R . AZi = di B (i = 1, . . . , κ + 1) 6 m(κ+1) d ⊗ Im = (di Im ) ∈ R 7κ := dim E Theorem 31 (Burke, Gao, H. ’17). We have conv D(A, B) = Ω(A, B). In particular, ( ) 1 conv D(0, 0) = (Y, W) ∈ E AY = B and YY T + W 0 . 2 Proof. Notes. Corollary 32 (Conjugate of GMF). We have ∗ σD(A,B) = δΩ(A,B). Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals The closed convex hull of D(A, B) with applications Closed convex hull of D(A, B) : A new description Define ( ) 1 ◦ Ω(A, B) := (Y, W) ∈ E AY = B and YY T + W ∈ K , (13) 2 A and observe that ◦ n T o KA = R+conv −vv | v ∈ ker A . Corollary 32 (Conjugate of GMF). We have ∗ σD(A,B) = δΩ(A,B). Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals The closed convex hull of D(A, B) with applications Closed convex hull of D(A, B) : A new description Define ( ) 1 ◦ Ω(A, B) := (Y, W) ∈ E AY = B and YY T + W ∈ K , (13) 2 A and observe that ◦ n T o KA = R+conv −vv | v ∈ ker A . Theorem 31 (Burke, Gao, H. ’17). We have conv D(A, B) = Ω(A, B). In particular, ( ) 1 conv D(0, 0) = (Y, W) ∈ E AY = B and YY T + W 0 . 2 Proof. Notes. Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals The closed convex hull of D(A, B) with applications Closed convex hull of D(A, B) : A new description Define ( ) 1 ◦ Ω(A, B) := (Y, W) ∈ E AY = B and YY T + W ∈ K , (13) 2 A and observe that ◦ n T o KA = R+conv −vv | v ∈ ker A . Theorem 31 (Burke, Gao, H. ’17). We have conv D(A, B) = Ω(A, B). In particular, ( ) 1 conv D(0, 0) = (Y, W) ∈ E AY = B and YY T + W 0 . 2 Proof. Notes. Corollary 32 (Conjugate of GMF). We have ∗ σD(A,B) = δΩ(A,B). Proposition 33 (Burke, Gao, H. ’17). Let Ω(A, B) be given as above. Then: n o ∈ 1 T ∈ K ◦ a) ri Ω(A, B) = (Y, W) E AY = B and 2 YY + W ri ( A ) . n o ∈ 1 T ∈ K ◦ b) aff Ω(A, B) = (Y, W) E AY = B and 2 YY + W span A . T c) Ω(A, B)◦ = (X, V) rge (X) ⊂ rge M(V), V ∈ K , 1 tr (X) M(V)†(X) ≤ 1 . B A 2 B B ∞ { } × K ◦ d) Ω(A, B) = 0n×m A . Proposition 34 (Burke, Gao, H. ’17). Let Ω(A, B) be given as above and let (Y, W) ∈ Ω(A, B). Then * + 1 T V ∈ KA , V, YY + W = 0 N (Y, W) = (X, V) ∈ E 2 . Ω(A,B) ⊥ and rge (X − VY) ⊂ (ker A) Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals The closed convex hull of D(A, B) with applications Convex geometry of Ω(A, B) n o ∈ 1 T ∈ K ◦ Recall that Ω(A, B) := (Y, W) E AY = B and 2 YY + W A n o ∈ 1 T ∈ K ◦ b) aff Ω(A, B) = (Y, W) E AY = B and 2 YY + W span A . T c) Ω(A, B)◦ = (X, V) rge (X) ⊂ rge M(V), V ∈ K , 1 tr (X) M(V)†(X) ≤ 1 . B A 2 B B ∞ { } × K ◦ d) Ω(A, B) = 0n×m A . Proposition 34 (Burke, Gao, H. ’17). Let Ω(A, B) be given as above and let (Y, W) ∈ Ω(A, B). Then * + 1 T V ∈ KA , V, YY + W = 0 N (Y, W) = (X, V) ∈ E 2 . Ω(A,B) ⊥ and rge (X − VY) ⊂ (ker A) Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals The closed convex hull of D(A, B) with applications Convex geometry of Ω(A, B) n o ∈ 1 T ∈ K ◦ Recall that Ω(A, B) := (Y, W) E AY = B and 2 YY + W A Proposition 33 (Burke, Gao, H. ’17). Let Ω(A, B) be given as above. Then: n o ∈ 1 T ∈ K ◦ a) ri Ω(A, B) = (Y, W) E AY = B and 2 YY + W ri ( A ) . T c) Ω(A, B)◦ = (X, V) rge (X) ⊂ rge M(V), V ∈ K , 1 tr (X) M(V)†(X) ≤ 1 . B A 2 B B ∞ { } × K ◦ d) Ω(A, B) = 0n×m A . Proposition 34 (Burke, Gao, H. ’17). Let Ω(A, B) be given as above and let (Y, W) ∈ Ω(A, B). Then * + 1 T V ∈ KA , V, YY + W = 0 N (Y, W) = (X, V) ∈ E 2 . Ω(A,B) ⊥ and rge (X − VY) ⊂ (ker A) Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals The closed convex hull of D(A, B) with applications Convex geometry of Ω(A, B) n o ∈ 1 T ∈ K ◦ Recall that Ω(A, B) := (Y, W) E AY = B and 2 YY + W A Proposition 33 (Burke, Gao, H. ’17). Let Ω(A, B) be given as above. Then: n o ∈ 1 T ∈ K ◦ a) ri Ω(A, B) = (Y, W) E AY = B and 2 YY + W ri ( A ) . n o ∈ 1 T ∈ K ◦ b) aff Ω(A, B) = (Y, W) E AY = B and 2 YY + W span A . ∞ { } × K ◦ d) Ω(A, B) = 0n×m A . Proposition 34 (Burke, Gao, H. ’17). Let Ω(A, B) be given as above and let (Y, W) ∈ Ω(A, B). Then * + 1 T V ∈ KA , V, YY + W = 0 N (Y, W) = (X, V) ∈ E 2 . Ω(A,B) ⊥ and rge (X − VY) ⊂ (ker A) Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals The closed convex hull of D(A, B) with applications Convex geometry of Ω(A, B) n o ∈ 1 T ∈ K ◦ Recall that Ω(A, B) := (Y, W) E AY = B and 2 YY + W A Proposition 33 (Burke, Gao, H. ’17). Let Ω(A, B) be given as above. Then: n o ∈ 1 T ∈ K ◦ a) ri Ω(A, B) = (Y, W) E AY = B and 2 YY + W ri ( A ) . n o ∈ 1 T ∈ K ◦ b) aff Ω(A, B) = (Y, W) E AY = B and 2 YY + W span A . T c) Ω(A, B)◦ = (X, V) rge (X) ⊂ rge M(V), V ∈ K , 1 tr (X) M(V)†(X) ≤ 1 . B A 2 B B Proposition 34 (Burke, Gao, H. ’17). Let Ω(A, B) be given as above and let (Y, W) ∈ Ω(A, B). Then * + 1 T V ∈ KA , V, YY + W = 0 N (Y, W) = (X, V) ∈ E 2 . Ω(A,B) ⊥ and rge (X − VY) ⊂ (ker A) Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals The closed convex hull of D(A, B) with applications Convex geometry of Ω(A, B) n o ∈ 1 T ∈ K ◦ Recall that Ω(A, B) := (Y, W) E AY = B and 2 YY + W A Proposition 33 (Burke, Gao, H. ’17). Let Ω(A, B) be given as above. Then: n o ∈ 1 T ∈ K ◦ a) ri Ω(A, B) = (Y, W) E AY = B and 2 YY + W ri ( A ) . n o ∈ 1 T ∈ K ◦ b) aff Ω(A, B) = (Y, W) E AY = B and 2 YY + W span A . T c) Ω(A, B)◦ = (X, V) rge (X) ⊂ rge M(V), V ∈ K , 1 tr (X) M(V)†(X) ≤ 1 . B A 2 B B ∞ { } × K ◦ d) Ω(A, B) = 0n×m A . Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals The closed convex hull of D(A, B) with applications Convex geometry of Ω(A, B) n o ∈ 1 T ∈ K ◦ Recall that Ω(A, B) := (Y, W) E AY = B and 2 YY + W A Proposition 33 (Burke, Gao, H. ’17). Let Ω(A, B) be given as above. Then: n o ∈ 1 T ∈ K ◦ a) ri Ω(A, B) = (Y, W) E AY = B and 2 YY + W ri ( A ) . n o ∈ 1 T ∈ K ◦ b) aff Ω(A, B) = (Y, W) E AY = B and 2 YY + W span A . T c) Ω(A, B)◦ = (X, V) rge (X) ⊂ rge M(V), V ∈ K , 1 tr (X) M(V)†(X) ≤ 1 . B A 2 B B ∞ { } × K ◦ d) Ω(A, B) = 0n×m A . Proposition 34 (Burke, Gao, H. ’17). Let Ω(A, B) be given as above and let (Y, W) ∈ Ω(A, B). Then * + 1 T V ∈ KA , V, YY + W = 0 N (Y, W) = (X, V) ∈ E 2 . Ω(A,B) ⊥ and rge (X − VY) ⊂ (ker A) Corollary 35 (The subdifferential of σD(A,B)). For all (X, V) ∈ dom σD(A,B), we have ∃Z ∈ Rp×m : X = VY + A T Z, * + ∂σD(A,B) = (Y, W) ∈ Ω(A, B) 1 . V, YY T + W = 0 2 Corollary 36. The GMF σD(A,B) is (continuously) differentiable on the interior of its domain with ! 1 T ∇σD (X, V) = Y, − YY ((X, V) ∈ int (dom σD )) (A,B) 2 (A,B) where Y := A †B + (P(PT VP)−1PT )(X − A †X),P ∈ Rn×(n−p) is any matrix whose columns form an orthonormal basis of ker A and p := rank A. Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals The closed convex hull of D(A, B) with applications Subdifferentiation of the GMF For any set C recall that n o ∂σC (x) = z ∈ conv C x ∈ Nconv C (z) (14) Corollary 36. The GMF σD(A,B) is (continuously) differentiable on the interior of its domain with ! 1 T ∇σD (X, V) = Y, − YY ((X, V) ∈ int (dom σD )) (A,B) 2 (A,B) where Y := A †B + (P(PT VP)−1PT )(X − A †X),P ∈ Rn×(n−p) is any matrix whose columns form an orthonormal basis of ker A and p := rank A. Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals The closed convex hull of D(A, B) with applications Subdifferentiation of the GMF For any set C recall that n o ∂σC (x) = z ∈ conv C x ∈ Nconv C (z) (14) Corollary 35 (The subdifferential of σD(A,B)). For all (X, V) ∈ dom σD(A,B), we have ∃Z ∈ Rp×m : X = VY + A T Z, * + ∂σD(A,B) = (Y, W) ∈ Ω(A, B) 1 . V, YY T + W = 0 2 Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals The closed convex hull of D(A, B) with applications Subdifferentiation of the GMF For any set C recall that n o ∂σC (x) = z ∈ conv C x ∈ Nconv C (z) (14) Corollary 35 (The subdifferential of σD(A,B)). For all (X, V) ∈ dom σD(A,B), we have ∃Z ∈ Rp×m : X = VY + A T Z, * + ∂σD(A,B) = (Y, W) ∈ Ω(A, B) 1 . V, YY T + W = 0 2 Corollary 36. The GMF σD(A,B) is (continuously) differentiable on the interior of its domain with ! 1 T ∇σD (X, V) = Y, − YY ((X, V) ∈ int (dom σD )) (A,B) 2 (A,B) where Y := A †B + (P(PT VP)−1PT )(X − A †X),P ∈ Rn×(n−p) is any matrix whose columns form an orthonormal basis of ker A and p := rank A. n n S+ is the smallest closed convex cone in S with respect to which F is convex; n −hzn σM ⊃ S+. In particular, F is (−hzn σM )-convex. Theorem 37 (Jalali et al. ’17/ Burke, Gao, H. ’19). ⊂ n ∗ Let M S+ be nonempty, convex and compact. Then ΩM is finite-valued and given by ∗ 1 n † o Ω (X) = min tr (X T V X) rge X ⊂ rge V . 2 V∈M Proof. Blackboard/Notes. Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Applications of the GMF Conjugate of variational Gram functions n For M ⊂ S+ (w.lo.g.) closed, convex, the associated variational Gram function (VGF) is given by 1 Ω : Rn×m → R ∪ {+∞}, Ω (X) = σ (XX T ). M M 2 M With 1 F : Rn×m → Sn , F(X) = XX T . (15) 2 ΩM = σM ◦ F fits the composite scheme studied in Section 2. Theorem 37 (Jalali et al. ’17/ Burke, Gao, H. ’19). ⊂ n ∗ Let M S+ be nonempty, convex and compact. Then ΩM is finite-valued and given by ∗ 1 n † o Ω (X) = min tr (X T V X) rge X ⊂ rge V . 2 V∈M Proof. Blackboard/Notes. Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Applications of the GMF Conjugate of variational Gram functions n For M ⊂ S+ (w.lo.g.) closed, convex, the associated variational Gram function (VGF) is given by 1 Ω : Rn×m → R ∪ {+∞}, Ω (X) = σ (XX T ). M M 2 M With 1 F : Rn×m → Sn , F(X) = XX T . (15) 2 ΩM = σM ◦ F fits the composite scheme studied in Section 2. n n S+ is the smallest closed convex cone in S with respect to which F is convex; n −hzn σM ⊃ S+. In particular, F is (−hzn σM )-convex. Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Applications of the GMF Conjugate of variational Gram functions n For M ⊂ S+ (w.lo.g.) closed, convex, the associated variational Gram function (VGF) is given by 1 Ω : Rn×m → R ∪ {+∞}, Ω (X) = σ (XX T ). M M 2 M With 1 F : Rn×m → Sn , F(X) = XX T . (15) 2 ΩM = σM ◦ F fits the composite scheme studied in Section 2. n n S+ is the smallest closed convex cone in S with respect to which F is convex; n −hzn σM ⊃ S+. In particular, F is (−hzn σM )-convex. Theorem 37 (Jalali et al. ’17/ Burke, Gao, H. ’19). ⊂ n ∗ Let M S+ be nonempty, convex and compact. Then ΩM is finite-valued and given by ∗ 1 n † o Ω (X) = min tr (X T V X) rge X ⊂ rge V . 2 V∈M Proof. Blackboard/Notes. Theorem 38. Let p : Rn×m → R be defined by D¯ E p(X) = inf σΩ(A,0)(X, V) + U, V V∈Sn n o ¯ ∈ n ∩ ¯ 1 T ¯ for some U S+ Ker A and set C(U) := Y 2 YY U . Then we have: ∗ a) p = δC(U¯)∩Ker A is closed, proper, convex. b) p = σC(U¯)∩Ker A = γC(U¯)◦+Rge AT is sublinear, finite-valued, nonnegative and symmetric (i.e. a seminorm). ¯ ¯ T n×n T c) If U 0 with 2U = LL (L ∈ R ) and A = 0 then p = σC(U¯) = kL (·)k∗, i.e. p is a norm with ¯ ◦ C(U) as its unit ball and γC(U¯) as its dual norm. Proof. Blackboard/Notes. Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Applications of the GMF Nuclear norm smoothing For A ∈ Rp×n set n×n n n×n o Ker A := V ∈ R | AV = 0 and Rge A := W ∈ R rge W ⊂ rge A . Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Applications of the GMF Nuclear norm smoothing For A ∈ Rp×n set n×n n n×n o Ker A := V ∈ R | AV = 0 and Rge A := W ∈ R rge W ⊂ rge A . Theorem 38. Let p : Rn×m → R be defined by D¯ E p(X) = inf σΩ(A,0)(X, V) + U, V V∈Sn n o ¯ ∈ n ∩ ¯ 1 T ¯ for some U S+ Ker A and set C(U) := Y 2 YY U . Then we have: ∗ a) p = δC(U¯)∩Ker A is closed, proper, convex. b) p = σC(U¯)∩Ker A = γC(U¯)◦+Rge AT is sublinear, finite-valued, nonnegative and symmetric (i.e. a seminorm). ¯ ¯ T n×n T c) If U 0 with 2U = LL (L ∈ R ) and A = 0 then p = σC(U¯) = kL (·)k∗, i.e. p is a norm with ¯ ◦ C(U) as its unit ball and γC(U¯) as its dual norm. Proof. Blackboard/Notes. Subdifferential analysis for convex convex-composites, unification with the nonconvex convex-composite case (BCQ etc.) Learn more about existing literature! Generalized matrix-fractional function Systematic study of (partial) infimal projections p(X) = inf σΩ(A,B)(X, V) + h(V). V∈Sn n for h ∈ Γ0(S ). → SIOPT article to appear. Numerical methods based on GMF. Compute (analytically/numerically) the projection onto Ω(A, B) (→ projection/proximal-based algorithms). Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Applications of the GMF Current and future directions K-convexity When is conv (gph F) = K-epi F for FK-convex ? Learn more about existing literature! Generalized matrix-fractional function Systematic study of (partial) infimal projections p(X) = inf σΩ(A,B)(X, V) + h(V). V∈Sn n for h ∈ Γ0(S ). → SIOPT article to appear. Numerical methods based on GMF. Compute (analytically/numerically) the projection onto Ω(A, B) (→ projection/proximal-based algorithms). Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Applications of the GMF Current and future directions K-convexity When is conv (gph F) = K-epi F for FK-convex ? Subdifferential analysis for convex convex-composites, unification with the nonconvex convex-composite case (BCQ etc.) Generalized matrix-fractional function Systematic study of (partial) infimal projections p(X) = inf σΩ(A,B)(X, V) + h(V). V∈Sn n for h ∈ Γ0(S ). → SIOPT article to appear. Numerical methods based on GMF. Compute (analytically/numerically) the projection onto Ω(A, B) (→ projection/proximal-based algorithms). Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Applications of the GMF Current and future directions K-convexity When is conv (gph F) = K-epi F for FK-convex ? Subdifferential analysis for convex convex-composites, unification with the nonconvex convex-composite case (BCQ etc.) Learn more about existing literature! Numerical methods based on GMF. Compute (analytically/numerically) the projection onto Ω(A, B) (→ projection/proximal-based algorithms). Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Applications of the GMF Current and future directions K-convexity When is conv (gph F) = K-epi F for FK-convex ? Subdifferential analysis for convex convex-composites, unification with the nonconvex convex-composite case (BCQ etc.) Learn more about existing literature! Generalized matrix-fractional function Systematic study of (partial) infimal projections p(X) = inf σΩ(A,B)(X, V) + h(V). V∈Sn n for h ∈ Γ0(S ). → SIOPT article to appear. Compute (analytically/numerically) the projection onto Ω(A, B) (→ projection/proximal-based algorithms). Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Applications of the GMF Current and future directions K-convexity When is conv (gph F) = K-epi F for FK-convex ? Subdifferential analysis for convex convex-composites, unification with the nonconvex convex-composite case (BCQ etc.) Learn more about existing literature! Generalized matrix-fractional function Systematic study of (partial) infimal projections p(X) = inf σΩ(A,B)(X, V) + h(V). V∈Sn n for h ∈ Γ0(S ). → SIOPT article to appear. Numerical methods based on GMF. Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Applications of the GMF Current and future directions K-convexity When is conv (gph F) = K-epi F for FK-convex ? Subdifferential analysis for convex convex-composites, unification with the nonconvex convex-composite case (BCQ etc.) Learn more about existing literature! Generalized matrix-fractional function Systematic study of (partial) infimal projections p(X) = inf σΩ(A,B)(X, V) + h(V). V∈Sn n for h ∈ Γ0(S ). → SIOPT article to appear. Numerical methods based on GMF. Compute (analytically/numerically) the projection onto Ω(A, B) (→ projection/proximal-based algorithms). Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Applications of the GMF Current and future directions K-convexity When is conv (gph F) = K-epi F for FK-convex ? Subdifferential analysis for convex convex-composites, unification with the nonconvex convex-composite case (BCQ etc.) Learn more about existing literature! Generalized matrix-fractional function Systematic study of (partial) infimal projections p(X) = inf σΩ(A,B)(X, V) + h(V). V∈Sn n for h ∈ Γ0(S ). → SIOPT article to appear. Numerical methods based on GMF. Compute (analytically/numerically) the projection onto Ω(A, B) (→ projection/proximal-based algorithms). Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Applications of the GMF References H. Attouch and H. Brezis:´ Duality for the Sum of Convex Functions in General Banach Spaces. In Aspects of Mathematics and its Applications, Eds. l.A. Barroso, North Holland, Amsterdam, 1986, pp.125–133. H.H. Bauschke and P.L. Combettes: Convex analysis and Monotone Operator Theory in Hilbert Spaces. CMS Books in Mathematics, Springer-Verlag, New York, 2011. J.V. Burke and T. Hoheisel: Matrix support functionals for inverse problems, regularization, and learning. SIAM Journal on Optimization 25(2), 2015, pp. 1135–1159. J.V. Burke, T. Hoheisel, and Y. Gao: Convex geometry of the generalized matrix-fractional function. SIAM Journal on Optimization 28(3), 2018, pp. 2189–2200. J.V. Burke, T. Hoheisel, and Y. Gao: Variational properties of matrix functions via the generalized matrix-fractional function. SIAM Journal on Optimization, to appear. J.V. Burke, T. Hoheisel, and Q.V. Nguyen: A note on the Fenchel conjugate of composite functions via infimal convolution. Working paper, McGill University, 2019. M. Fazel, H. Hindi, and S. Boyd: A Rank Minimization Heuristic with Application to Minimum Order System Approximation. Proceedings of the American Control Conference, Arlington, Virginia, June 2001. Fundamentals from Convex Analysis Conjugacy of composite functions via K-convexity and inf-convolution A new class of matrix support functionals Applications of the GMF L. Hormander:¨ Sur la fonction d’appui des ensembles convexes dans un espace localement convexe. Arkiv for¨ Matematik 21(3), 1955, pp. 181–186. C.-J. Hsieh and P. Olsen: Nuclear norm minimization via active subspace selection. Proceedings of the 31st International Conference on Machine Learning (ICML-14), 2014, pp. 575-583. A. Jalali, M. Fazel, and L. Xiao: Variational Gram functions: Convex analysis and optimization. SIAM Journal on Optimimization 27(4), 2017, pp. 2634–2661. Teemu Pennanen: Graph-Convex Mappings and K-Convex Functions. Journal of Convex Analysis 6(2), 1999, pp. 235–266. R.T. Rockafellar: Convex Analysis. Princeton University Press, Princeton, New Jersey, 1970. R.T. Rockafellar and R.J.-B. Wets: Variational Analysis. A Series of Comprehensive Studies in Mathematics, Vol. 317, Springer, Berlin, Heidelberg, 1998. J.F. Toland: A duality principle for non-convex optimisation and the calculus of variations. Archive for Rational Mechanics and Analysis 71(1), 1979, pp. 41-61.