Discrete Convex Analysis I

Home , Arg max, Legendre transformation

Hausdorﬀ School: Economics and Tropical Geometry Bonn, May 9-13, 2016

Discrete Convex Analysis I:

Concepts of Discrete Convex Functions

Kazuo Murota (Tokyo Metropolitan University)

160509BonnHCMecon1 1 Convex Function f(x) f(x)

convex nonconvex f : Rn → R is convex ⇐⇒ λf(x) + (1 − λ)f(y) ≥ f(λx + (1 − λ)y) (0 < ∀ λ < 1)

f(x) 6

x z y - k R = R ∪ {+∞} λx + (1 − λ)y 2 Features of Convex Functions

• Occurrence in many models motivations, applications • Operations and transformations

• Suﬃcient structure for a theory mathematically beautiful, practically useful • Minimization algorithms

3 Features of Convex Functions = Issues in discrete convex analysis • Occurrence in many models ? motivations, applications • Operations and transformations ?

• Suﬃcient structure for a theory ? mathematically beautiful, practically useful • Minimization algorithms ?

4 Contents of Part I Concepts of Discrete Convex Functions

C1. Univariate Discrete Convex Functions C2. Classes of Discrete Convex Functions C3. L-convex Functions C4. M-convex Functions C5. Remarks on Submodular Set Functions

Part II: Properties, Part III: Algorithms 5 C1. Univariate Discrete Convex Functions

Ingredients of convex analysis

6 Deﬁnition of Convex Function f : Z → R R = R ∪ {+∞} f(x − 1) + f(x + 1) ≥ 2f(x) ⇐⇒ f(x) + f(y) ≥ f(x + 1) + f(y − 1) (x < y) ⇐⇒ f is convex-extensible, i.e., ∃ convex f : R → R s.t. f(x) = f(x) (∀x ∈ Z)

convex non-convex

7 Local vs Global Optimality f : Z → R

Theorem: x∗: global opt (min) ⇐⇒ x∗: local opt (min) f(x∗) ≤ min{f(x∗ − 1), f(x∗ + 1)}

convex non-convex

8 Intuition of Legendre Transformation

y 6 y 6 f(x) f(x)

- - x x

9 Intuition of Legendre Transformation

y 6 y 6 f(x)

- - x x

10 Intuition of Legendre Transformation

y 6 y 6 f(x)

- - x x

6 6 y y

- - x x

11 Legendre Transformation f : Z → Z (integer-valued)

Deﬁne discrete Legendre transform of f by f •(p) = sup{px − f(x) | x ∈ Z} (p ∈ Z)

6 6 y f(x) y

−f •(p) slope p - - x x Theorem: (1) f • is Z-valued convex function, f • : Z → Z (2) (f •)• = f (biconjugacy)

12 Separation Theorem f : Z → R f(x) f(x) convex p∗ h : Z → R concave h(x) h(x)

Theorem (Discrete Separation Theorem) (1) f(x) ≥ h(x)(∀x ∈ Z) ⇒ ∃ α∗ ∈ R, ∃ p∗ ∈ R: f(x) ≥ α∗ + p∗ x ≥ h(x)(∀x ∈ Z)

(2) f, h: integer-valued ⇒ α∗ ∈ Z, p∗ ∈ Z

13 Separation Theorem f(x) p∗ f : Z → R convex h : Z → R concave h(x)

Theorem (Discrete Separation Theorem) (1) f(x) ≥ h(x)(∀x ∈ Z) ⇒ ∃ α∗ ∈ R, ∃ p∗ ∈ R: f(x) ≥ α∗ + p∗ x ≥ h(x)(∀x ∈ Z)

(2) f, h: integer-valued ⇒ α∗ ∈ Z, p∗ ∈ Z

14 Fenchel Duality (Min-Max) f : Z → Z: convex, h : Z → Z: concave

Legendre transforms: f•(p) = sup{px − f(x) | x ∈ Z} h◦(p) = inf{px − h(x) | x ∈ Z}

Theorem: inf {f(x) − h(x)} = sup {h◦(p) − f•(p)} x∈Z p∈Z

15 Five Properties of “Convex” Functions

1. convex extension 2. local opt = global opt 3. Legendre transform (biconjugacy) 4. separation theorem 5. Fenchel duality hold for univariate discrete convex functions

16 C2. Classes of Discrete Convex Functions

17 Classes of Discrete Convex Functions

1. Submodular set fn (on {0, 1}n) 1. Separable-convex fn on Zn 1. Integrally-convex fn on Zn

2. L-convex (L\-convex) fn on Zn 2. M-convex (M\-convex) fn on Zn

3. M-convex fn on jump systems 3. L-convex fn on graphs 18 Submodular Function R = R ∪ {+∞} Set function ρ : 2V → R is submodular ⇐⇒ X ∪ Y ρ(X) + ρ(Y ) ≥ ρ(X ∪ Y ) + ρ(X ∩ Y ) XYX ∩ Y cf. |X| + |Y | = |X ∪ Y | + |X ∩ Y |

Set function ⇐⇒ Function on {0, 1}n

19 Separable-convex Function f : Zn → R is separable-convex ⇐⇒ f(x) = ϕ1(x1) + ϕ2(x2) + ··· + ϕn(xn) ϕi: univariate convex ϕi 6

20 Five Properties of “Convex” Functions

1. convex extension 2. local opt = global opt 3. Legendre transform (biconjugacy) 4. separation theorem 5. Fenchel duality hold for separable discrete convex functions

21 Some History 1935 Matroid Whitney, Nakasawa 1965 Submodular function Edmonds 1969 Convex network ﬂow (electr.circuit) Iri 1982 Submodularity and convexity Frank, Fujishige, Lov´asz 1990 Valuated matroid Dress–Wenzel Integrally convex fn Favati–Tardella 1996 Discrete convex analysis Murota 2000 Submodular minimization algorithm Iwata–Fleischer–Fujishige, Schrijver 2006 M-convex fn on jump system Murota 2012 L-convex fn on graph Hirai, Kolmogorov

22 Motivations/Applications/Connections

1. submodular MANY problems graph cut, convex game 1. separable-conv MANY problems min-cost ﬂow, resource allocation 1. integrally-conv [mathematical aesthetics]

2. L-conv (Zn) network tension, image processing OR (inventory, scheduling) 2. M-conv (Zn) network ﬂow, conjestion game economics (game, auction) mixed polynomial matrix 3. M-conv (jump) deg sequence, (2-)matching polynomial (half-plane property) 3. L-conv (graph) multiﬂow, multifacility location

23 Books (discrete convex analysis) 2000: Murota, Matrices and Matroids for Systems Analysis, Springer 2003: Murota, Discrete Convex Analysis, SIAM 2005: Fujishige, Submodular Functions and Optimization, 2nd ed., Elsevier 2014: Simchi-Levi, Chen, Bramel, The Logic of Logistics, 3rd ed., Springer

24 Convex Extension • f : Zn → R is convex-extensible ⇔ ∃ convex f : Rn → R: f(x) = f(x) (∀x ∈ Zn) • f is a convex extension of f • convex closure = (pointwise) max convex extension

convex-extensible NOT convex-extensible

25 Integrally Convex Function (Favati-Tardella 1990) n n N(x) = {y ∈ Z | kx − yk∞ < 1} (x ∈ R )

x x

Local convex extension: f˜(x) = sup{hp, xi + α | hp, yi + α ≤ f(y)(∀y ∈ N(x))} p,α

Def: f is integrally convex ⇐⇒ f˜ is convex

Ex: f(x1, x2) = |x1 − 2x2| is NOT integrally convex f = 1 f = 0 f˜(x) = 1 x = (1, 1/2) f(x) = 0 f = 0 f = 1 26 Integrally Convex Set

YES NO

27 Five Properties of “Convex” Functions

1. convex extension 2. local opt = global opt hold for integrally convex functions

3. Legendre transform (biconjugacy) 4. separation theorem 5. Fenchel duality fail for integrally convex functions

28 Deﬁnitions

1. submodular (set fn) ρ(X) + ρ(Y ) ≥ ρ(X ∪ Y ) + ρ(X ∩ Y ) 1. separable f(x) = ϕ1(x1) + ϕ2(x2) + ··· + ϕn(xn) -conv ϕi(t − 1) + ϕi(t + 1) ≥ 2ϕi(t)(∀t ∈ Z) 1. integrally -conv Local convex ext f˜(x) is convex 2. L-conv(Zn)

2. M-conv(Zn)

3. M-conv(jump)

3. L-conv(graph)

29 Classes of Discrete Convex Functions f : Zn → R convex-extensible integrally convex

submod M\-convex set fn separable convex L\-convex

L-convex M-convex on graph on jump

30 Bivariate L\- and M\-convex Functions

f(x)

g(p)

p1 x1

L\-convex fn M\-convex fn

31 C3. L-convex Functions

32 L-convex Function (L = Lattice) (Murota 98) g : Zn → R ∪ {+∞} q p ∨ q p ∨ q compnt-max p ∧ q compnt-min © p p ∧ q Def: g is L-convex ⇐⇒ • Submodular: g(p) + g(q) ≥ g(p ∨ q) + g(p ∧ q) • Translation: ∃r, ∀p: g(p + 1) = g(p) + r 1 = (1, 1,..., 1) 6 L

- L\ 33 L\-convexity from Submodularity (Murota 98, Fujishige–Murota 2000) g : Zn → R L\-convex ⇐⇒ g˜(p0, p) = g(p − p01) is submodular in (p0, p) g˜ : Zn+1 → R, 1 = (1, 1,..., 1) q˜ p˜ ∨ q˜ g˜(˜p) +g ˜(˜q) ≥ g˜ (˜p ∨ q˜) +g ˜ (˜p ∧ q˜)

p˜ p˜ ∧ q˜ \ Ln+1 ' Ln ) Ln

34 L\-convexity from Mid-pt-convexity (Favati-Tardella 1990, Fujishige–Murota 2000)

6 q q

=⇒ p+q p - 2 p q p+q p 2 Mid-point convex (g: Rn → R): p+q g(p) + g(q) ≥ 2g 2 n =⇒ Discrete mid-pointl convexm (jg : Z →kR) p+q p+q g(p) + g(q) ≥ g 2 + g 2 L\-convex function (L = Lattice)

35 Mid-pt Convexity for 01-Vectors l m 6 p+q q = p ∨ q For p, q ∈ {0, 1}n 2

p j k - p+q 2 = p ∧ q

Discrete mid-pt convexity:l m j k p+q p+q g(p) + g(q) ≥ g 2 + g 2 ⇐⇒ Submodularity: g(p) + g(q) ≥ g (p ∨ q) + g (p ∧ q)

36 Translation Submodularity (L\) g(p) + g(q) ≥ g((p − α1) ∨ q) + g(p ∧ (q + α1)) (α ≥ 0) p q q + α1

α = 2 © p − α1 p q discrete mid-pt convex

g˜(p0, p) = g(p − p01) is submodular in (p0, p) (Fujishige-Murota 00) ⇔ translation submodular (Fujishige-Murota 00) ⇔ discrete mid-pt convex (Favati-Tardella 90) ⇔ submod. integ. convex

37 Rem: L\-convex vs Submodular

n = 1

Fact 1: Every g : Z → R is submodular

Fact 2: Function g : Z → R is L\-convex ⇐⇒ g(p − 1) + g(p + 1) ≥ 2g(p) for all p ∈ Z

38 L\-convex Function: Examples X X \ Quadratic: g(p) = aijpipj is L -convex i j X ⇔ aij ≤ 0 (i 6= j), aij ≥ 0 (∀i) j Energy function: For univariate convex ψi and ψij X X g(p) = ψi(pi) + ψij(pi − pj) i i6=j Range: g(p) = max{p1, p2, . . . , pn} − min{p1, p2, . . . , pn} Submodular set function: ρ : 2V → R \ ⇔ ρ(X) = g(χX) for some L -convex g Multimodular: h : Zn → R is multimodular ⇔ \ h(p) = g(p1, p1 + p2, . . . , p1 + ··· + pn) for L -convex g

39 Five Properties of “Convex” Functions

1. convex extension 2. local opt = global opt 3. Legendre transform (biconjugacy) 4. separation theorem 5. Fenchel duality hold for L-convex functions ⇒ Part II

40 Bivariate L\- and M\-convex Functions

f(x)

g(p)

p1 x1

L\-convex fn M\-convex fn

41 C4. M-convex Functions

42 M\-convexity from Equi-dist-convexity (Murota 1996, Murota –Shioura 1999) 6 j 6 y y 0 y x0 =⇒ x - - i y y0 x0 x x Equi-distance convex (f : Rn → R): f(x) + f(y) ≥ f(x − α(x − y)) + f(y + α(x − y))

n =⇒ Exchange (f : Z → R) ∀x, y, ∀i : xi > yi f(x) + f(y) ≥ min f(x − ei) + f(y + ei), min {f(x − ei + ej) + f(y + ei − ej)} xj

43 M-convex Function (M = Matroid) f : Zn → R ∪ {+∞} j (Murota 96) y 6 ei: i-th unit vector

- Def: f is M-convex x i ⇐⇒ ∀x, y, ∀i : xi > yi, ∃j : xj < yj: f(x) + f(y) ≥ f(x − ei + ej) + f(y + ei − ej)

dom f ⊆ const-sum hyperplane 6 M \ Mn+1 ' Mn ) Mn

- M\ 44 Gross Substitutes (for set function) f : 2V → R utility (reservation value) function p price vector D(p) = arg max(f −p) = {X | f(X)−p(X) is maximum} demand correspondence

Gross substitutes property: (Kelso–Crawford 82) X ∈ D(p), p ≤ q ⇒ ∃ Y ∈ D(q) : {i ∈ X | pi = qi} ⊆ Y

Equiv. cond. for D(p) (Gul–Stacchetti 99) Equiv. cond. for f (Reijnierse–van Gallekom–Potters 02) & equivalence to M\-concavity (Fujishige–Yang 03)

45 Gross Substitutes for f (not for D(p)) f : 2V → R (set function) f: gross substitutes ⇐⇒ (i) f(S ∪ {i, j}) + f(S) ≤ f(S ∪ {i}) + f(S ∪ {j}) (submodular) (ii) f(S ∪ {i, j}) + f(S ∪ {k}) ≤ max[f(S ∪ {i, k}) + f(S ∪ {j}), f(S ∪ {j, k}) + f(S ∪ {i})]

(Reijnierse–van Gallekom–Potters 02)

cf. Local exchange axiom of M\-concave functions

46 M\-concavity = Gross Substitutes f : Zn → R

M\-concave ⇐⇒ Gross substitutes (+ ∗ ∗)

(Reijnierse–van Gallekom–Potters 02, Fujishige-Yang 03 Danilov-Koshevoy-Lang 03, Murota -Tamura 03) ⇒ Shioura-Tamura’s survey (J. OR. Soc. Japan, 2015) https://www.jstage.jst.go.jp/article/jorsj/58/1/58 61/ pdf

Gross substitutes: (f: utility, p: price) x ∈ arg max(f − p), p ≤ q,

⇒ ∃ y ∈ arg max(f − q) : yi ≥ xi if pi = qi

47 M\-convex Function: Examples X \ Quadratic: f(x) = aijxixj is M -convex

⇔ aij ≥ 0, aij ≥ min(aik, ajk)(∀k 6∈ {i, j})

Min value: f(X) = min{ai | i ∈ X} [unit preference]

Cardinality convex: f(X) = ϕ(|X|) (ϕ: convex) X Separable convex: f(x) = ϕi(xi) (ϕi: convex) X Laminar convex: f(x) = ϕA(x(A)) (ϕA: convex) A {A, B, ....}: laminar ⇔ A ∩ B = ∅ or A ⊆ B or A ⊇ B

48 M\-concave Functions from Matroids

Matroid rank: f(X) = r(X)(rank of X) X Matroid rank sum: f(X) = αiri(X)

ri ← ri+1 (strong quotient), αi ≥ 0 (Shioura 12)

Weighted matroid: w: weight vector

f(X) = max{w(Y ) | Y : indep ⊆ X} (Shioura 12)

Valuated matroid: ω : 2V → R

⇔ ω(X) = f(χX) for some M-concave f

49 Matching / Assignment

UV X = {u1, u2} X = {u1, u2} u1 v1 u1 v1 u1 v1 u2 v2 u2 v2 u2 u3 v3 v3 u4 E M1 M2 Max weight for X ⊆ U (w: given weight) X f(X) = max{ w(e) | M: matching, U ∩ ∂M = X} e∈M

Max-weight func f is M\-concave (cf. Murota 1996) • Proof by augmenting path

Assignment valuation is GS (cf. Hatﬁeld-Milgrom 2005)

50 Polynomial Matrix (Dress-Wenzel 90) Valuated Matroid s + 1 s 1 0 A = ω(J) = deg det A[J] 1 1 1 1 B = {J | J is a base of column vectors}

Grassmann-Pl¨ucker ⇒ Exchange (M-concave) For any J, J0 ∈ B, i ∈ J \ J0, there exists j ∈ J0 \ J s.t. J − i + j ∈ B, J0 + i − j ∈ B, ω(J) + ω(J0) ≤ ω(J − i + j) + ω(J0 + i − j)

Ex. J = {1, 2}, J0 = {3, 4}, i = 1 det A[{1, 2}] = det A[{3, 4}] = 1, ω(J) = ω(J0) = 0 Can take j = 3: J − i + j = {3, 2}, J0 + i − j = {1, 4} ω(J − i + j) = 1, ω(J0 + i − j) = 1

51 Five Properties of “Convex” Functions

1. convex extension 2. local opt = global opt 3. Legendre transform (biconjugacy) 4. separation theorem 5. Fenchel duality hold for M-convex functions ⇒ Part II

52 C5. Remarks on Submodular Set Functions

53 Submodularity & Convexity in 1980’s ρ(X) + ρ(Y ) ≥ ρ(X ∪ Y ) + ρ(X ∩ Y ) • min/max algorithms (Grötschel–Lovász–Schrijver/ Jensen–Korte, Lovász) min ⇒ polynomial, max ⇒ NP-hard

• Convex extension (Lov´asz) set fn is submod ⇔ Lov´aszext is convex

• Duality theorems (Edmonds, Frank, Fujishige) discrete separation, Fenchel min-max

Submodular set functions = Convexity + Discreteness

54 Set Function and Extensions Set function ⇐⇒ Function on {0, 1}n ρ(X) = ρˆ(χX) Every set function ρ : {0, 1}n → R can be extended to convex/concave function

6 convex ext 6 concave ext 1 0 1 0

0 1 - 0 1 - cf. Lov´aszextension 55 Five Properties of “Convex” Functions

1. convex extension 2. local opt = global opt 3. Legendre transform (biconjugacy) 4. separation theorem 5. Fenchel duality hold for submodular set functions (1980’s)

56 Submodular Set Function in DCA • Submodular set fn = L\-convex on {0, 1}n • M\-concave fns form a nice subclass • M\ + M\ are polynomially tractable • Sums of M\-concave fns are still OK

f : {0, 1}n → R submod=L\-convex f : Zn → R submodular M\ + M\ \ M\- L\- M - concave convex concave

57 Dual Character of Matroid Rank Fn

Given a matroid: ρ(X) = max{|I| | I : independent,I ⊆ X} is L\-convex and M\-concave

Polymatroid rank function is NOT M\-concave

58 END