Calculus with Convex Sets in a Nutshell

Christian Schilling

April 2, 2019 Preliminaries Basic convex sets Set operations Advanced convex sets Support function Conclusion

Overview

Preliminaries

Basic convex sets

Set operations

Advanced convex sets

Support function

Conclusion

1 / 25 Preliminaries Basic convex sets Set operations Advanced convex sets Support function Conclusion

Overview

Preliminaries

Basic convex sets

Set operations

Advanced convex sets

Support function

Conclusion

2 / 25 2.0 2.0

1.5 1.5

1.0 1.0

0.5 0.5

0.0 0.0 -1.0 -0.5 0.0 0.5 1.0 0.0 0.5 1.0 1.5 2.0 Convex Not convex

Preliminaries Basic convex sets Set operations Advanced convex sets Support function Conclusion

Convex sets

n • We consider the vector space R

Deﬁnition (Convex set) X is convex if X = {λ · ~x + (1 − λ) · ~y | ~x,~y ∈ X, λ ∈ [0, 1] ⊆ R}

3 / 25 Preliminaries Basic convex sets Set operations Advanced convex sets Support function Conclusion

Convex sets

n • We consider the vector space R

Deﬁnition (Convex set) X is convex if X = {λ · ~x + (1 − λ) · ~y | ~x,~y ∈ X, λ ∈ [0, 1] ⊆ R}

2.0 2.0

1.5 1.5

1.0 1.0

0.5 0.5

0.0 0.0 -1.0 -0.5 0.0 0.5 1.0 0.0 0.5 1.0 1.5 2.0 Convex Not convex

3 / 25 Deﬁnition (Bounded set) X is bounded if ∃δ ∈ R ∀~x,~y ∈ X : kx − yk ≤ δ Deﬁnition (Compact set) A set is compact if it is closed and bounded

Preliminaries Basic convex sets Set operations Advanced convex sets Support function Conclusion

Compact sets

Deﬁnition (Closed set) A set is closed if it contains all its boundary points

0 1 2 3 0 1 2 3 1 1 1 ≤ x1 ≤ 2 (⊆ R ) 1 < x1 < 2 (⊆ R )

4 / 25 Deﬁnition (Compact set) A set is compact if it is closed and bounded

Preliminaries Basic convex sets Set operations Advanced convex sets Support function Conclusion

Compact sets

Deﬁnition (Closed set) A set is closed if it contains all its boundary points Deﬁnition (Bounded set) X is bounded if ∃δ ∈ R ∀~x,~y ∈ X : kx − yk ≤ δ

0 1 2 3 0 1 2 3 1 1 1 ≤ x1 ≤ 2 (⊆ R ) 1 ≤ x1 (⊆ R )

4 / 25 Preliminaries Basic convex sets Set operations Advanced convex sets Support function Conclusion

Compact sets

Definition (Closed set) A set is closed if it contains all its boundary points Definition (Bounded set) X is bounded if ∃δ ∈ R ∀~x,~y ∈ X : kx − yk ≤ δ Definition (Compact set) A set is compact if it is closed and bounded

4 / 25 Preliminaries Basic convex sets Set operations Advanced convex sets Support function Conclusion

Overview

Preliminaries

Basic convex sets

Set operations

Advanced convex sets

Support function

Conclusion

5 / 25 Preliminaries Basic convex sets Set operations Advanced convex sets Support function Conclusion

Simplest examples

2 2

1 1

0 0 0 1 2 0 1 2 Empty set Singleton

6 / 25 Unit ball in ∞-norm Unit ball in 2-norm Unit ball in 1-norm aka hypercube aka hypersphere aka cross-polytope

Deﬁnition (p-norm) pp p p k~x = (x1,..., xn)kp := |x1| + ··· + |xn| . • Balls in the p-norm are convex for p ≥ 1.

Preliminaries Basic convex sets Set operations Advanced convex sets Support function Conclusion

More examples

1.0 1.0 1.0

0.5 0.5 0.5

0.0 0.0 0.0

-0.5 -0.5 -0.5

-1.0 -1.0 -1.0 -1.0 -0.5 0.0 0.5 1.0 -1.0 -0.5 0.0 0.5 1.0 -1.0 -0.5 0.0 0.5 1.0

7 / 25 Preliminaries Basic convex sets Set operations Advanced convex sets Support function Conclusion

Unit balls

1.0 1.0 1.0

0.5 0.5 0.5

0.0 0.0 0.0

-0.5 -0.5 -0.5

-1.0 -1.0 -1.0 -1.0 -0.5 0.0 0.5 1.0 -1.0 -0.5 0.0 0.5 1.0 -1.0 -0.5 0.0 0.5 1.0 Unit ball in ∞-norm Unit ball in 2-norm Unit ball in 1-norm aka hypercube aka hypersphere aka cross-polytope

Deﬁnition (p-norm) pp p p k~x = (x1,..., xn)kp := |x1| + ··· + |xn| . • Balls in the p-norm are convex for p ≥ 1.

7 / 25 Preliminaries Basic convex sets Set operations Advanced convex sets Support function Conclusion

Unit balls

1.0 1.0 1.0

0.5 0.5 0.5

0.0 0.0 0.0

-0.5 -0.5 -0.5

-1.0 -1.0 -1.0 -1.0 -0.5 0.0 0.5 1.0 -1.0 -0.5 0.0 0.5 1.0 -1.0 -0.5 0.0 0.5 1.0 Unit ball in 3-norm Unit ball in 42-norm Unit ball in (π − 2)-norm

Deﬁnition (p-norm) pp p p k~x = (x1,..., xn)kp := |x1| + ··· + |xn| . • Balls in the p-norm are convex for p ≥ 1.

7 / 25 Preliminaries Basic convex sets Set operations Advanced convex sets Support function Conclusion

Unit balls

https://commons.wikimedia. org/wiki/File:Astroid.svg Unit ball in 2/3-norm (not convex!) Deﬁnition (p-norm) pp p p k~x = (x1,..., xn)kp := |x1| + ··· + |xn| . • Balls in the p-norm are convex for p ≥ 1.

7 / 25 Preliminaries Basic convex sets Set operations Advanced convex sets Support function Conclusion

Unbounded sets

1.5 1.5

1.0 1.0

0.5 0.5

0.0 0.0

-0.5 -0.5 -0.5 0.0 0.5 1.0 1.5 -0.5 0.0 0.5 1.0 1.5 Hyperplane h~d,~xi = c Half-space h~d,~xi ≤ c

8 / 25 Preliminaries Basic convex sets Set operations Advanced convex sets Support function Conclusion

Overview

Preliminaries

Basic convex sets

Set operations

Advanced convex sets

Support function

Conclusion

9 / 25 Preliminaries Basic convex sets Set operations Advanced convex sets Support function Conclusion

Minkowski sum

Deﬁnition X ⊕ Y := {~x + ~y | ~x ∈ X,~y ∈ Y }

10 / 25 Preliminaries Basic convex sets Set operations Advanced convex sets Support function Conclusion

Minkowski sum

Deﬁnition X ⊕ Y := {~x + ~y | ~x ∈ X,~y ∈ Y }

2.5

2.0

1.5

1.0

0.5

0.0 0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5

Translation

10 / 25 Preliminaries Basic convex sets Set operations Advanced convex sets Support function Conclusion

Minkowski sum

Deﬁnition X ⊕ Y := {~x + ~y | ~x ∈ X,~y ∈ Y }

1.00

0.75

0.50

0.25

0.00 0.0 0.5 1.0 1.5

Square ⊕ circle centered in the origin

10 / 25 Preliminaries Basic convex sets Set operations Advanced convex sets Support function Conclusion

Linear map

Deﬁnition M · X := {M · ~x | ~x ∈ X}

11 / 25 Preliminaries Basic convex sets Set operations Advanced convex sets Support function Conclusion

Linear map

Deﬁnition M · X := {M · ~x | ~x ∈ X}

3.5

  3.0 2 0.5 M =   2.5 0 1.5 2.0

1.5 1 2 3 4

Invertible map

11 / 25 Preliminaries Basic convex sets Set operations Advanced convex sets Support function Conclusion

Linear map

Deﬁnition M · X := {M · ~x | ~x ∈ X}

2.4

  2.2 0 0 M =   2.0 0 1 1.8

1.6

0.0 0.5 1.0 1.5 Projection

11 / 25 Preliminaries Basic convex sets Set operations Advanced convex sets Support function Conclusion

Convex hull

Deﬁnition CH(X) := smallest set Y s.t. Y = X ∪ {λ · ~x + (1 − λ) · ~y | ~x,~y ∈ Y , λ ∈ [0, 1] ⊆ R}

12 / 25 Preliminaries Basic convex sets Set operations Advanced convex sets Support function Conclusion

Convex hull

Deﬁnition CH(X) := smallest set Y s.t. Y = X ∪ {λ · ~x + (1 − λ) · ~y | ~x,~y ∈ Y , λ ∈ [0, 1] ⊆ R}

3.5

3.0

2.5

2.0

1.5

1.0 0.0 0.5 1.0 1.5 2.0 2.5 Union (not convex)

12 / 25 Preliminaries Basic convex sets Set operations Advanced convex sets Support function Conclusion

Convex hull

Deﬁnition CH(X) := smallest set Y s.t. Y = X ∪ {λ · ~x + (1 − λ) · ~y | ~x,~y ∈ Y , λ ∈ [0, 1] ⊆ R}

3.5

3.0

2.5

2.0

1.5

1.0 0.0 0.5 1.0 1.5 2.0 2.5 Convex hull of the union

12 / 25 Preliminaries Basic convex sets Set operations Advanced convex sets Support function Conclusion

Intersection

Deﬁnition X ∩ Y := {~x | ~x ∈ X ∧ ~x ∈ Y }

13 / 25 Preliminaries Basic convex sets Set operations Advanced convex sets Support function Conclusion

Intersection

Deﬁnition X ∩ Y := {~x | ~x ∈ X ∧ ~x ∈ Y }

-1 -1 0 1 2 3 4 Disjoint

13 / 25 Preliminaries Basic convex sets Set operations Advanced convex sets Support function Conclusion

Intersection

Deﬁnition X ∩ Y := {~x | ~x ∈ X ∧ ~x ∈ Y }

-1 -1 0 1 2 3 Intersecting

13 / 25 Preliminaries Basic convex sets Set operations Advanced convex sets Support function Conclusion

Overview

Preliminaries

Basic convex sets

Set operations

Advanced convex sets

Support function

Conclusion

14 / 25 Preliminaries Basic convex sets Set operations Advanced convex sets Support function Conclusion

Generalizations of balls

1.5 1.5

1.0 1.0

0.5 0.5

0.0 0.0

-0.5 -0.5

-1.0 -1.0

-1.5 -1.5 -1.0 -0.5 0.0 0.5 1.0 -1.0 -0.5 0.0 0.5 1.0 Hyperrectangle (Hyper-)Ellipsoid

15 / 25 Preliminaries Basic convex sets Set operations Advanced convex sets Support function Conclusion

Polytopes Deﬁnition (Vertex representation) A polytope is the convex hull of the union of ﬁnitely many singletons

2.5

2.0

1.5

1.0

0.5

0.0

-0.5 -1.5 -1.0 -0.5 0.0 0.5 1.0 1.5

16 / 25 Preliminaries Basic convex sets Set operations Advanced convex sets Support function Conclusion

Polytopes Definition (Vertex representation) A polytope is the convex hull of the union of finitely many singletons Definition (Constraint (or half-space) representation) A polytope is the bounded intersection of finitely many half-spaces

2.5

2.0

1.5

1.0

0.5

0.0

-0.5 -1.5 -1.0 -0.5 0.0 0.5 1.0 1.5

16 / 25 Preliminaries Basic convex sets Set operations Advanced convex sets Support function Conclusion

2.5

2.0

1.5

1.0

0.5

0.0

-0.5 -1.5 -1.0 -0.5 0.0 0.5 1.0 1.5 Polyhedron (unbounded)

16 / 25 1.0

0.5

0.0

-0.5

-1.0

-1.5 -1.0 -0.5 0.0 0.5 1.0 1.5

• Centrally symmetric polytope

Preliminaries Basic convex sets Set operations Advanced convex sets Support function Conclusion

Zonotopes • Minkowski sum of line segments ( p )

X ~c + ξi · ~gi ξi ∈ [−1, 1] i=1

17 / 25 Preliminaries Basic convex sets Set operations Advanced convex sets Support function Conclusion

Zonotopes • Minkowski sum of line segments ( p )

X ~c + ξi · ~gi ξi ∈ [−1, 1] i=1

1.0

0.5

0.0

-0.5

-1.0

-1.5 -1.0 -0.5 0.0 0.5 1.0 1.5

• Centrally symmetric polytope

17 / 25 Preliminaries Basic convex sets Set operations Advanced convex sets Support function Conclusion

Closure properties

X ⊕ Y M · X CH(X ∪ Y ) X ∩ Y Hyperrectangle 1 Ellipsoid , / / , Zonotope / , / / Polytope , , / / , , , ,

1Unless the intersection is empty 18 / 25 Preliminaries Basic convex sets Set operations Advanced convex sets Support function Conclusion

Overview

Preliminaries

Basic convex sets

Set operations

Advanced convex sets

Support function

Conclusion

19 / 25 ~d ~d

~d x2 X

x1 2 ~ ~ {~x ∈ R : hd,~xi ≤ ρX (d)}

Preliminaries Basic convex sets Set operations Advanced convex sets Support function Conclusion

Support function

Deﬁnition (Support function) n n Let ∅ ( X ⊆ R be a compact convex set and ~d ∈ R a direction

n ρX : R → R ~ ~ ρX (d) := maxhd,~xi ~x∈X

20 / 25 ~d ~d

Preliminaries Basic convex sets Set operations Advanced convex sets Support function Conclusion

Support function

Deﬁnition (Support function) n n Let ∅ ( X ⊆ R be a compact convex set and ~d ∈ R a direction

n ρX : R → R ~ ~ ρX (d) := maxhd,~xi ~x∈X

~d x2 X

x1 2 ~ ~ {~x ∈ R : hd,~xi ≤ ρX (d)}

20 / 25 Preliminaries Basic convex sets Set operations Advanced convex sets Support function Conclusion

Support function

Deﬁnition (Support function) n n Let ∅ ( X ⊆ R be a compact convex set and ~d ∈ R a direction

n ρX : R → R ~ ~ ρX (d) := maxhd,~xi ~x∈X

~d x2 ~d X ~d x1 2 ~ ~ {~x ∈ R : hd,~xi ≤ ρX (d)}

20 / 25 Preliminaries Basic convex sets Set operations Advanced convex sets Support function Conclusion

Properties of the support function Proposition ~ ~ • ρλ·X (d) = ρX (λ · d) ~ ~ ~ • ρX⊕Y (d) = ρX (d) + ρY (d) ~ T ~ • ρM·X (d) = ρX (M · d) ~ ~ ~ • ρCH(X∪Y )(d) = max(ρX (d), ρY (d))

Proposition n For every compact convex set X 6= ∅ and D ⊆ R we have \ ~ ~ X ⊆ hd,~xi ≤ ρX (d) ~d∈D

n and equality holds for D = R

21 / 25 Preliminaries Basic convex sets Set operations Advanced convex sets Support function Conclusion

Complexity

• Optimization of a linear function over a convex set → convex optimization (eﬃcient!)

• Even more eﬃcient for speciﬁc set representations • Polytopes: linear program • Zonotopes: O(p · n2)(p generators) • Ellipsoids: O(n2) • Hyperrectangles: O(n)

• Let c(X) be the complexity for set X • X ⊕ Y : O(c(X) + c(Y )) • M · X: O(n2 + c(X)) • CH(X, Y ): O(c(X) + c(Y ))

22 / 25 • Template directions

• ε-close approximation

Preliminaries Basic convex sets Set operations Advanced convex sets Support function Conclusion

Overapproximation using support function

1.0

0.5

0.0

-0.5

-1.0

-1.0 -0.5 0.0 0.5 1.0

23 / 25 • Template directions

• ε-close approximation

Preliminaries Basic convex sets Set operations Advanced convex sets Support function Conclusion

Overapproximation using support function

1.0

0.5

0.0

-0.5

-1.0

-1.0 -0.5 0.0 0.5 1.0

23 / 25 • Template directions

• ε-close approximation

Preliminaries Basic convex sets Set operations Advanced convex sets Support function Conclusion

Overapproximation using support function

1.0

0.5

0.0

-0.5

-1.0

-1.0 -0.5 0.0 0.5 1.0

23 / 25 Preliminaries Basic convex sets Set operations Advanced convex sets Support function Conclusion

Overapproximation using support function

1.0

0.5

0.0

-0.5

-1.0

-1.0 -0.5 0.0 0.5 1.0

• Template directions

• ε-close approximation

23 / 25 Preliminaries Basic convex sets Set operations Advanced convex sets Support function Conclusion

Overview

Preliminaries

Basic convex sets

Set operations

Advanced convex sets

Support function

Conclusion

24 / 25 Preliminaries Basic convex sets Set operations Advanced convex sets Support function Conclusion

Conclusion

• Convex sets are expressive

• Closure under most standard set operations

• Support function allows for eﬃcient lazy computations

• Non-convex sets: approximate by (union of) convex sets

• Implemented in the Julia package LazySets (joint work with Marcelo Forets from Universidad de la Republica,´ Uruguay) and available on Github

25 / 25