Tropical Geometry, OMT and Neural Network Na Lei1 1DUT-RU International School of Information Sciences and Engineering Dalian University of Technology Geometric Analysis Approach to AI CMSA, Harvard 2019-01-22 Na Lei ([email protected]) (DLUT) Tropical Geometry, OMT and NN 01/22/2019 1 / 62 Thanks Thanks for the invitation. Na Lei ([email protected]) (DLUT) Tropical Geometry, OMT and NN 01/22/2019 2 / 62 Collaborators This is a joint work with Professor David Xianfeng Gu, Zhongxuan Luo, Lianbao Jin, Hao wang and so on. Na Lei ([email protected]) (DLUT) Tropical Geometry, OMT and NN 01/22/2019 3 / 62 Outline 1 Introduction to Tropical Geometry 2 Tropical Geometry for Neural Network 3 Optimal Transport Mapping 4 OMT by Neural Network Na Lei ([email protected]) (DLUT) Tropical Geometry, OMT and NN 01/22/2019 4 / 62 Introduction to Tropical Geometry Na Lei ([email protected]) (DLUT) Tropical Geometry, OMT and NN 01/22/2019 5 / 62 Tropical Algebra Tropical geometry is based on tropical semiring. Tropical semiring 1 Tropical Addition: ; ⊕ 2 Tropical Multiplication: ; 3 Tropical semiring: T := (R ¥ ; ; ). [ {− g ⊕ Tropical semiring is also known as the max-plus algebra. Na Lei ([email protected]) (DLUT) Tropical Geometry, OMT and NN 01/22/2019 6 / 62 Tropical Algebra Tropical operators For x;y R, 2 1 their tropical sum is x y := max x;y ; ⊕ f g 2 their tropical product is x y := x + y; 3 the tropical quotient of x over y is x y := x y. − ¥ x = x = 0 x; ¥ x = ¥: − ⊕ − − ¥: the tropical additive identity, 0: the tropical multiplicative identity. − Associativity, Commutativity, Distributivity. Lack of additive inverse. T := (R ¥ ; ; ) is a semiring. [ {− g ⊕ Na Lei ([email protected]) (DLUT) Tropical Geometry, OMT and NN 01/22/2019 7 / 62 Tropical Algebra Tropical operators For x;y R, 2 1 their tropical sum is x y := max x;y ; ⊕ f g 2 their tropical product is x y := x + y; 3 the tropical quotient of x over y is x y := x y. − ¥ x = x = 0 x; ¥ x = ¥: − ⊕ − − ¥: the tropical additive identity, 0: the tropical multiplicative identity. − Associativity, Commutativity, Distributivity. Lack of additive inverse. T := (R ¥ ; ; ) is a semiring. [ {− g ⊕ Na Lei ([email protected]) (DLUT) Tropical Geometry, OMT and NN 01/22/2019 7 / 62 Tropical Algebra Tropical operators For x;y R, 2 1 their tropical sum is x y := max x;y ; ⊕ f g 2 their tropical product is x y := x + y; 3 the tropical quotient of x over y is x y := x y. − ¥ x = x = 0 x; ¥ x = ¥: − ⊕ − − ¥: the tropical additive identity, 0: the tropical multiplicative identity. − Associativity, Commutativity, Distributivity. Lack of additive inverse. T := (R ¥ ; ; ) is a semiring. [ {− g ⊕ Na Lei ([email protected]) (DLUT) Tropical Geometry, OMT and NN 01/22/2019 7 / 62 Tropical Algebra Tropical operators For x;y R, 2 1 their tropical sum is x y := max x;y ; ⊕ f g 2 their tropical product is x y := x + y; 3 the tropical quotient of x over y is x y := x y. − ¥ x = x = 0 x; ¥ x = ¥: − ⊕ − − ¥: the tropical additive identity, 0: the tropical multiplicative identity. − Associativity, Commutativity, Distributivity. Lack of additive inverse. T := (R ¥ ; ; ) is a semiring. [ {− g ⊕ Na Lei ([email protected]) (DLUT) Tropical Geometry, OMT and NN 01/22/2019 7 / 62 Tropical Algebra Tropical power Let N = n Z : n 0 . For any integer a N, raising x R to the a th power: f 2 ≥ g 2 2 − a x := x x = a x ··· · where the last denotes standard product of real numbers. · For any a N, 2 ( a ¥ if a > 0; ¥ := − − 0 if a = 0: Every x R has a tropical multiplicative inverse given by its standard additive2 inverse: 1 x − := x: − Thus T is in fact a semifield. Na Lei ([email protected]) (DLUT) Tropical Geometry, OMT and NN 01/22/2019 8 / 62 Tropical Algebra Tropical power Let N = n Z : n 0 . For any integer a N, raising x R to the a th power: f 2 ≥ g 2 2 − a x := x x = a x ··· · where the last denotes standard product of real numbers. · For any a N, 2 ( a ¥ if a > 0; ¥ := − − 0 if a = 0: Every x R has a tropical multiplicative inverse given by its standard additive2 inverse: 1 x − := x: − Thus T is in fact a semifield. Na Lei ([email protected]) (DLUT) Tropical Geometry, OMT and NN 01/22/2019 8 / 62 Tropical Algebra One may therefore also raise x R to a negative power a Z by raising its tropical multiplicative2 inverse x to the positive power2 a, i.e., x a = ( x) a. − − − − ¥ does not have a tropical multiplicative inverse and ( ¥) a is − − undefined for a < 0. Na Lei ([email protected]) (DLUT) Tropical Geometry, OMT and NN 01/22/2019 9 / 62 Tropical Algebra One may therefore also raise x R to a negative power a Z by raising its tropical multiplicative2 inverse x to the positive power2 a, i.e., x a = ( x) a. − − − − ¥ does not have a tropical multiplicative inverse and ( ¥) a is − − undefined for a < 0. Na Lei ([email protected]) (DLUT) Tropical Geometry, OMT and NN 01/22/2019 9 / 62 Tropical Algebra Tropical monomial A tropical monomial in d variables x1;:::;xd is an expression of the form a a a ad cx = c x 1 x 2 x 1 2 ··· d d where c R ¥ , a1;:::;ad N, a = (a1;:::;ad ) N and 2 [ {− g 2 2 x = (x1;:::;xd ). Note that xa = 0 xa as 0 is the tropical multiplicative identity. A tropical monomial is indeed a linear function, or equivalently, a hyperplane: a x + a x + + a x + c: 1 1 2 2 ··· d d Na Lei ([email protected]) (DLUT) Tropical Geometry, OMT and NN 01/22/2019 10 / 62 Tropical Algebra Tropical monomial A tropical monomial in d variables x1;:::;xd is an expression of the form a a a ad cx = c x 1 x 2 x 1 2 ··· d d where c R ¥ , a1;:::;ad N, a = (a1;:::;ad ) N and 2 [ {− g 2 2 x = (x1;:::;xd ). Note that xa = 0 xa as 0 is the tropical multiplicative identity. A tropical monomial is indeed a linear function, or equivalently, a hyperplane: a x + a x + + a x + c: 1 1 2 2 ··· d d Na Lei ([email protected]) (DLUT) Tropical Geometry, OMT and NN 01/22/2019 10 / 62 Tropical Algebra Tropical polynomial A tropical polynomial f (x) = f (x1;:::;xd ) is a finite tropical sum of tropical monomials a1 ar f (x) = c x cr x ; 1 ⊕ ··· ⊕ d where ci R ¥ and ai = (ai1; ;aid ) N for i = 1; ;r: 2 [ {− g ··· 2 ··· A monomial of a given multiindex appears at most once in the sum, i.e., a = a for any i = j. i 6 j 6 A tropical polynomial is indeed a piecewise linear function, or equivalently, an upper envelope of the r hyperplanes: maxr a x + a x + + a x + c : i=1f i1 1 i2 2 ··· id d i g Na Lei ([email protected]) (DLUT) Tropical Geometry, OMT and NN 01/22/2019 11 / 62 Tropical Algebra Tropical polynomial A tropical polynomial f (x) = f (x1;:::;xd ) is a finite tropical sum of tropical monomials a1 ar f (x) = c x cr x ; 1 ⊕ ··· ⊕ d where ci R ¥ and ai = (ai1; ;aid ) N for i = 1; ;r: 2 [ {− g ··· 2 ··· A monomial of a given multiindex appears at most once in the sum, i.e., a = a for any i = j. i 6 j 6 A tropical polynomial is indeed a piecewise linear function, or equivalently, an upper envelope of the r hyperplanes: maxr a x + a x + + a x + c : i=1f i1 1 i2 2 ··· id d i g Na Lei ([email protected]) (DLUT) Tropical Geometry, OMT and NN 01/22/2019 11 / 62 Tropical Algebra Tropical rational function A tropical rational function is a standard difference, or, equivalently, a tropical quotient of two tropical polynomials f (x) and g(x): f (x) g(x) = f (x) g(x): − The set of tropical polynomials T[x1;:::;xd ] forms a semiring under the standard extension of and to tropical polynomials, and likewise the ⊕ set of tropical rational functions T(x1;:::;xd ) forms a semifield. We regard a tropical polynomial f = f 0 as a special cases of a tropical rational function and thus T[x1;:::;xd ] T(x1;:::;xd ): ⊆ Na Lei ([email protected]) (DLUT) Tropical Geometry, OMT and NN 01/22/2019 12 / 62 Tropical Algebra Tropical rational function A tropical rational function is a standard difference, or, equivalently, a tropical quotient of two tropical polynomials f (x) and g(x): f (x) g(x) = f (x) g(x): − The set of tropical polynomials T[x1;:::;xd ] forms a semiring under the standard extension of and to tropical polynomials, and likewise the ⊕ set of tropical rational functions T(x1;:::;xd ) forms a semifield. We regard a tropical polynomial f = f 0 as a special cases of a tropical rational function and thus T[x1;:::;xd ] T(x1;:::;xd ): ⊆ Na Lei ([email protected]) (DLUT) Tropical Geometry, OMT and NN 01/22/2019 12 / 62 Tropical Algebra Tropical rational function A tropical rational function is a standard difference, or, equivalently, a tropical quotient of two tropical polynomials f (x) and g(x): f (x) g(x) = f (x) g(x): − The set of tropical polynomials T[x1;:::;xd ] forms a semiring under the standard extension of and to tropical polynomials, and likewise the ⊕ set of tropical rational functions T(x1;:::;xd ) forms a semifield.