Multivariate Quantiles and Ranks using Optimal Transportation Bodhisattva Sen1 Department of Statistics Columbia University, New York Department of Statistics George Mason University Joint work with Promit Ghosal (Columbia University) 05 April, 2019 1Supported by NSF grants DMS-1712822 and AST-1614743 Ranks and quantiles when d = 1 X is a random variable with c.d.f. F Rank: The rank of x R is F (x) 2 Property: If F is continuous, F (X ) Unif([0; 1]) ∼ Quantile: The quantile function is F −1 Property: If F is continuous, F −1(U) F where U Unif([0; 1]) ∼ ∼ How to define ranks and quantiles in Rd , d > 1? Quantile: The quantile function is F −1 Property: If F is continuous, F −1(U) F where U Unif([0; 1]) ∼ ∼ How to define ranks and quantiles in Rd , d > 1? Ranks and quantiles when d = 1 X is a random variable with c.d.f. F Rank: The rank of x R is F (x) 2 Property: If F is continuous, F (X ) Unif([0; 1]) ∼ How to define ranks and quantiles in Rd , d > 1? Ranks and quantiles when d = 1 X is a random variable with c.d.f. F Rank: The rank of x R is F (x) 2 Property: If F is continuous, F (X ) Unif([0; 1]) ∼ Quantile: The quantile function is F −1 Property: If F is continuous, F −1(U) F where U Unif([0; 1]) ∼ ∼ Many notions of multivariate quantiles/ranks have been suggested: Puri and Sen (1971), Chaudhuri and Sengupta (1993), M¨ott¨onenand Oja (1995), Chaudhuri (1996), Liu and Singh (1993), Serfling (2010) ... Spatial median and geometric quantile Spatial median: M := arg min E X m d m2R k − k Quantile when d = 1: For u (0; 1), 2 −1 F (u) = arg min E X x (2u 1)x x2R j − j − − h i Geometric quantile [Chaudhuri (1996)]: For u < 1, let k k Q(u) := arg min E X x u; x x2 d k − k − h i R h i Defining quantiles, ranks, depth, etc. difficult when d > 1 Lack of a natural ordering in Rd , when d > 1 Spatial median and geometric quantile Spatial median: M := arg min E X m d m2R k − k Quantile when d = 1: For u (0; 1), 2 −1 F (u) = arg min E X x (2u 1)x x2R j − j − − h i Geometric quantile [Chaudhuri (1996)]: For u < 1, let k k Q(u) := arg min E X x u; x x2 d k − k − h i R h i Defining quantiles, ranks, depth, etc. difficult when d > 1 Lack of a natural ordering in Rd , when d > 1 Many notions of multivariate quantiles/ranks have been suggested: Puri and Sen (1971), Chaudhuri and Sengupta (1993), M¨ott¨onenand Oja (1995), Chaudhuri (1996), Liu and Singh (1993), Serfling (2010) ... Quantile when d = 1: For u (0; 1), 2 −1 F (u) = arg min E X x (2u 1)x x2R j − j − − h i Geometric quantile [Chaudhuri (1996)]: For u < 1, let k k Q(u) := arg min E X x u; x x2 d k − k − h i R h i Defining quantiles, ranks, depth, etc. difficult when d > 1 Lack of a natural ordering in Rd , when d > 1 Many notions of multivariate quantiles/ranks have been suggested: Puri and Sen (1971), Chaudhuri and Sengupta (1993), M¨ott¨onenand Oja (1995), Chaudhuri (1996), Liu and Singh (1993), Serfling (2010) ... Spatial median and geometric quantile Spatial median: M := arg min E X m d m2R k − k Geometric quantile [Chaudhuri (1996)]: For u < 1, let k k Q(u) := arg min E X x u; x x2 d k − k − h i R h i Defining quantiles, ranks, depth, etc. difficult when d > 1 Lack of a natural ordering in Rd , when d > 1 Many notions of multivariate quantiles/ranks have been suggested: Puri and Sen (1971), Chaudhuri and Sengupta (1993), M¨ott¨onenand Oja (1995), Chaudhuri (1996), Liu and Singh (1993), Serfling (2010) ... Spatial median and geometric quantile Spatial median: M := arg min E X m d m2R k − k Quantile when d = 1: For u (0; 1), 2 −1 F (u) = arg min E X x (2u 1)x x2R j − j − − h i Defining quantiles, ranks, depth, etc. difficult when d > 1 Lack of a natural ordering in Rd , when d > 1 Many notions of multivariate quantiles/ranks have been suggested: Puri and Sen (1971), Chaudhuri and Sengupta (1993), M¨ott¨onenand Oja (1995), Chaudhuri (1996), Liu and Singh (1993), Serfling (2010) ... Spatial median and geometric quantile Spatial median: M := arg min E X m d m2R k − k Quantile when d = 1: For u (0; 1), 2 −1 F (u) = arg min E X x (2u 1)x x2R j − j − − h i Geometric quantile [Chaudhuri (1996)]: For u < 1, let k k Q(u) := arg min E X x u; x x2 d k − k − h i R h i Outline 1 Introduction to Optimal Transportation Monge's Problem Kantorovich Relaxation: Primal Problem A Geometric Approach 2 Quantile and Rank Functions in Rd (d 1) ≥ 3 Some Applications is Statistics Two-sample Goodness-of-fit Testing Independence Testing Outline 1 Introduction to Optimal Transportation Monge's Problem Kantorovich Relaxation: Primal Problem A Geometric Approach 2 Quantile and Rank Functions in Rd (d 1) ≥ 3 Some Applications is Statistics Two-sample Goodness-of-fit Testing Independence Testing Goal: inf Eµ[c(X ; T (X ))] T :T (X )∼ν µ (on ) and ν (on ) probability measures, R dµ(x) = R dν(y) = 1 X Y X Y c(x; y) 0: cost of transporting x to y (e.g., c(x; y) = x y p) ≥ k − k T transports µ to ν, i.e., T (X ) ν where X µ, or, ∼ ∼ ν(B) = µ(T −1(B))= dµ, B −1 ⊂ Y ZT (B) Monge Problem What’s the cheapest way to transport a pile of sand to cover a Gaspard Mongesinkhole? (1781): What is the cheapest way to transport a pile of sand to cover a sinkhole? Blanchet (Columbia U. and Stanford U.) 5/60 Monge Problem What’s the cheapest way to transport a pile of sand to cover a Gaspard Mongesinkhole? (1781): What is the cheapest way to transport a pile of sand to cover a sinkhole? Goal: inf Eµ[c(X ; T (X ))] Blanchet (Columbia U.T and:T Stanford(X )∼ U.)ν 5/60 µ (on ) and ν (on ) probability measures, R dµ(x) = R dν(y) = 1 X Y X Y c(x; y) 0: cost of transporting x to y (e.g., c(x; y) = x y p) ≥ k − k T transports µ to ν, i.e., T (X ) ν where X µ, or, ∼ ∼ ν(B) = µ(T −1(B))= dµ, B −1 ⊂ Y ZT (B) Figure 3: Two densities p and q and the optimal transport map to that morphs p into q. where p 1. When p =1thisisalsocalledtheEarth Mover distance. The minimizer J ⇤ (which does≥ exist) is called the optimal transport plan or the optimal coupling.Incasethere is an optimal transport map T then J is a singular measure with all its mass on the set (x, T (x)) . { } It can be shown that W p(P, Q)=sup (y)dQ(y) φ(x)dP (x) p − ,φ Z Z where (y) φ(x) x y p.Thisiscalledthedualformulation.Inspecialcasewhere p = 1 we have− the very|| simple− || representation W (P, Q) = sup f(x)dP (x) f(x)dQ(x): f 1 − 2F (Z Z ) where denotes all maps from Rd to R such that f(y) f(x) x y for all x, y. F | − ||| − || When d =1,thedistancehasaclosedform: 1 1/p 1 1 p W (P, Q)= F − (z) G− (z) p | − | ✓Z0 ◆ 4 One-dimensional optimal transport Suppose ; R; µ, ν abs. cont.; Fµ and Fν c.d.f.'s X Y ⊂ Goals: (i) Transport µ to ν; i.e., find T s.t. if X µ then T (X ) ν ∼ ∼ 2 2 (ii) T minimizes cost Eµ[(X T (X )) ]; assume c(x; y) = (x y) − − Figure 3: Two densities p and q and the optimal transport map to that morphs p into q. where p 1. When p =1thisisalsocalledtheEarth Mover distance. The minimizer J ⇤ (which does≥ exist) is called the optimal transport plan or the optimal coupling.Incasethere is an optimal transport map T then J is a singular measure with all its mass on the set (x, T (x)) . { } It can be shown that W p(P, Q)=sup (y)dQ(y) φ(x)dP (x) p − ,φ Z Z where (y) φ(x) x y p.Thisiscalledthedualformulation.Inspecialcasewhere p = 1 we have− the very|| simple− || representation W (P, Q) = sup f(x)dP (x) f(x)dQ(x): f 1 − 2F (Z Z ) where denotes all maps from Rd to R such that f(y) f(x) x y for all x, y. F | − ||| − || When d =1,thedistancehasaclosedform: 1 1/p 1 1 p W (P, Q)= F − (z) G− (z) p | − | ✓Z0 ◆ 4 One-dimensional optimal transport Suppose ; R; µ, ν abs. cont.; Fµ and Fν c.d.f.'s X Y ⊂ Goals: (i) Transport µ to ν; i.e., find T s.t. if X µ then T (X ) ν ∼ ∼ 2 2 (ii) T minimizes cost Eµ[(X T (X )) ]; assume c(x; y) = (x y) − − This means that if x > x then T (x ) T (x ) 1 0 1 ≥ 0 So T must be a monotone nondecreasing function Therefore, choose T ( ) so that (recall: ν(B) = dµ) · T −1(B) x T (x) R dµ(x) = dν(y) Fµ(x) = Fν (T (x)) ) Z−∞ Z−∞ −1 Thus, T = F Fµ (and this map T is unique) ν ◦ Figure 3: Two densities p and q and the optimal transport map to that morphs p into q.