Depth, Outlyingness, Quantile, and Rank Functions in Multivariate & Other Data Settings Eserved@D = *@Let@Token

DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS – CONCEPTS, PERSPECTIVES, CHALLENGES

Depth, Outlyingness, Quantile, and Rank Functions in Multivariate & Other Data Settings

Robert Serﬂing1 Serﬂing & Thompson Statistical Consulting and Tutoring

ASA Alabama-Mississippi Chapter Mini-Conference University of Mississippi, Oxford April 5, 2019

1 www.utdallas.edu/∼serfling DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS – CONCEPTS, PERSPECTIVES, CHALLENGES

“Don’t walk in front of me, I may not follow. Don’t walk behind me, I may not lead. Just walk beside me and be my friend.” – Albert Camus DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS – CONCEPTS, PERSPECTIVES, CHALLENGES OUTLINE

Depth, Outlyingness, Quantile, and Rank Functions on Rd

Depth Functions on Arbitrary Data Space X

Depth Functions on Arbitrary Parameter Space Θ

Concluding Remarks DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS – CONCEPTS, PERSPECTIVES, CHALLENGES d DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS ON R PRELIMINARY PERSPECTIVES Depth functions are a nonparametric approach

I The setting is nonparametric data analysis. No parametric or semiparametric model is assumed or invoked.

I We exhibit the geometric structure of a data set in terms of a center, quantile levels, measures of outlyingness for each point, and identiﬁcation of outliers or outlier regions.

I Such data description is developed in terms of a depth function that measures centrality from a global viewpoint and yields center-outward ordering of data points.

I This diﬀers from the density function, which measures local probability mass.

I This extends the univariate median, quantiles, ranks, and outlyingness functions.

I This also yields depth-based inference procedures. DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS – CONCEPTS, PERSPECTIVES, CHALLENGES d DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS ON R PRELIMINARY PERSPECTIVES

Depth functions on Rd must satisfy essential criteria

I Two criteria are of paramount importance:

I Invariance under aﬃne transformation to new coordinates,

I Robustness against inﬂuence of discordant data points.

I A proposed “depth function” deﬁcient with respect to these criteria is not acceptable for nonparametric description of a data cloud in terms of center, quantiles, outlyingness, and ranks. DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS – CONCEPTS, PERSPECTIVES, CHALLENGES d DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS ON R PRELIMINARY PERSPECTIVES Depth functions are neither a parametric approach nor a semiparametric approach

I In parametric or semiparametric inference, with primary goals of model-ﬁtting and model-testing, the criteria of invariance and robustness become seriously compromised against the key and overriding criterion of high eﬃciency.

I Proposed depth functions compromising aﬃne invariance and robustness cannot be advanced on the basis that their associated rank tests are eﬃcient in some parametric or semiparametric setting.

I The fundamental role of depth functions is nonparametric centrality-based data description and ordering. DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS – CONCEPTS, PERSPECTIVES, CHALLENGES d DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS ON R PRELIMINARY PERSPECTIVES A depth function in the multivariate setting should properly extend the univariate case.

I In the univariate setting, depth (centrality) has been only implicit, with emphasis instead on equivalent outlyingness.

I Extensions of univariate median, quantiles, outlyingness, and ranks to the multivariate setting should reduce to these when the dimension is 1.

I This is a necessary condition for a proposed depth function.

I However, this condition is not suﬃcient as validation of a proposed depth function. DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS – CONCEPTS, PERSPECTIVES, CHALLENGES d DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS ON R GENERAL NOTION OF DEPTH FUNCTION Depth functions – the general notion

I Relative to distribution F on space X , a depth function D(x, F ) provides a center-outward ordering of x in X .

I Higher depth represents greater “centrality”.

I Maximum depth points deﬁne “center” or “median”.

I Nested contours of equal depth enclose the median.

I Orientation to a “center” compensates for the typical lack of a linear order in X .

I For a sample Xn = {X1,..., Xn} in X following distribution F , some sample version D(x, Xn) is constructed. I Implementation of this simple notion for specific choices of X poses interesting challenges. d I For X = R , some general treatments: Liu, Parelius, and Singh (1999), Zuo and Serfling (2000), Liu, Souvaine, and Serfling (2006). DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS – CONCEPTS, PERSPECTIVES, CHALLENGES d DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS ON R GENERAL NOTION OF DEPTH FUNCTION Centrality likes symmetry

I Notions of symmetry yield notions of “center”.

I “Centers” provide reference points for “centrality”.

I If F is symmetric about θ, then we might require of a depth function D(x, F ):

I D(x, F ) is maximal at θ.

I D(X, F ) is symmetric about θ.

I D(x, F ) decreases along rays from θ. d I In R , we have a hierarchy of notions of symmetry : spherical ≺ elliptical ≺ central ≺ angular ≺ halfspace DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS – CONCEPTS, PERSPECTIVES, CHALLENGES d DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS ON R GENERAL NOTION OF DEPTH FUNCTION Centrality ignores multimodality

d I Depth functions on R measure centrality without regard to multimodality.

I Is this a virtue or a defect?

I Multimodality might be regarded as the business of density functions rather than depth functions.

I Are we really interested in “multi-centrality”?

I One resolution: Carry out cluster analysis ﬁrst, then carry out depth analysis within clusters. DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS – CONCEPTS, PERSPECTIVES, CHALLENGES d DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS ON R GENERAL NOTION OF DEPTH FUNCTION Depth and density functions are very diﬀerent

I Centrality is a global concept. Thus depth functions are center-oriented in the sense of median-oriented.

I In comparison, density functions provide local measures of probability mass.

I For example, for the uniform distribution on the d-cube, the density is constant, but typical depth functions are not.

I Thus the likelihood function is not a depth function.

I Features such as multimodality and nonconvexity are to be handled via the density function, not the depth function.

I Nevertheless, for an ellipsoidally symmetric distribution, the contours of a depth function should coincide with those of the density function. DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS – CONCEPTS, PERSPECTIVES, CHALLENGES d DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS ON R d DEPTH FUNCTIONS ON R The story began formally with Tukey (1975) d I The “Tukey” or “halfspace” depth on X = R is d D(x, F ) = inf{P(H): x ∈ H closed halfspace}, x ∈ R , the minimal probability attached to any halfspace with x on the boundary.

I Antecedents: Hotelling (1929) I Can use other classes besides halfspaces (Small, 1987) I Aﬃne invariant I Robust sample versions I However,

I Computationally intensive

I Complicated asymptotics

I Exhibits some undesirable anomalous behavior (Dutta, Ghosh, and Chaudhuri, 2011) DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS – CONCEPTS, PERSPECTIVES, CHALLENGES d DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS ON R d DEPTH FUNCTIONS ON R Measuring centrality by halfspace depth

Halfspace Depth Function

0.5

0.4

0.3

0.2

0.1

0 0 0 0.2 0.2 0.4 0.4 y 0.6 0.6 x 0.8 0.8 1 1

Halfspace depth for F uniform on [0, 1]2. For comparison, the Halfspace Depth Function for Uniform Distribution on [0, 1]2 density function for this distribution is constant. DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS – CONCEPTS, PERSPECTIVES, CHALLENGES d DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS ON R d DEPTH FUNCTIONS ON R The story advanced ... d I The simplicial depth (Liu, 1988, 1990) on R is d D(x, F ) = P(x ∈ S[X1,..., Xd+1]), x ∈ R , d for S[X1,..., Xd+1] a simplex in R having independent observations X1,..., Xd+1 from F as vertices.

I For d = 2: probability that a random triangle covers x

I Can use other shapes besides simplices

I The sample version is a U-statistic pointwise in x I Aﬃne invariant I However,

I Poor robustness (a U-statistic is an average)

I Computational burden increases with d I With the introduction of this depth by Regina Liu, it was realized that “depth” is a general concept with many quite diﬀerent implementations. This spawned an “industry”! DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS – CONCEPTS, PERSPECTIVES, CHALLENGES d DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS ON R d DEPTH FUNCTIONS ON R

The “spatial” (or “L1”) depth d I The spatial depth (Vardi and Zhang, 2000) on R is d D(x, F ) = 1 − kES(x − X)k, x ∈ R , where S(y) = y/kyk (= 0 if y = 0) is the sign function on Rd and k · k is the Euclidean norm. I Only orthogonally invariant I The sample version involves basically a sum, n X S(x − Xi ), 1 so it is easily computed and its asymptotic behavior is readily derived via the CLT.

I However, its robustness decreases away from the center. DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS – CONCEPTS, PERSPECTIVES, CHALLENGES d DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS ON R d DEPTH FUNCTIONS ON R Halfspace and spatial depths, F Uniform on [0, 1]2

Halfspace Depth Contours Halfspace Depth Function Halfspace Depth Function

0.8

0.5 0.5

0.4 0.4 0.6

0.3 0.3 y 0.2 0.2

0.4 0.1 0.1

0 0 0 0 0 0

0.2 0.2 0.2 0.2 0.2 0.4 0.4 0.4 0.4

y 0.6 0.6 x y 0.6 0.6 x

0.8 0.8 0.8 0.8

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 1 1 1 x

Spatial Depth Contours Spatial Depth Function Spatial Depth Function

0.8

1 1

0.6 0.8 0.8

0.6 0.6 y

0.4 0.4 0.4

0.2 0.2

0 0 0 0

0.2 0.2 0.2 0.2 0.2

0.4 0.4 0.4 0.4

y 0.6 0.6 x y 0.6 0.6 x

0.8 0.8 0.8 0.8 0 0 0.2 0.4 0.6 0.8 1 1 1 1 x

Upper row: halfspace depth. Lower row: spatial depth. DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS – CONCEPTS, PERSPECTIVES, CHALLENGES d DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS ON R d DEPTH FUNCTIONS ON R The TR spatial depth

I A modiﬁed spatial depth is fully aﬃne invariant:

d D(x, F ) = 1 − kES(M(F )(x − X))k, x ∈ R , with M(F ) any transformation-retransformation (TR) matrix, i.e., an inverse square root of a scatter matrix (Serﬂing, 2010).

I The price: sample versions now involve M(Xn), with

I additional and usually substantial computational burden

I additional robustness issues

I more complicated asymptotics

I This “invariance-computation-robustness trade-oﬀ” poses a central challenge in choosing and using depth functions. DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS – CONCEPTS, PERSPECTIVES, CHALLENGES d DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS ON R d DEPTH FUNCTIONS ON R The Mahalanobis distance depth

I With µ(F ) and Σ(F ) location and scatter measures deﬁned on F , the Mahalanobis distance depth (Liu and Singh, 1993; Zuo and Serﬂing, 2000) is 1 D(x, F ) = , x ∈ d . − 1 R 1 + kΣ(F ) 2 (x − µ(F ))k

I Aﬃne invariant I Computational burden and robustness of D(x, Xn) depend on the choices of µ(Xn) and Σ(Xn). I Contours are necessarily ellipsoidal, regardless of F .

I Reduces to a scaled deviation measure in univariate case. DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS – CONCEPTS, PERSPECTIVES, CHALLENGES d DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS ON R d DEPTH FUNCTIONS ON R The projection depth

I With univariate location and spread measures µ(·) and σ(·), the projection depth (Liu, 1992; Zuo, 2003) is 1 D(x, F ) = , x ∈ d . 0 R u x−µ(Fu0X) 1 + supkuk=1 σ(Fu0X)

I Aﬃne invariant

I Highly robust using µ = Median and σ = MAD. I However,

I computationally intensive due to inﬁnitely many projections

I complicated asymptotics

I The preceding depth functions have convex or at least nearly convex contours even if the population or sample exhibits nonconvex support and/or multimodality.

I Some researchers argue that density and depth functions can and should have complementary roles.

I Some others, who want the depth contours to follow the density contours, have developed notions of “local depth” that possess this feature.

I Chen, Bart, Dang, and Peng (2007)

I Agostinelli and Romanazzi (2007, 2008)

I Paindaveine and Van bever (2013) (in DepthTools R package) DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS – CONCEPTS, PERSPECTIVES, CHALLENGES d DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS ON R d DEPTH FUNCTIONS ON R Sample depths, bivariate normal data

● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●

● ● ● ●

● ● ● ● ● ●

● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●

● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●

● ● ● ● ● ● ● ● ● ● ● ● ● ●

● ●

● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●

● ● ● ●

● ● ● ● ● ●

● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●

● ● ● ● ● ● ● ● ● ● ● ● ● ●

● ●

Upper left: simplicial. Upper right: Mahalanobis. Lower left: spatial. Lower right: projection. DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS – CONCEPTS, PERSPECTIVES, CHALLENGES d DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS ON R d DEPTH FUNCTIONS ON R Sample depths, contaminated bivariate normal data

● ● ● ● ●●●● ● ●●●● ● ● ● ● ● ● ●

● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●

● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ● ●●● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●

● ● ● ● ● ● ● ● ● ● ● ● ● ●

● ●

● ● ● ● ●●●● ● ●●●● ● ● ● ● ● ● ●

● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●

● ● ● ● ● ● ● ● ● ● ● ● ● ●

● ●

Upper: population halfspace depth contours (left), sample of size 100 (right). Lower: sample depth contours (left), sample nonparametric density contours (right). Note: the population density has no contours, of course! DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS – CONCEPTS, PERSPECTIVES, CHALLENGES d DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS ON R d DEPTH FUNCTIONS ON R Depth and density contours, mixture of two bivariate normal distributions

Upper: sample of size 100 (left), sample nonparametric density contours (right). Lower: sample halfspace depth contours (left), sample local depth contours (right). DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS – CONCEPTS, PERSPECTIVES, CHALLENGES d DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS ON R d DEPTH FUNCTIONS ON R Some perspectives (details coming) Depth functions on Rd , d ≥ 2, ... I yield associated outlyingness, quantile, and rank functions d I yield order statistics for a data set Xn in R I pose conceptual challenges – what properties are required?

I pose technical issues for study – invariance, robustness, asymptotics, computational ease, etc.

I are not viewable for d ≥ 3 – we must rely on and trust algorithmic formulations

I provide tools for nonparametric description of a data set and for a variety of inference problems – location, spread, classiﬁcation, regression, shape-ﬁtting, etc.

I comprise a relatively new yet quite natural fundamental methodology in nonparametric multivariate data analysis DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS – CONCEPTS, PERSPECTIVES, CHALLENGES d DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS ON R d DEPTH FUNCTIONS ON R Desirable technical properties of a depth function

I Aﬃne invariance

I Maximality at center of symmetry

I Symmetry of D(x, F )

I Decreasing along rays from center

I Vanishing at inﬁnity

I Continuity, or merely semicontinuity, of D(x, F ) as a function of x (where meaningful for X )

I Continuity of D(x, F ) as a functional of F .

I Quasi-concavity as a function of x.

I And more ... DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS – CONCEPTS, PERSPECTIVES, CHALLENGES d DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS ON R d DEPTH FUNCTIONS ON R Illustrations of what we can compute via depth

I Contours I Depth-order statistics – ordering by depth, center-outward I Depth-weighted location functionals R d x W (D(x, F )) dF (x) RR . d W (D(x), F )) dF (x) R

I Depth-weighted scatter matrix functionals I Scale curves – plot of volumes within contours as a function of probability weight

I Skewness functionals – scaled diﬀerence of two location functionals

I Kurtosis functionals – via transformation of scale curve I And more ... DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS – CONCEPTS, PERSPECTIVES, CHALLENGES d DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS ON R d DEPTH FUNCTIONS ON R Illustrations of depth-based statistical procedures

I Bagplots, sunburst plots Extending boxplot to d = 2, 3, with contours for “middle half”, rays for whiskers, and contours for outlyingness thresholds.

I DD, PP, QQ plots Compare two samples by a plot of depth values of combined sample against each other, or of volumes of sample central regions, or of kurtosis curves, or of depth-based quantiles.

I Comparison of several distributions Plot their scale curves in a single exhibit. Or kurtosis curves.

I Nonparametric description of multivariate distributions Measures of location, spread, asymmetry, kurtosis. DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS – CONCEPTS, PERSPECTIVES, CHALLENGES d DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS ON R d DEPTH FUNCTIONS ON R

I Diagnosis of nonnormality Use trimmed depth-weighted scatter matrix. Or kurtosis curve.

I Testing multivariate symmetry For spherical symmetry, plot fraction of data in smallest sphere containing the pth (probability weight) sample central region. For central symmetry, plot fraction of data within intersection of pth sample central region and its reﬂection.

I Nonparametric outlier identiﬁcation

I P-values in hypothesis testing via bootstrap and data depth

I One- and multi-sample multivariate rank statistics deﬁned on depth-based ranks DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS – CONCEPTS, PERSPECTIVES, CHALLENGES d DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS ON R d DEPTH FUNCTIONS ON R

I Statistical process control procedures Can use depth-based ranks for monitoring and thresholding with multivariate data using univariate quality control procedures. Compare outlyingness of a new observation relative to in-control reference point cloud.

I Multivariate density estimation by probing depth

I Depth-based quality indices

I Depth-based classiﬁcation rules DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS – CONCEPTS, PERSPECTIVES, CHALLENGES d DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS ON R d DEPTH FUNCTIONS ON R Application contexts of depth-based methods

I Depth-based methods provide new competitors for standard approaches having diverse applications:

I Exploratory data analysis

I Multi-sample inference

I Regression

I Classiﬁcation, discrimination

I Directional analysis DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS – CONCEPTS, PERSPECTIVES, CHALLENGES d DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS ON R d DEPTH FUNCTIONS ON R

I Further, in some contexts, depth-based methods happen to be especially natural or advantageous:

I Monitoring of aviation safety data

I Industrial quality control

I Measuring economic disparity and concentration

I Social choice and voting – ordering social choices by voters’ preferences, and ﬁnding the “median voter”

I Game-theoretic analysis of competition

I ... DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS – CONCEPTS, PERSPECTIVES, CHALLENGES d DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS ON R d DEPTH FUNCTIONS ON R Computing depth functions uses computational geometry

d−1 I halfspace and simplicial depth: O(n log n) d−1 I nested convex hulls of fractions of data: O(n log n) 2 I bivariate halfspace contours, bagplot, data depths: O(n ) d I deepest regression hyperplane: O(n ) d−2 I regression depth of k-ﬂat: O(n +n log n), 1 ≤ k ≤ d−2 d+1 I Stahel-Donoho estimators: O(n )

I volume of sample pth central region

I simulation and bootstrap: many samples needed

I issues of NP-hard, NP-complete, coNP-complete, etc. DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS – CONCEPTS, PERSPECTIVES, CHALLENGES d DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS ON R d DEPTH FUNCTIONS ON R Desirable results for sample depth functions

I Consistency:

kD(x, Xn) − D(x, F )k∞ → 0, n → ∞

I Weak convergence:

1/2 n [D(x, Xn) − D(x, F )] converges weakly, n → ∞

I Convergence of sample central regions – e.g., in Hausdorﬀ distance

I Convergence of functionals of sample depth or of sample central regions

I Robustness of functionals of sample depth or of sample central regions – e.g., breakdown points, inﬂuence curves.

I Favorable behavior of bootstrap DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS – CONCEPTS, PERSPECTIVES, CHALLENGES d DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS ON R d OUTLYINGNESS FUNCTIONS ON R Associated outlyingness functions on Rd

I A depth function D(x, F ) decreases as x moves out from the “center”. An equivalent outlyingness function O(x, F ) increases as x moves out from the center.

I The equivalence is deﬁned via “inverse” relationships like O(x, F ) = 1 − D(x, F ) or O(x, F ) = 1/[1 + D(x, F )].

I Typically, we let D(x, F ) and O(x, F ) take values in [0, 1].

I They generate the same order statistics and contours.

I Despite their equivalence, the roles and interpretations of depth and outlyingness functions are very diﬀerent.

I Also, the respective inference and estimation problems are very diﬀerent. Robust estimation of a “center” and robust estimation of an outlyingness threshold are quite diﬀerent problems, both conceptually and analytically. DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS – CONCEPTS, PERSPECTIVES, CHALLENGES d DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS ON R OUTLYINGNESS FUNCTIONS AS A FOCUS Outlier and anomaly detection - who, what, where?

I “Outliers ” have long been under discussion: Francis Bacon, 1620 – Daniel Bernoulli, 1777 – Benjamin Pierce, 1852 – Mosteller and Tukey, 1977 – Barnett and Lewis, 1995 – ...

I Beyond the settings of multivariate and functional data, outlier and anomaly detection has become a key target in data mining, in broad settings involving complex data types and structures and possibly massive data.

I As a highly computational enterprise with diverse ad hoc procedures, outlier and anomaly detection are a part of computer science as much as of statistical science!

I A formal understanding, however, requires robustness and eﬃciency studies – the domain of statistical science with its formal depth-based concepts and methods. DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS – CONCEPTS, PERSPECTIVES, CHALLENGES d DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS ON R OUTLYINGNESS FUNCTIONS AS A FOCUS Robustness of outlier identiﬁers

I In general, we want our statistical procedures to be robust, i.e., not unduly inﬂuenced by outliers.

I An outlier identification method should itself be robust and not unduly influenced by the outliers it seeks to identify. I Key robustness measures for outlier identifiers:

I Masking breakdown point (MBP) - minimal fraction of data which if replaced arbitarily can make arbitrarily extreme outliers become misclassiﬁed as nonoutliers

I Swamping breakdown point (SBP) - minimal fraction of data which if replaced arbitrarily can make arbitarily central nonoutliers become misclassiﬁed as outliers

I Formal theory: Davies and Gather (1993), Becker (1996), Becker and Gather (1999), Dang and Serfling (2010), Serfling and Wang (2014), Wang and Serfling (2015, 2017) DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS – CONCEPTS, PERSPECTIVES, CHALLENGES d DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS ON R OUTLYINGNESS FUNCTIONS AS A FOCUS Representations for MBP and SBP

I It turns out that the MBP and SBP can be represented as simple ordinary breakdown points (BP) of certain very complicated statistics. This holds very generally (Serﬂing and Wang, 2014). I For outlyingness function O(x, F ) on any space X , put out(λ, F ) = {x : O(x, F ) > λ} and let out(λ, Xn) be the sample analogue. Then ! MBP(λ, Xn) = BPexplosion sup O(y, F ) y ∈ out(λ,Xn) SBP(λ, Xn) = BPimplosion inf O(y, F ) y ∈ out(λ,Xn)

d I Results for X = R, R : Wang and Serﬂing (2015, 2017) DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS – CONCEPTS, PERSPECTIVES, CHALLENGES d DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS ON R OUTLYINGNESS FUNCTIONS AS A FOCUS A challenge and an opportunity –

The various ad hoc outlier identifiers arising in the field of “data science” need to be evaluated critically using formal criteria and tools arising in “statistical science”: MBP, SBP, influence functions. DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS – CONCEPTS, PERSPECTIVES, CHALLENGES d DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS ON R d QUANTILE FUNCTIONS ON R Associated median for the distribution F

I A depth function D(·, F ) determines an associated “median” M(F ), the point x of maximal depth.

I Diﬀerent depth functions can induce diﬀerent notions of median M(F ).

I However, if F is symmetric about a point θ0 in some sense, then we want M(F ) to coincide with θ0.

I We also want any such notion of median to reduce to the usual notion in the univariate case.

I The nested contours of equal depth enclose M. DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS – CONCEPTS, PERSPECTIVES, CHALLENGES d DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS ON R d QUANTILE FUNCTIONS ON R Associated quantile functions on Rd

I The contours of equal depth induce a quantile function Q(u, F ) deﬁned on the unit ball Bd .

I A given x has a quantile representation x = Q(u, F ) given by an indexing u ∈ Bd such that u and its magnitude kuk represent in some sense the direction and outlyingness of x from the center.

I In particular, we arrange that kuk = O(x, F ) in [0, 1].

I Various such constructions are possible:

I Example 1. Take u in the direction x − M(F ).

I Example 2. Take u in the expected direction ES(x − X) for X distributed as F . DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS – CONCEPTS, PERSPECTIVES, CHALLENGES d DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS ON R d QUANTILE FUNCTIONS ON R Quantile functions and multivariate L-statistics

I Univariate “L-statistics” are linear functions of the order statistics and also can be represented as weighted integrals of the sample quantile function. Many useful statistics are of this form. I To deﬁne formal multivariate analogues of the univariate L-statistics, the depth-induced order statistics (the data values in order of center-outward outlyingness) can be used. However, although the resulting statistical procedures are useful, these “order statistics” are only nondirectional ranks and lose the geometry of the data cloud. I On the other hand, the depth-induced quantile function retains the geometry and it yields multivariate analogues of the univariate L-statistics in a diﬀerent way, as we examine next. DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS – CONCEPTS, PERSPECTIVES, CHALLENGES d DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS ON R d QUANTILE FUNCTIONS ON R Multivariate L-statistics

I Univariate L-statistics can be expressed in median-oriented form as Z 1 L(Xn) = Q(u, Xn) M(du) −1 with M(·) a signed measure on [−1, 1]. For convenience, consider the case of continuous weighting of quantiles: Z 1 L(Xn) = Q(u, Xn) J(u) du. −1 d I Now extend to R : d 1. Replace the interval [−1, 1] by the unit ball B d 2. Replace Q(u, Xn) by some choice of Q(u, Xn), u ∈ B 3. Deﬁne J(u) in some sense 4. Deﬁne “product” Q(u, Xn)J(u) in some sense DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS – CONCEPTS, PERSPECTIVES, CHALLENGES d DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS ON R d QUANTILE FUNCTIONS ON R Illustrative possibilities for multivariate L-statistics

I Diﬀerent interpretations of Z L(Xn) = Q(u, Xn) J(u) du d B yield diﬀerent types of multivariate L-statistic: scalar, vector, and matrix-valued.

I For J(u) scalar, the product is scalar multiplication and then L(Xn) is a vector. I If also J(u) equals J0(kuk), i.e., is constant on contours of Q, then this agrees with a certain form of depth-weighted R averaging, x W (D(x, Xn)) dR(x, Xn), but diﬀers from R x W (D(x, Xn)) dFbn(x), however, unless d = 1.

I For J(u) a vector, then with inner product L(Xn) is scalar, and with Kronecker product L(Xn) is matrix-valued. DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS – CONCEPTS, PERSPECTIVES, CHALLENGES d DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS ON R d RANK FUNCTIONS ON R Rank functions in the univariate setting

I Rank functions have had considerable development in the univariate case for hypothesis testing.

I The classical univariate sign test statistic is equivalent to evaluating the (centered) rank function at the hypothetical median.

I The classical univariate Wilcoxon signed rank test statistic is just the sign test statistic based on the modiﬁed data set consisting of the pairwise averages of the original data. DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS – CONCEPTS, PERSPECTIVES, CHALLENGES d DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS ON R d RANK FUNCTIONS ON R Depth-based rank functions on Rd

d I A depth function induces a B -valued rank function R(x, F ) as the inverse of the quantile function. d I For x = M(F ), put R(M(F ), F ) = 0 (origin in R ).

I For x 6= M(F ), deﬁne R(x, F ) as the solution u of the equation x = Q(u, F ) .

I That is, the index of the quantile representation of x serves as its “rank”. d I Thus R(x, F ) takes values in the unit ball B .

I kR(x, F )k = O(x, F ), the outlyingness function.

I For the null hypothesis H0 : M(F ) = θ0, a natural test statistic is R(θ0, Xn), extending the univariate sign test. DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS – CONCEPTS, PERSPECTIVES, CHALLENGES d DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS ON R d RANK FUNCTIONS ON R Multivariate sign, rank, and signed-rank tests

I The univariate tests have been extended to multivariate data using the centered rank function R(x, Xn) associated with the spatial depth (Möttönenand Oja, 1995; Möttönen, Oja, and Tienari, 1997).

I Generalized spatial signed rank tests applying the spatial sign test to m-wise averages of the original data have been developed (Möttönen,Oja, and Serfling, 2004).

I Comprehensive methodology has been developed (Oja, 2010) based on the spatial rank function.

I Numerous other univariate rank methods, and several specialized bivariate approaches, should be extended.

I A comprehensive depth-based approach should be taken. DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS – CONCEPTS, PERSPECTIVES, CHALLENGES d DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS ON R DEPTH FUNCTIONS IN THE UNIVARIATE SETTING

For perspective, consider depth functions on R

−1 I The natural order in R and the quantile function F (p), 0 < p < 1, associated with a given distribution F suﬃce for demarking −1 1 I the median F 2 I speciﬁed lower and upper fractions of the population

I central regions such as “middle half” or “middle 90%”

I The order also induces the “centered rank function”: R(x, F ) = 2F (x) − 1, x ∈ R. I The associated “outlyingness function” is then O(x, F ) = |2F (x) − 1|, x ∈ R. I This yields an associated and rather natural “rank depth”: D(x, F ) = 1 − |2F (x) − 1|, x ∈ R. DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS – CONCEPTS, PERSPECTIVES, CHALLENGES d DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS ON R DEPTH FUNCTIONS IN THE UNIVARIATE SETTING

But depth functions are not needed on R I With orientation to the natural order in R, centrality (depth) is not a focus and not needed to deﬁne order statistics.

I However, the notion of “outlyingness” does indeed receive major emphasis, with two important types:

I centered rank outlyingness O(x, F ) = |2F (x) − 1|, x ∈ R

I scaled deviation outlyingness O(x, F ) = |x − µ(F )|/σ(F ), x ∈ R

I This depth-based perspective explains the boxplot! The boxplot has a hybrid design, employing the centered rank outlyingness to deﬁne the middle half but using a scaled deviation outlyingness O(x, F ) = |x − Q3(F )|/IQR(F ) to deﬁne the upper fence, and similarly for the lower fence. DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS – CONCEPTS, PERSPECTIVES, CHALLENGES d DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS ON R d DEPTH IN R VERSUS DEPTH IN R , d ≥ 2 Passing from univariate to multivariate setting d I For extension to R , d ≥ 2, we compensate for lack of an order by orienting to a “center”, which serves as starting point for a median-oriented formulation of quantiles.

I To see how this looks in the univariate setting, transform F −1(p) via u = 2p−1 to a median-oriented version Q(u, F ) −1 1+u = F 2 , −1 < u < 1 (a re-indexing of the quantiles). −1 1 I the median is now Q(0, F ) = M(F ) = F 2

I for given x, the equation x = Q(u, F ) yields the centered rank function, R(x, F ) = 2F (x) − 1 −1 I of course, no advantage in using Q(u, F ) over F (p) d d I In R , the quantile function Q(u, F ), u ∈ B , generalizes Q(u, F ), −1 < u < 1, rather than F −1(p), 0 < p < 1. DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS – CONCEPTS, PERSPECTIVES, CHALLENGES d DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS ON R d DEPTH IN R VERSUS DEPTH IN R , d ≥ 2 But “multivariate Tukey” does not reduce to “univariate Tukey”

I The Tukey halfspace depth reduces in R to 1 D(x, F ) = min{F (x), 1 − F (x)} = 2 (1 − |2F (x) − 1|), equivalent to the centered rank outlyingness O(x, F ) = |2F (x) − 1| (likewise for the simplicial depth).

I However, Mosteller and Tukey (1977) promote the use of scaled deviation outlyingness, O(x, F ) = |x −µ(F )|/σ(F ), whose has two multivariate generalizations, Mahalanobis outlyingness and projection outlyingness, neither of which is the Tukey outlyingness. ·· ^

DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS – CONCEPTS, PERSPECTIVES, CHALLENGES d DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS ON R THE DEPTH-OUTLYINGNESS-QUANTILE-RANK (D-O-Q-R) PARADIGM An equivalence and a paradigm for exploitation

I Statistical methods based on “order statistics”, “outlier identiﬁcation”, “quantiles”, “signs”, and “ranks” have their own distinctive appeals, their own special roles, and their own “practitioners” and “aﬃcionados”.

I Along with “symmetry”, these comprise the fundamental elements of nonparametric description.

I Intuitively and conceptually, depth (D), outlyingness (O), quantiles (Q), and ranks (R) are clearly interrelated. In fact, they are equivalent: the D-O-Q-R Paradigm.

I This is a consequence of making the right deﬁnitions. DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS – CONCEPTS, PERSPECTIVES, CHALLENGES d DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS ON R THE DEPTH-OUTLYINGNESS-QUANTILE-RANK (D-O-Q-R) PARADIGM

The D-O-Q-R Paradigm on Rd

I D(x, F ) and O(x, F ) are equivalent (inversely).

I Q(u, F ) and R(x, F ) are equivalent (inversely).

I These couplets are linked by d a) D(x, F ) induces a corresponding Q(u, F ) on B , b) O(x, F ) = kR(x, F )k = kuk. d I Depth, outlyingness, quantiles, and ranks in R are thus equivalent. Each of D, O, Q, and R generates the others, although they diﬀer in conceptual meaning and appeal.

I The interconnections in the D-O-Q-R paradigm can yield new procedures as well as new insights about existing ones. DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS – CONCEPTS, PERSPECTIVES, CHALLENGES d DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS ON R THE DEPTH-OUTLYINGNESS-QUANTILE-RANK (D-O-Q-R) PARADIGM

Example: the spatial depth on Rd , revisited I Spatial depth function (Vardi and Zhang, 2000): d D(x, F ) = 1 − kES(x − X)k, x ∈ R . d I Spatial outlyingness: O(x, F ) = kES(x − X)k, x ∈ R . I Spatial rank function (M¨ott¨onenand Oja, 1995): d R(x, F ) = ES(x − X), x ∈ R .

I Spatial quantile function (Dudley and Koltchinski, 1992; Chaudhuri, 1996): the solution x = Q(u, F ) of the equation ES(x − X) = u, for u ∈ Bd . Equivalently, Q = Q(u, F ) minimizes Φ(u, X − Q) − Φ(u, X), where Φ(u, t) = ktk + hu, ti, with h·, ·i the usual Euclidean inner product. DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS – CONCEPTS, PERSPECTIVES, CHALLENGES d DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS ON R INVARIANCE AND EQUIVARIANCE OF DOQR FUNCTIONS Invariance and equivariance ... d I Depth and outlyingness functions in R are desirably invariant.

I The relative centrality or outlyingness of two points in d R should not depend on the coordinate system.

I Typically, we seek aﬃne invariance.

I Quantile and rank functions are desirably equivariant.

I Typically, aﬃne equivariance is desired.

I Invariance/equivariance comes at a price, however

I Higher computational burden, or less robustness, or both

I In selecting a depth function, or outlyingness, or quantile or rank function, one selects a trade-oﬀ among robustness, computational burden, and invariance/equivariance. DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS – CONCEPTS, PERSPECTIVES, CHALLENGES DEPTH FUNCTIONS ON AN ARBITRARY DATA SPACE X THE QUESTION We ask:

For how abstract and general a space X can we construct meaningful depth functions D(x, P), x ∈ X , for P a probability model on X , and likewise sample depth functions D(x, Xn) for Xn a data set in X ? DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS – CONCEPTS, PERSPECTIVES, CHALLENGES DEPTH FUNCTIONS ON AN ARBITRARY DATA SPACE X d SOME DESIRABLE EXTENSIONS OF THE R NOTIONS Various extensions have been formulated

m I Depth on submanifolds of R

I for directional data on circles, spheres

I Depth on general inﬁnite dimensional data spaces

I for functional data analysis (we will take a look)

I for image analysis, mean functional estimation, covariance operator estimation Some further goals:

I Depth on spatio-temporal data

I Depth on complex data arising in data mining settings

I Depth on ... ? DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS – CONCEPTS, PERSPECTIVES, CHALLENGES DEPTH FUNCTIONS ON AN ARBITRARY DATA SPACE X SOME PARTICULAR EXTENSIONS The spatial approach in general d I Recall the spatial quantile function (Chaudhuri, 1996) in R : I Objective function involves `2 norm and inner product d I Indexed by the unit ball in R I Orthogonally (but not fully aﬃne) equivariant I Extends to d-dim `p spaces (Chakraborty, 2001) I Indexed by unit ball in dual space `q, for 1/p + 1/q = 1.

I But not even orthogonally equivariant when p 6= 2 I Extends to inﬁnite-dimensional Hilbert or Banach spaces

I Kemperman, 1987; Chaudhuri, 1996; Chakraborty and Chaudhuri, 2014

I Indexed by the unit ball in the dual space

I For Banach B, objective function uses norm of B and product hx, x ∈ B, h in dual B∗

I Entails handling noncompactness of closed unit ball and Gâteaux or Fréchet differentiability of norm

I Glivenko-Cantelli, Donsker results for spatial rank function process DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS – CONCEPTS, PERSPECTIVES, CHALLENGES DEPTH FUNCTIONS ON AN ARBITRARY DATA SPACE X SOME PARTICULAR EXTENSIONS Extensions of the halfspace and projection depths

I Dutta, Ghosh, and Chaudhuri (2011); Chakraborti and Chaudhuri (2014) d I Recall: in R , the halfspace and the projection depths are deﬁned in terms of linear projections u0x (inner products) ∗ I Extension to Banach space B uses hx, x ∈ B, h in dual B I But halfspace depth in inﬁnite-dimensional settings exhibits some anomalous behavior.

I For example, in a separable Hilbert space there exist symmetric probability measures which may even have independent Gaussian marginals, with the halfspace depth function identically equal to zero except on a subset having zero probability measure.

I Nevertheless, such symmetric probability measures will have a well-deﬁned half-space median that achieves the depth value 0.5. DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS – CONCEPTS, PERSPECTIVES, CHALLENGES DEPTH FUNCTIONS ON AN ARBITRARY DATA SPACE X ANOTHER GENERAL APPROACH

Integrated dual depth (Cuevas and Fraiman, 2009) ∗ I Banach data space X and dual X

I Take any d-dimensional depth and let D(y, G) denote its univariate (d = 1) implementation ∗ I For x ∈ B, take “projections” hx, h ∈ X , and let Fh be the distribution of hX when X has distribution F on X ∗ I Let Q be any measure on X

I The integrated dual depth (IDD): Z D(x, F ) = D(hx, Fh)dQ(h)

I Many possibilities for investigation and application ... DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS – CONCEPTS, PERSPECTIVES, CHALLENGES DEPTH FUNCTIONS ON AN ARBITRARY DATA SPACE X SPECIALIZATION TO THE FUNCTIONAL DATA SETTING With functional data, X is a Hilbert space

I The data points are curves, e.g., electricity loading curves, average temperature curves, child growth curves, etc.

I Typically, X is a real, separable, inﬁnite-dimensional Hilbert space with some norm k · k.

I Cuevas, Febrero, Fraiman, 2006, 2007; Lopez-Pintado and Romo, 2006, 2009, ...; Febraro, Galeano, Gonzalez-Mantiega, 2008, ... explosion! – much current activity.

I A major current focus: multivariate functional data d I the “curves” are R -valued for some d ≥ 2 I Each data point now consists of d curves in a 2-D plot! DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS – CONCEPTS, PERSPECTIVES, CHALLENGES DEPTH FUNCTIONS ON AN ARBITRARY DATA SPACE X SPECIALIZATION TO THE FUNCTIONAL DATA SETTING Connections with standard multivariate data

I Through discretization, functional data can be treated as a special case of multivariate data for some dimension d.

I The components of each data point follow an order.

I The data points (curves) can be viewed in a 2-D plot, regardless of the dimension d.

I However, the DOQR functions are more complicated to formulate, because internal structure due to the d-variate data points being curves must be taken into account. DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS – CONCEPTS, PERSPECTIVES, CHALLENGES DEPTH FUNCTIONS ON AN ARBITRARY DATA SPACE X SPECIALIZATION TO THE FUNCTIONAL DATA SETTING

I Invariance requirements are less stringent than for standard multivariate data. Full aﬃne invariance is not of interest! We want invariance merely under shift transformations and homogeneous scale transformations.

I Outlier identiﬁcation is much more challenging than for standard multivariate data. Besides “location” outliers that lie apart from the main body of data and are easy to identify, one must also identify

I “shape” outliers, for example reﬂecting phase variability, that can be hidden within the main body of curves, and

I “covariance” outliers, with variance-covariance diﬀering from that of the “regular” curves from the reference model. DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS – CONCEPTS, PERSPECTIVES, CHALLENGES DEPTH FUNCTIONS ON AN ARBITRARY DATA SPACE X SPECIALIZATION TO THE FUNCTIONAL DATA SETTING Illustration of what can go wrong with outlier identiﬁcation with functional data

I We consider four outlier contamination models (Arribas-Gil and Romo, 2013; Serﬂing and Wijesuriya, 2017).

I We consider the spatial depth approach and another popular approach denoted by “X”.

I We compare, for each model, the spatial and the “X” depth-based boxplots showing the middle half and the corresponding “fences” for demarking outliers.

I When a shape outlier is given too low an outlyingness by a depth function, it becomes included in the middle half and thereby distorts the shape of the middle half.

I Then the fences become mishaped, thus defeating outlier detection. Method “X” sometimes fails in this way. DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS – CONCEPTS, PERSPECTIVES, CHALLENGES DEPTH FUNCTIONS ON AN ARBITRARY DATA SPACE X

SPECIALIZATION TO THE FUNCTIONAL DATA SETTING

¡ ¢ £ ¤ ¥ ¦ § ¨ ¡ § © ¥ ¦ § ¥ © ¦ ¡ § £ ¦ © ¦ ¡ §

ଷ ଷ ! " ߳ ݐ א ݐ

% % &

# #

ଶ ଶ '

) ݐ െ ݐ ߳ ݐ ݐ െ ݐ ߳ ݐ ି ௦ି௧ ଴ ଷ

݁ ݐ ݏ ߛ !

* +

. / 0 1 2 3 4 1 1 0 3 5 6 0 7 8 9 : 3 5 ; : < = 3 5 >

߳ ݐ - א మ ݐ

A B

௧ିఓ A

) ? ௨ ି ? ି ௦ି௧

଴, ଴ଶ @ 3 < 0 3 5 : ݐ ݁ ߤ ݏ ݐ ߳ ݐ ݐ െ ݁ ߳ ݐ = ߛ ξ ߨ

ݑ C

F G HI J K L I IHK M N HOP Q R K M S R TU K M V

E ߳ ݐ א ݐ

Y Z

ݐ ߳ ݐ ݐ ݐ ߠ ߨ ߳ ݐ Y W

W ି ௦ି௧

[ X K THK M R

ݐ ݁ ߠ ݏ U ߛ ^

] ௧ିఓ మ

a b c d e f g d d c f h i c j k l m f h n m o p f h q

߳ ݐ ` א ݐ ି

t t u

଴_ ଴଴଴ଶ r

ݐ ߳ ݐ ݐ ݁ ߳ ݐ r ି ௦ି௧

s f o c f h m v ݐ ݁ ߤ ݏ ξ ߨ p ߛ

Figure: Four simulation models, each with a main component and a contamination component. DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS – CONCEPTS, PERSPECTIVES, CHALLENGES DEPTH FUNCTIONS ON AN ARBITRARY DATA SPACE X

SPECIALIZATION TO THE FUNCTIONAL DATA SETTING

¥ ¦ § ¨ § © § ¦ © § * + , - , . / , 0 1 + 2 ., 3 4 5 6 7 8 9 4 : ; 6 ; < ; : 8

)

( ¤

% >

! " # BCD BCE BCF BCG H

N OP Q P R S P T U O V RP W XY Z[ \] X ^ _ Z_ ` _ ^ \ r s t u t v w t x y s z v t { | } ~ | ~

c d

f g h f g i f g j f g k l

DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS – CONCEPTS, PERSPECTIVES, CHALLENGES DEPTH FUNCTIONS ON AN ARBITRARY DATA SPACE X

SPECIALIZATION TO THE FUNCTIONAL DATA SETTING

§ ¨ © ¨ ¨ ¨ © 0 1 2 3 4 5 6 7 5 8 9 5 : 1 ; < 4 1 = : > ? 5 @ 1 8 A 5 = B 1 @ C

! H

¢ +

)

" # $ " #% " #& " #' ( I JK I JL I JM I JN O

l m n o p q r s t u s v

S T U VW X

P i

w x y z { s | v { t y } s ~ { t ~ s z

Y Z [ \ T X ] ^ W _ ` T a V b X

Q j

t y { t y { t ~ s z

Z c Y d e ` f T X g T h

R k

Figure: Spatial Method versus Method X: Model 1 DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS – CONCEPTS, PERSPECTIVES, CHALLENGES DEPTH FUNCTIONS ON AN ARBITRARY DATA SPACE X

SPECIALIZATION TO THE FUNCTIONAL DATA SETTING

(

)

(

! " # $ % C DE C DF C DG C DH I

M N O PQ R f g h i j k l m n o m p

J c

q r s t u u s w u t

S T U V N R WXQ Y Z N [ P\ R m v p n m x y n x z { m

K d

| s u s u t

T ]S ^ _ Z ` N R a N b z n y } n y z n x z { m e L

Figure: Spatial Method versus Method X: Model 2 DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS – CONCEPTS, PERSPECTIVES, CHALLENGES DEPTH FUNCTIONS ON AN ARBITRARY DATA SPACE X

SPECIALIZATION TO THE FUNCTIONAL DATA SETTING

(

)

(

! " # $ % C DE C DF C DG C DH I

M N O PQ R f g h i j k l m n o m p

J c

N R WXQ YZ N [ P\ R q r s t u m v p u n s w m x y u n x z { m t

S T U V

K d

Z ` N R a N b | z n s y } u n y s u z n x z { m t

T ]S ^ _

L e

Figure: Spatial Method versus Method X: Model 3 DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS – CONCEPTS, PERSPECTIVES, CHALLENGES DEPTH FUNCTIONS ON AN ARBITRARY DATA SPACE X

SPECIALIZATION TO THE FUNCTIONAL DATA SETTING

)

(

)

(

!" !# ! $ !% & D EF D EG D EH D EI J

N O P QR S g h i j k l m n o p n q

K d

r s t u v v t x v u

T U V W O S XYR Z [ O \ Q] S n w q o n y z o y { | n

L e

} t v t v u

U ^ T _ ` [ a O S b O c { o z ~ o z { o y { | n f M

Figure: Spatial Method versus Method X: Model 4 DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS – CONCEPTS, PERSPECTIVES, CHALLENGES DEPTH FUNCTIONS ON AN ARBITRARY PARAMETER SPACE Θ TARGET QUESTION, KEY IDEA, GENERAL APPROACH Target question, key idea, general approach

I Target Question. Do depth and outlyingness have roles in inference problems other than location in Rd ? I Key Idea. The inference problem using data set Xn from distribution F is deﬁned relative to a target parameter θ(F ) in some space Θ, concerning, for example, location, scale, skewness, kurtosis, or a regression line or hyperplane.

I General Approach. Deﬁne a data-based function of θ representing its “outlyingness” in Θ when considered as an estimate of θ(F ). Estimate θ(F ) by the data-based minimum-outlyingness element of the parameter space Θ.

I Analogy with maximum likelihood. For data following density f (x; θ), x ∈ X , we estimate θ by maximizing the data-based likelihood function L(θ) = f (Xn; θ), θ ∈ Θ. DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS – CONCEPTS, PERSPECTIVES, CHALLENGES DEPTH FUNCTIONS ON AN ARBITRARY PARAMETER SPACE Θ TARGET QUESTION, KEY IDEA, GENERAL APPROACH General formulation in arbitrary data and parameter spaces

I Parameter space Θ of arbitrary type

I Data Xn = (X1,..., Xn) in an arbitrary space X

I Goal. Find point θ ∈ Θ which is a “best ﬁt” to the set of data points Xn, or – optionally – to those points remaining after identiﬁcation and elimination of outliers.

I This can be thought of as a shape-ﬁtting problem.

I Maximal depth approach. Introduce a relevant data-based depth function D(θ, Xn), θ ∈ Θ, deﬁned on the parameter space Θ, with “best ﬁt” equivalent to maximal depth.

I This is a nonparametric approach. DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS – CONCEPTS, PERSPECTIVES, CHALLENGES DEPTH FUNCTIONS ON AN ARBITRARY PARAMETER SPACE Θ FIRST EXAMPLES

Location estimation in Rd d I Θ = X = R d I A depth function D(x, F ), x ∈ R , is maximized at M deﬁning a location parameter. The sample version D(x, Xn), d x ∈ R , represents, by reinterpretation, a depth De(θ, Xn) on the possible values for the location parameter, d De(θ, Xn), θ ∈ Θ (= R ), with maximal De-depth equivalent to maximal D-depth, via De = D. d I Location estimation by maximizing sample depth on R is equivalent to maximizing sample depth in the parameter space, since the data space X and the parameter space Θ are identical. DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS – CONCEPTS, PERSPECTIVES, CHALLENGES DEPTH FUNCTIONS ON AN ARBITRARY PARAMETER SPACE Θ FIRST EXAMPLES

Estimation of dispersion in Rd I Θ = {covariance matrices Σ}.

I For univariate data Yn = (Y1,..., Yn), deﬁne outlyingness of a scale parameter σ by

µ ({|Yi − m(Yn)|, 1 ≤ i ≤ n}) O1n(σ, Yn) = − 1 , σ with µ(·) and m(·) univariate location statistics. d I For R -valued Xn = (X1,..., Xn), deﬁne outlyingness of a covariance matrix Σ via the projection-pursuit approach: √ 0 0 Odn(Σ, Xn) = sup O1n u Σu, u Xn . kuk=1

I Minimization of Odn(Σ, Xn) with respect to Σ yields a “maximum dispersion depth estimator”. DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS – CONCEPTS, PERSPECTIVES, CHALLENGES DEPTH FUNCTIONS ON AN ARBITRARY PARAMETER SPACE Θ FIRST EXAMPLES Regression with p regressors

p+1 I Θ = {hyperplanes in R }

p+1 I For R -valued data Xn = (X1,..., Xn), deﬁne the depth of a hyperplane h as the minimum fraction of observations in Xn whose removal makes h a “nonﬁt” (Rousseeuw and Hubert, 1999).

I The maximum depth point is the “regression ﬁt”.

I We make “fit” and “nonfit” precise in the more general setting considered next. DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS – CONCEPTS, PERSPECTIVES, CHALLENGES DEPTH FUNCTIONS ON AN ARBITRARY PARAMETER SPACE Θ GENERAL EXAMPLE: SHAPE FITTING PROBLEMS IN COMPUTATIONAL GEOMETRY Shape-fitting problems in computational geometry

I Shape-ﬁtting problems arise not only in regression but also in computer vision, machine learning, data mining, etc.

I Find the point, line, hypersphere, sphere, or cylinder best ﬁtting a point set Xn. I I.e., the minimum radius sphere, minimum width cylinder, smallest width slab, minimum radius spherical shell, or minimum radius cylindrical shell enclosing Xn. I This is the realm of computational geometry.

I Given a family of shapes Θ, a set of input points Xn, and a “fitting criterion” β(θ, Xn) for closeness of θ to Xn, which shape θ fits best, i.e., minimizes β(θ, Xn)? DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS – CONCEPTS, PERSPECTIVES, CHALLENGES DEPTH FUNCTIONS ON AN ARBITRARY PARAMETER SPACE Θ GENERAL EXAMPLE: SHAPE FITTING PROBLEMS IN COMPUTATIONAL GEOMETRY Formulation of shape-fitting criterion functions

I Assumption. Let the shape-ﬁtting criterion β(θ, Xn) be a function of pointwise functions of form

β(Xi , θ, α(Xn)), 1 ≤ i ≤ n, defined on the input data space and Θ, and measuring the closeness of each data point to the candidate fit θ. DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS – CONCEPTS, PERSPECTIVES, CHALLENGES DEPTH FUNCTIONS ON AN ARBITRARY PARAMETER SPACE Θ GENERAL EXAMPLE: SHAPE FITTING PROBLEMS IN COMPUTATIONAL GEOMETRY Formulation of shape-fitting criterion functions

d I Example A. With X = R , and shape space Θ = some class of subsets in Rd , the pointwise objective function β(x, θ) = min kx − yk y∈θ

is the minimum distance from x to a point in θ, and

β(θ, n) = max β(Xi , θ) X 1≤i≤n

is the maximal distance from a point in Xn to the nearest point in θ.

I For example, ﬁtting a line to Xn becomes the same as ﬁnding the minimum-radius enclosing cylinder θ. DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS – CONCEPTS, PERSPECTIVES, CHALLENGES DEPTH FUNCTIONS ON AN ARBITRARY PARAMETER SPACE Θ GENERAL EXAMPLE: SHAPE FITTING PROBLEMS IN COMPUTATIONAL GEOMETRY

∗ I Example A . In Example A, replace maximum by sum, X β(θ, Xn) = g(β(Xi , θ)), 1≤i≤n and introduce a function g(·) of the criterion function.

I With g(·) the identity function, this gives the total distance from the points of Xn to θ as the optimization criterion. I Example: ﬁtting a line to Xn now yields the well-known principal components regression line.

I Example: for location problems with θ representing a point, β(x, θ) reduces to kx − θk and yields the spatial median as the optimal ﬁt.

I With g(·) an increasing function and θ now representing a hyperplane, other objective functions arising in regression problems are obtained. DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS – CONCEPTS, PERSPECTIVES, CHALLENGES DEPTH FUNCTIONS ON AN ARBITRARY PARAMETER SPACE Θ GENERAL EXAMPLE: SHAPE FITTING PROBLEMS IN COMPUTATIONAL GEOMETRY

I Example B. Clustering. (i) The k-center problem involves the objective function of Example A. (ii) The k-median problem involves the objective function of Example A∗ with g(·) the identity function and is a special case of the facility location problem.

I Example C. Projective Clustering. (i) The “approximate k-flat problem” involves the objective function of Example A, with θ a k-flat in Rd . (ii) The “j approximate k-flat problem” involves the maximum-type objective function with pointwise function

β(x, θ) = min min kx − yk, 1≤i≤j y∈Fi

d where θ is a vector (F1,..., Fj ) of j k-ﬂats in R . DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS – CONCEPTS, PERSPECTIVES, CHALLENGES DEPTH FUNCTIONS ON AN ARBITRARY PARAMETER SPACE Θ GENERAL EXAMPLE: SHAPE FITTING PROBLEMS IN COMPUTATIONAL GEOMETRY A key issue

I A key issue is identiﬁcation and handing of outliers.

I Whereas ingenious ad hoc solutions in special cases have been given, a general approach is needed.

I Here is where depth and outlyingness functions can play a role. DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS – CONCEPTS, PERSPECTIVES, CHALLENGES DEPTH FUNCTIONS ON AN ARBITRARY PARAMETER SPACE Θ GENERAL EXAMPLE: SHAPE FITTING PROBLEMS IN COMPUTATIONAL GEOMETRY Depth function approach to shape-ﬁtting with outlier handling

I To address the outlier problem in shape-ﬁtting, the notion of “depth” arises very naturally, but initially as a function of “ﬁts” rather than of data points (as we have illustrated for location, dispersion, and regression inference).

I Informal notion: In objective optimization with a space of fits Θ and objective function β(θ, Xn), the depth of θ ∈ Θ is the minimum fraction of input observations whose removal makes θ a “nonfit”, i.e., inadmissible. Take the “maximal depth” fit as the “best fit”.

I In the location estimation problem, for example, the points θ outside the convex hull of Xn are the “nonfits”. DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS – CONCEPTS, PERSPECTIVES, CHALLENGES DEPTH FUNCTIONS ON AN ARBITRARY PARAMETER SPACE Θ GENERAL EXAMPLE: SHAPE FITTING PROBLEMS IN COMPUTATIONAL GEOMETRY Data-based depth functions defined on fits θ

I Approach 1. Deﬁne the depth Dn(θ) of a candidate ﬁt θ as the minimum fraction of data points in Xn whose removal yielding reduced data Xen makes θ “inadmissible”, i.e., uniformly strictly improvable by some other value θ˜: ˜ β(x, θ, α(Xen)) < β(x, θ, α(Xn)), x in Xen. I Extends Rousseeuw and Hubert (1999), Mizera (2002), and Zhang (2002)

I For a special case, the maximal depth point generalizes the halfspace median. I Approach 2. Alternatively, deﬁne the depth of θ by ∗ 1 Dn (θ) = P . 1 + 1≤i≤n β(Xi , θ, α(Xn))

I For a special case, the maximal depth point generalizes the spatial median. DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS – CONCEPTS, PERSPECTIVES, CHALLENGES DEPTH FUNCTIONS ON AN ARBITRARY PARAMETER SPACE Θ GENERAL EXAMPLE: SHAPE FITTING PROBLEMS IN COMPUTATIONAL GEOMETRY

Outlier handling in Xn via depth on θ

I From depth and outlyingness defined on fits, pass to 1. identification of outliers in the input, i.e., those points which cause the fit to be significantly less than “optimal”, 2. optional removal of selected outliers, 3. optimal fit to the remaining points.

I One approach toward Step 1: residuals analysis DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS – CONCEPTS, PERSPECTIVES, CHALLENGES DEPTH FUNCTIONS ON AN ARBITRARY PARAMETER SPACE Θ GENERAL EXAMPLE: SHAPE FITTING PROBLEMS IN COMPUTATIONAL GEOMETRY A general “residuals approach” for depth-based outlier identiﬁcation and handling in shape-ﬁtting

∗ 1. Obtain an initial robust fit θn as a maximum depth fit, ∗ employing for example either Dn(θ) or Dn (θ). 2. Define an outlyingness measure for each data point x in X ∗ via On(x) = β(x, θn, α(Xn)). 3. Set a threshold on On(x) for removal of outliers in Xn, and after optional removal of outliers, optimize using the reduced input Xen, yielding a robust optimal fit θn. 4. Carry out confirmatory residual analysis now using θn as reference point. 5. Iterate these steps to attain a satisfactory “optimal fit”. DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS – CONCEPTS, PERSPECTIVES, CHALLENGES DEPTH FUNCTIONS ON AN ARBITRARY PARAMETER SPACE Θ GENERAL EXAMPLE: SHAPE FITTING PROBLEMS IN COMPUTATIONAL GEOMETRY More elaborate criterion functions for shape fitting

I Let the shape-ﬁtting criterion β(θ, Xn) be a function of setwise functions deﬁned on subsets of X and on Θ by

β({Xi1 ,..., Xim }, θ, α(Xn)),

measuring closeness of the set {Xi1 ,..., Xim } of data points n to a given ﬁt θ, for all m m-sets {Xi1 ,..., Xim }. I For example, β(·, θ, α(Xn)) might depend upon the set

{Xi1 ,..., Xim } through h(Xi1 ,..., Xim ) for a “kernel” h as used with U-statistics.

I Involves multivariate “U-quantiles” (Zhou and Serﬂing, 2008a,b).

I Special case: location estimation. Let h be the m-wise average (for m = 2, a Hodges-Lehmann approach). DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS – CONCEPTS, PERSPECTIVES, CHALLENGES CONCLUDING REMARKS KEY ASPECTS OF THE DEPTH FUNCTION APPROACH We have seen that depth functions ...

I provide a general and powerful nonparametric approach for ordering data points and developing notions of quantiles, outlyingness and ranks

I generalize univariate notions I focus on centrality I adopt center-outward ordering as a compensation for the lack of a natural order

I require invariance and robustness as key criteria I augment density functions, but do not compete with them I ignore nonconvexity and multimodality, which are handled by the density function

I augment eﬃcient parametric and semiparametric methods, but do not compete with them DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS – CONCEPTS, PERSPECTIVES, CHALLENGES CONCLUDING REMARKS KEY ASPECTS OF THE DEPTH FUNCTION APPROACH Depth functions and clustering

I Depth functions are not a clustering method.

I When applicable, clustering should be done ﬁrst.

I Then depth-based nonparametric description can be carried out separately within clusters. DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS – CONCEPTS, PERSPECTIVES, CHALLENGES CONCLUDING REMARKS SOME DIRECTIONS FOR INVESTIGATION Vector-valued depth functions

I As seen with functional data, for example, there can be several complementary notions of outlyingness (and hence of depth) in a given context, each treating outlyingness in a diﬀerent sense. Collectively, these deﬁne a vector-valued outlyingness function on the given data points.

I Also, in the case of competing depth functions in a given context, one might treat them in a uniﬁed and coherent fashion through a vector-valued depth function.

I A formal theory of vector-valued depth functions would be useful.

I This becomes even more challenging in examples such as multivariate functional data. DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS – CONCEPTS, PERSPECTIVES, CHALLENGES CONCLUDING REMARKS SOME DIRECTIONS FOR INVESTIGATION The DOQR paradigm in general

I The DOQR paradigm exposited for Euclidean data needs explicit development in other contexts, such as functional data.

I A convenient starting point is the case of the spatial depth approach in general settings. DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS – CONCEPTS, PERSPECTIVES, CHALLENGES CONCLUDING REMARKS SOME DIRECTIONS FOR INVESTIGATION

Construction of outlyingness functions O(x, Xn)

I We have seen that the masking and swamping breakdown point robustness measures can be represented as regular breakdown points and then computed in principle, for any outlyingness (depth) function deﬁned on any data space.

I We have raised the question of how generally one can deﬁne depth functions.

I In particular, we have seen a number of depth functions for Euclidean data and a few ideas on depth functions for functional and abstract data.

I Fuller exploration of the formulation of depth functions D(x, Xn) in an arbitrary space X is desirable. DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS – CONCEPTS, PERSPECTIVES, CHALLENGES CONCLUDING REMARKS SOME DIRECTIONS FOR INVESTIGATION Depth-based analysis of network data

I Special centrality measures have long been considered in the context of description and analysis of network data.

I This topic has developed completely apart from that of depth functions for Euclidean, functional, and other data.

I The relevant concepts and formulations are quite diﬀerent.

I A uniﬁed study of depth functions as developed here and centrality measures as developed for network data is of interest. DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS – CONCEPTS, PERSPECTIVES, CHALLENGES CONCLUDING REMARKS SOME DIRECTIONS FOR INVESTIGATION Depth on parameter spaces

I We have treated depth on parameter spaces for shape- ﬁtting problems in the setting of computational geometry.

I A number of examples, some new and unexplored, have been sketched.

I A formal, comprehensive treatment of depth on parameter space, its theory and application, is of interest. DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS – CONCEPTS, PERSPECTIVES, CHALLENGES CONCLUDING REMARKS A WORK IN PROGRESS “Depth and quantile functions for nonparametric multivariate analysis and beyond”

I This sounds like a tentative title for a book in progress ... DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS – CONCEPTS, PERSPECTIVES, CHALLENGES ACKNOWLEDGMENTS

Acknowledgments

I The speaker thanks G. L. Thompson, Probal Chaudhuri, Ricardo Fraiman, Marc Hallin, Regina Liu, Alicia Nieto- Reyes, Hannu Oja, Wolfgang Polonik, Juan Romo, the speakers’ students, and many others including anonymous commentators, for very thoughtful and helpful remarks and encouragement.

I The speaker especially thanks Xin Dang and colleagues in the Department of Mathematics at the University of Misssissippi for the kind invitation to speak in the 2018- 2019 ASA Alabama Chapter Mini-Conference.

I Support by NSF Grants DMS-9705209, DMS-0103698, CCF- 0430366, and DMS-1106691 is greatly appreciated. DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS – CONCEPTS, PERSPECTIVES, CHALLENGES THANK YOU FOR YOUR ATTENTION DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS – CONCEPTS, PERSPECTIVES, CHALLENGES SOME RELEVANT REFERENCES

Agostinelli, C. and Romanazzi, M. (2007). Local depth of univariate distributions. Working paper n. 1/2007, Dipartimento di Statistica, UniversitaCaFoscari, Venezia. Agostinelli, C. and Romanazzi, M. (2008). Local depth of univariate distributions. Local depth of multidimensional data. Working paper n. 3/2008, Dipartimento di Statistica, UniversitaCaFoscari, Venezia. Barnett, V., and Lewis, T. (1995). Outliers in Statistical Data, 3rd edition. John Wiley & Sons, New York. Becker, C. (1996). Bruchpunkt und Bias zur Beurteilung multi- variater Ausreisseridentifizierung. Ph.D. Dissertation, Universität Dortmund. Becker, C. and Gather, U. (1999). The masking breakdown point of multivariate outlier identification rules. Journal of the American Statistical Association 94 947–955. Chakraborty, A. and Chaudhuri, P. (2014). The spatial distribu- DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS – CONCEPTS, PERSPECTIVES, CHALLENGES SOME RELEVANT REFERENCES

tion in infinite dimensional spaces and related quantiles and depths. Annals of Statistics 42 1203–1231. Chakraborty, B. (2001). On affine equivariant multivariate quantiles. Annals of the Institute of Statistical Mathematics 53 380–403. Chaudhuri, P. (1996). On a geometric notion of quantiles for multivariate data. Journal of the American Statistical Association 91 862–872. Chen, Y., Bart, Jr., H. L., Dang, X., and Peng, H. (2007). Depth- Based novelty detection and its application to taxonomic research. Seventh IEEE International Conference on Data Mining (ICDM, 2007) 113–122. Cuesta-Albertos, J. A., Fraiman, R. and Matrán, C. (2007). The random projection method in goodness of fit for functional data. Computational Statistics and Data Analysis 51 4814–4831. Cuevas A., Febrero, M. and Fraiman, R. (2006). On the use of the DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS – CONCEPTS, PERSPECTIVES, CHALLENGES SOME RELEVANT REFERENCES

bootstrap for estimating functions with functional data. Computa- tional Statistics and Data Analysis 51 1063–1074. Cuevas A., Febrero, M. and Fraiman, R. (2007). Robust estimation and classification for functional data via projection-based depth notions. Computational Statistics 22 481–496. Cuevas, A. and Fraiman, R. (2009). On depth measures and dual statistics. A methodology for dealing with general data. Journal of Multivariate Analysis 100 753–766. Dang, X. and Serfling, R. (2010). Nonparametric depth-based multivariate outlier identifiers, and masking robustness properties. Jour- nal of Statistical Planning and Inference 140 198–213. Davies, L. and Gather, U. (1993). The identification of multiple outliers. Journal of the American Statistical Association 88 782– 801. Dudley, R. M. and Koltchinskii, V. I. (1992). The spatial quantiles. DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS – CONCEPTS, PERSPECTIVES, CHALLENGES SOME RELEVANT REFERENCES

Preprint. Dutta, S., Ghosh, A. K. and Chaudhuri, P. (2011). Some intriguing properties of Tukey’s half-space depth. Bernoulli 17 1420–1434. Febrero, M., Galeano, P. and Gonz´alez-Manteiga, W. (2008). Out- lier detection in functional data by depth measures, with application to identify abnormal NOx levels. Environmetrics 19 331–345. Hotelling, H. (1929). Stability in competition. Econometrics Journal 39 41–57. Kemperman, J. H. B. (1987). The median of a ﬁnite measure on a Banach space. In Statistical Data Analysis Based On the L1-Norm and Related Methods (Y. Dodge, ed.), pp. 217–230, North-Holland. Liu, R. Y. (1988). On a notion of simplicial depth. Proceedings of the National Academy of Science USA 85 1732–1734. Liu, R. Y. (1990). On a notion of data depth based on random simplices. Annals of Statistics 18 405–414. DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS – CONCEPTS, PERSPECTIVES, CHALLENGES SOME RELEVANT REFERENCES

Liu, R. Y. (1992). Data depth and multivariate rank tests. In L1-Statistics and Related Methods (Y. Dodge, ed.), pp. 279–294, North-Holland, Amsterdam. Liu, R. Y., Parelius, J. M. and Singh, K. (1999). Multivariate analysis by data depth: Descriptive statistics, graphics and inference (with discussion). Annals of Statistics 27 783–858. Liu, R. Y., Serfling, R. and Souvaine, D. L. (eds.), Data Depth: Robust Multivariate Analysis, Computational Geometry and Appli- cations, DIMACS Series in Discrete Mathematics and Theoretical Computer Science, Volume 72, American Mathematical Society, Providence, Rhode Island, 2006. Liu, R. Y. and Singh, K. (1993). A quality index based on data depth and multivariate rank tests. Journal of the American Statistical Association 88 252–260. López-Pintado, S. and Romo, J. (2006). Depth-based classification DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS – CONCEPTS, PERSPECTIVES, CHALLENGES SOME RELEVANT REFERENCES

for functional data. In Data Depth: Robust Multivariate Analysis, Computational Geometry and Applications (R. Y. Liu, R. Serfling, D. L. Souvaine, eds.), pp. 103–121. DIMACS Series in Discrete Math- ematics and Theoretical Computer Science, Volume 72, American Mathematical Society, Providence, Rhode Island. López-Pintado, S. and Romo, J. (2009). On the concept of depth for functional data. Journal of the American Statistical Association 104 718–734. Mizera, I. (2002). On depth and deep points: a calculus. Annals of Statistics 30 1681–1736. Mosteller, C. F. and Tukey, J. W. (1977). Data Analysis and Re- gression. Addison-Wesley, Reading, Mass. Möttönen,J. and Oja, H. (1995). Multivariate spatial sign and rank methods. Journal of Nonparametric Statistics 5 201–213. Möttönen,J., Oja, H., and Serfling, R. (2004). Multivariate gener- DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS – CONCEPTS, PERSPECTIVES, CHALLENGES SOME RELEVANT REFERENCES

alized spatial signed-rank methods. Journal of Statistical Research 39 25–48. Möttönen,J., Oja, H., and Tienari, J. (1997). On the efficiency of multivariate spatial sign and rank tests. Annals of Statistics 25 542–552. Oja, H. (2010). Multivariate Nonparametric Methods with R: An approach based on spatial signs and ranks. Springer. Paindaveine, D., Van Bever, G. (2013). From depth to local depth: a focus on centrality. Journal of the American Statistical Association 105 1105–1119. Rousseeuw, P. J. and Hubert, M. (1999). Depth in an arrangement of hyperplanes. Discrete Computational Geometry 22 167–176. Serfling, R. (2006a). Depth functions in nonparametric multivariate analysis. In Data Depth: Robust Multivariate Analysis, Computa- tional Geometry and Applications (R. Y. Liu, R. Serfling, D. L. Sou- DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS – CONCEPTS, PERSPECTIVES, CHALLENGES SOME RELEVANT REFERENCES

vaine, eds.), pp. 1–16. DIMACS Series in Discrete Mathematics and Theoretical Computer Science, Volume 72, American Mathematical Society, Providence, Rhode Island. Serfling, R. (2006b). Multivariate symmetry and asymmetry. In The Encyclopedia of Statistical Sciences, Second Edition, Vol. 8 (S. Kotz, N. Balakrishnan, C. B. Read, B. Vidakovic and N. L. Johnson, eds.), pp. 5338–5345. Wiley. Serfling, R. (2010). Equivariance and invariance properties of multivariate quantile and related functions, and the role of standardiza- tion. Journal of Nonparametric Statistics 22 915–936. Serfling, R. and Wang, S. (2014). General foundations for studying masking and swamping robustness of outlier identifiers. Statistical Methodology 20 79–90. Serfling, R. and Wijesuriya, U. (2017). Depth-based nonparametric description of functional data, with emphasis on use of spatial depth. DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS – CONCEPTS, PERSPECTIVES, CHALLENGES SOME RELEVANT REFERENCES

Computational Statistics and Data Analysis 105 24–45. Small, C. G. (1987). Measures of centrality for multivariate and directional distributions. Canadian Journal of Statistics 15 31–39. Tukey, J. W. (1975). Mathematics and the picturing of data. In Proceedings of the International Congress of Mathematicians, Van- couver 1974 (R. D. James, ed.), 2 523–531.

Vardi, Y. and Zhang, C.-H. (2000). The multivariate L1-median and associated data depth. Proceedings of National Academy of Science USA 97 1423–1426. Wang, S. and Serfling, R. (2015). On masking and swamping robustness of leading nonparametric outlier identifiers for univariate data. Journal of Statistical Planning and Inference 162 62–74. Wang, S. and Serfling, R. (2017). On masking and swamping robustness of leading nonparametric outlier identifiers for multivariate data. In review. DEPTH, OUTLYINGNESS, QUANTILE, AND RANK FUNCTIONS – CONCEPTS, PERSPECTIVES, CHALLENGES SOME RELEVANT REFERENCES

Zhang, J. (2002). Some extensions of Tukey’s depth function. Jour- nal of Multivariate Analysis 82 134–165. Zhou, W. and Serfling, R. (2008a). Multivariate spatial U-quantiles: a Bahadur-Kiefer representation, a Theil-Sen estimator for multiple regression, and a robust dispersion estimator. Journal of Statistical Planning and Inference 138 1660–1678. Zhou, W. and Serfling, R. (2008b). Generalized multivariate rank type test statistics via spatial U-quantiles. Statistics & Probability Letters 78 376–383. Zuo, Y. (2003). Projection-based depth functions and associated medians.Annals of Statistics 31 1460–1490. Zuo, Y. and Serfling, R. (2000). General notions of statistical depth function. Annals of Statistics 28 461–482.