DOCSLIB.ORG

Lecture 15: Order Statistics Range of (A, B)

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Lecture 15: Order Statistics Range of (A, B) Section 4.6 Order Statistics Order Statistics Let X1; X2; X3; X4; X5 be iid random variables with a distribution F with a Lecture 15: Order Statistics range of (a; b). We can relabel these X's such that their labels correspond to arranging them in increasing order so that Statistics 104 X X X X X (1) ≤ (2) ≤ (3) ≤ (4) ≤ (5) Colin Rundel X(1) X(2) X(3) X(4) X(5) March 14, 2012 a X5 X1 X4 X2 X3 b In the case where the distribution F is continuous we can make the stronger statement that X(1) < X(2) < X(3) < X(4) < X(5) Since P(Xi = Xj ) = 0 for all i = j for continuous random variables. 6 Statistics 104 (Colin Rundel) Lecture 15 March 14, 2012 1 / 24 Section 4.6 Order Statistics Section 4.6 Order Statistics Order Statistics, cont. Notation Detour For X1; X2;:::; Xn iid random variables Xk is the kth smallest X , usually For a continuous random variable we can see that called the kth order statistic. f (x) ≈ P(x ≤ X ≤ x + ) = P(X 2 [x; x + ]) X(1) is therefore the smallest X and lim f (x) = lim P(X 2 [x; x + ]) !0 !0 f (x) = lim P(X 2 [x; x + ])/ X(1) = min(X1;:::; Xn) !0 Similarly, X(n) is the largest X and P (x X x + ✏)=P (X [x, x + ✏]) X(n) = max(X1;:::; Xn) 2 f(x) f(x + ✏) x x + ✏ Statistics 104 (Colin Rundel) Lecture 15 March 14, 2012 2 / 24 Statistics 104 (Colin Rundel) Lecture 15 March 14, 2012 3 / 24 Section 4.6 Order Statistics Section 4.6 Order Statistics Density of the maximum Density of the minimum For X1; X2;:::; Xn iid continuous random variables with pdf f and cdf F For X1; X2;:::; Xn iid continuous random variables with pdf f and cdf F the density of the maximum is the density of the minimum is P(X(n) 2 [x; x + ]) = P(one of the X 's 2 [x; x + ] and all others < x) P(X(1) 2 [x; x + ]) = P(one of the X 's 2 [x; x + ] and all others > x) n n X X = P(Xi 2 [x; x + ] and all others < x) = P(Xi 2 [x; x + ] and all others > x) i=1 i=1 = nP(X1 2 [x; x + ] and all others < x) = nP(X1 2 [x; x + ] and all others > x) = nP(X1 2 [x; x + ])P(all others < x) = nP(X1 2 [x; x + ])P(all others > x) = nP(X1 2 [x; x + ])P(X2 < x) ··· P(Xn < x) = nP(X1 2 [x; x + ])P(X2 > x) ··· P(Xn > x) = nf (x)F (x)n−1 = nf (x)(1 − F (x))n−1 n−1 n−1 f(n)(x) = nf (x)F (x) f(1)(x) = nf (x)(1 − F (x)) Statistics 104 (Colin Rundel) Lecture 15 March 14, 2012 4 / 24 Statistics 104 (Colin Rundel) Lecture 15 March 14, 2012 5 / 24 Section 4.6 Order Statistics Section 4.6 Order Statistics Density of the kth Order Statistic Cumulative Distribution of the min and max For X1; X2;:::; Xn iid continuous random variables with pdf f and cdf F For X1; X2;:::; Xn iid continuous random variables with pdf f and cdf F the density of the kth order statistic is the density of the kth order statistic is F(1)(x) = P(X(1) < x) = 1 − P(X(1) > x) P(X(k) 2 [x; x + ]) = P(one of the X 's 2 [x; x + ] and exactly k − 1 of the others < x) n = 1 − P(X1 > x;:::; Xn > x) = 1 − P(X1 > x) ··· P(Xn > x) X n = P(Xi 2 [x; x + ] and exactly k − 1 of the others < x) = 1 − (1 − F (x)) i=1 = nP(X1 2 [x; x + ] and exactly k − 1 of the others < x) F(n)(x) = P(X(n) < x) = 1 − P(X(n) > x) = nP(X1 2 [x; x + ])P(exactly k − 1 of the others < x) = P(X < x;:::; X < x) = P(X < x) ··· P(X < x) ! ! ! 1 n 1 n n − 1 n − 1 n = nP(X 2 [x; x + ]) P(X < x)k−1P(X > x)n−k = nf (x) F (x)k−=1(1F (−x)F (x))n−k 1 k − 1 k − 1 ! n − 1 d dF (x) f (x) = nf (x) F (x)k−1(1 − F (x))n−k f (x) = (1 − F (x))n = n(1 − F (x))n−1 = nf (x)(1 − F (x))n−1 (k) k − 1 (1) dx dx d dF (x) f (x) = F (x)n = nF (x)n−1 = nf (x)F (x)n−1 (n) dx dx Statistics 104 (Colin Rundel) Lecture 15 March 14, 2012 6 / 24 Statistics 104 (Colin Rundel) Lecture 15 March 14, 2012 7 / 24 Section 4.6 Order Statistics Section 4.6 Order Statistics Order Statistic of Standard Uniforms Beta Distribution iid The Beta distribution is a continuous distribution defined on the range Let X1; X2;:::; Xn Unif(0; 1) then the density of X(n) is given by ∼ (0; 1) where the density is given by ! n − 1 1 r−1 s−1 f (x) = nf (x) F (x)k−1(1 − F (x))n−k f (x) = x (1 − x) (k) k − 1 B(r; s) ( nn−1x k−1(1 − x)n−k if 0 < x < 1 where B(r; s) is called the Beta function and it is a normalizing constant = k−1 0 otherwise which ensures the density integrates to 1. Z 1 This is an example of the Beta distribution where r = k and s = n k + 1. 1 = f (x)dx − 0 Z 1 1 1 = x r−1(1 − x)s−1dx B(r; s) X(k) Beta(k; n k + 1) 0 ∼ − 1 Z 1 1 = x r−1(1 − x)s−1dx B(r; s) 0 Z 1 B(r; s) = x r−1(1 − x)s−1dx 0 Statistics 104 (Colin Rundel) Lecture 15 March 14, 2012 8 / 24 Statistics 104 (Colin Rundel) Lecture 15 March 14, 2012 9 / 24 Section 4.6 Order Statistics Section 4.6 Order Statistics Beta Function Beta Function - Expectation The connection between the Beta distribution and the kth order statistic Let X Beta(r; s) then of n standard Uniform random variables allows us to simplify the Beta ∼ function. Z 1 1 E(X ) = x x r−1(1 − x)s−1dx 0 B(r; s) Z 1 1 Z r−1 s−1 = ; 1x (r+1)−1(1 − x)s−1dx B(r; s) = x (1 − x) dx B(r; s) 0 0 1 B(r + 1; s) B(k; n − k + 1) = = n−1 B(r; s) n k−1 (k − 1)!(n − 1 − k + 1)! r!(s − 1)! (r + s − 1)! = = n(n − 1)! (r + s)! (r − 1)!(s − 1)! (r − 1)!(n − k)! r! (r + s − 1)! = = n! (r − 1)! (r + s)! r (r − 1)!(s − 1)! Γ(r)Γ(s) = = = r + s (r + s − 1)! Γ(r + s) Statistics 104 (Colin Rundel) Lecture 15 March 14, 2012 10 / 24 Statistics 104 (Colin Rundel) Lecture 15 March 14, 2012 11 / 24 Section 4.6 Order Statistics Section 4.6 Order Statistics Beta Function - Variance Beta Distribution - Summary Let X Beta(r; s) then If X Beta(r; s) then ∼ ∼ Z 1 1 1 E(X 2) = x2 xr−1(1 − x)s−1dx f (x) = x r−1(1 − x)s−1 0 B(r; s) B(r; s) B(r + 2; s) (r + 1)!(s − 1)! (r + s − 1)! Z x 1 B (r; s) = = F (x) = x r−1(1 − x)s−1dx = x B(r; s) (r + s + 1)! (r − 1)!(s − 1)! 0 B(r; s) B(r; s) (r + 1)r = (r + s + 1)(r + s) Z 1 (r − 1)!(s − 1)! Γ(r)Γ(s) B(r; s) = x r−1(1 − x)s−1dx = = (r + s − 1)! Γ(r + s) 2 2 0 Var(X ) = E(X ) − E(X ) Z x r−1 s−1 (r + 1)r r 2 Bx (r; s) = x (1 − x) dx = − 0 (r + s + 1)(r + s) (r + s)2 (r + 1)r(r + s) − r 2(r + s + 1) = r 2 E(X ) = (r + s + 1)(r + s) r + s rs = rs 2 Var(X ) = (r + s + 1)(r + s) (r + s)2(r + s + 1) Statistics 104 (Colin Rundel) Lecture 15 March 14, 2012 12 / 24 Statistics 104 (Colin Rundel) Lecture 15 March 14, 2012 13 / 24 Section 4.6 Order Statistics Section 4.6 Order Statistics Minimum of Exponentials Maximum of Exponentials iid iid Let X1; X2;:::; Xn Exp(λ), we previously derived a more general result Let X1; X2;:::; Xn Exp(λ) then the density of X(n) is given by where the X 's were∼ not identically distributed and showed that ∼ min(X1;:::; Xn) Exp(λ1 + + λn) = Exp(nλ) in this more restricted n−1 ∼ ··· f(n)(x) = nf (x)F (x) case. n−1 = n λe−λx 1 − e−λx Lets confirm that result using our new more general methods n−1 f(1)(x) = nf (x)(1 − F (x)) n−1 Which we can't do much with, instead we can try the cdf of the maximum. = n λe−λx 1 − [1 − e−λx ] n−1 = nλe−λx e−λx ] n = nλ e−λx ] = nλe−nλx Which is the density for Exp(nλ). Statistics 104 (Colin Rundel) Lecture 15 March 14, 2012 14 / 24 Statistics 104 (Colin Rundel) Lecture 15 March 14, 2012 15 / 24 Section 4.6 Order Statistics Section 4.6 Order Statistics Maximum of Exponentials, cont.
Load more

Recommended publications
  • Concentration and Consistency Results for Canonical and Curved Exponential-Family Models of Random Graphs
    CONCENTRATION AND CONSISTENCY RESULTS FOR CANONICAL AND CURVED EXPONENTIAL-FAMILY MODELS OF RANDOM GRAPHS BY MICHAEL SCHWEINBERGER AND JONATHAN STEWART Rice University Statistical inference for exponential-family models of random graphs with dependent edges is challenging. We stress the importance of additional structure and show that additional structure facilitates statistical inference. A simple example of a random graph with additional structure is a random graph with neighborhoods and local dependence within neighborhoods. We develop the first concentration and consistency results for maximum likeli- hood and M-estimators of a wide range of canonical and curved exponential- family models of random graphs with local dependence. All results are non- asymptotic and applicable to random graphs with finite populations of nodes, although asymptotic consistency results can be obtained as well. In addition, we show that additional structure can facilitate subgraph-to-graph estimation, and present concentration results for subgraph-to-graph estimators. As an ap- plication, we consider popular curved exponential-family models of random graphs, with local dependence induced by transitivity and parameter vectors whose dimensions depend on the number of nodes. 1. Introduction. Models of network data have witnessed a surge of interest in statistics and related areas [e.g., 31]. Such data arise in the study of, e.g., social networks, epidemics, insurgencies, and terrorist networks. Since the work of Holland and Leinhardt in the 1970s [e.g., 21], it is known that network data exhibit a wide range of dependencies induced by transitivity and other interesting network phenomena [e.g., 39]. Transitivity is a form of triadic closure in the sense that, when a node k is connected to two distinct nodes i and j, then i and j are likely to be connected as well, which suggests that edges are dependent [e.g., 39].
    [Show full text]
  • Use of Statistical Tables
    TUTORIAL | SCOPE USE OF STATISTICAL TABLES Lucy Radford, Jenny V Freeman and Stephen J Walters introduce three important statistical distributions: the standard Normal, t and Chi-squared distributions PREVIOUS TUTORIALS HAVE LOOKED at hypothesis testing1 and basic statistical tests.2–4 As part of the process of statistical hypothesis testing, a test statistic is calculated and compared to a hypothesised critical value and this is used to obtain a P- value. This P-value is then used to decide whether the study results are statistically significant or not. It will explain how statistical tables are used to link test statistics to P-values. This tutorial introduces tables for three important statistical distributions (the TABLE 1. Extract from two-tailed standard Normal, t and Chi-squared standard Normal table. Values distributions) and explains how to use tabulated are P-values corresponding them with the help of some simple to particular cut-offs and are for z examples. values calculated to two decimal places. STANDARD NORMAL DISTRIBUTION TABLE 1 The Normal distribution is widely used in statistics and has been discussed in z 0.00 0.01 0.02 0.03 0.050.04 0.05 0.06 0.07 0.08 0.09 detail previously.5 As the mean of a Normally distributed variable can take 0.00 1.0000 0.9920 0.9840 0.9761 0.9681 0.9601 0.9522 0.9442 0.9362 0.9283 any value (−∞ to ∞) and the standard 0.10 0.9203 0.9124 0.9045 0.8966 0.8887 0.8808 0.8729 0.8650 0.8572 0.8493 deviation any positive value (0 to ∞), 0.20 0.8415 0.8337 0.8259 0.8181 0.8103 0.8206 0.7949 0.7872 0.7795 0.7718 there are an infinite number of possible 0.30 0.7642 0.7566 0.7490 0.7414 0.7339 0.7263 0.7188 0.7114 0.7039 0.6965 Normal distributions.
    [Show full text]
  • The Probability Lifesaver: Order Statistics and the Median Theorem
    The Probability Lifesaver: Order Statistics and the Median Theorem Steven J. Miller December 30, 2015 Contents 1 Order Statistics and the Median Theorem 3 1.1 Definition of the Median 5 1.2 Order Statistics 10 1.3 Examples of Order Statistics 15 1.4 TheSampleDistributionoftheMedian 17 1.5 TechnicalboundsforproofofMedianTheorem 20 1.6 TheMedianofNormalRandomVariables 22 2 • Greetings again! In this supplemental chapter we develop the theory of order statistics in order to prove The Median Theorem. This is a beautiful result in its own, but also extremely important as a substitute for the Central Limit Theorem, and allows us to say non- trivial things when the CLT is unavailable. Chapter 1 Order Statistics and the Median Theorem The Central Limit Theorem is one of the gems of probability. It’s easy to use and its hypotheses are satisfied in a wealth of problems. Many courses build towards a proof of this beautiful and powerful result, as it truly is ‘central’ to the entire subject. Not to detract from the majesty of this wonderful result, however, what happens in those instances where it’s unavailable? For example, one of the key assumptions that must be met is that our random variables need to have finite higher moments, or at the very least a finite variance. What if we were to consider sums of Cauchy random variables? Is there anything we can say? This is not just a question of theoretical interest, of mathematicians generalizing for the sake of generalization. The following example from economics highlights why this chapter is more than just of theoretical interest.
    [Show full text]
  • Notes Mean, Median, Mode & Range
    Notes Mean, Median, Mode & Range How Do You Use Mode, Median, Mean, and Range to Describe Data? There are many ways to describe the characteristics of a set of data. The mode, median, and mean are all called measures of central tendency. These measures of central tendency and range are described in the table below. The mode of a set of data Use the mode to show which describes which value occurs value in a set of data occurs most frequently. If two or more most often. For the set numbers occur the same number {1, 1, 2, 3, 5, 6, 10}, of times and occur more often the mode is 1 because it occurs Mode than all the other numbers in the most frequently. set, those numbers are all modes for the data set. If each number in the set occurs the same number of times, the set of data has no mode. The median of a set of data Use the median to show which describes what value is in the number in a set of data is in the middle if the set is ordered from middle when the numbers are greatest to least or from least to listed in order. greatest. If there are an even For the set {1, 1, 2, 3, 5, 6, 10}, number of values, the median is the median is 3 because it is in the average of the two middle the middle when the numbers are Median values. Half of the values are listed in order. greater than the median, and half of the values are less than the median.
    [Show full text]
  • Sampling Student's T Distribution – Use of the Inverse Cumulative
    Sampling Student’s T distribution – use of the inverse cumulative distribution function William T. Shaw Department of Mathematics, King’s College, The Strand, London WC2R 2LS, UK With the current interest in copula methods, and fat-tailed or other non-normal distributions, it is appropriate to investigate technologies for managing marginal distributions of interest. We explore “Student’s” T distribution, survey its simulation, and present some new techniques for simulation. In particular, for a given real (not necessarily integer) value n of the number of degrees of freedom, −1 we give a pair of power series approximations for the inverse, Fn ,ofthe cumulative distribution function (CDF), Fn.Wealsogivesomesimpleandvery fast exact and iterative techniques for defining this function when n is an even −1 integer, based on the observation that for such cases the calculation of Fn amounts to the solution of a reduced-form polynomial equation of degree n − 1. We also explain the use of Cornish–Fisher expansions to define the inverse CDF as the composition of the inverse CDF for the normal case with a simple polynomial map. The methods presented are well adapted for use with copula and quasi-Monte-Carlo techniques. 1 Introduction There is much interest in many areas of financial modeling on the use of copulas to glue together marginal univariate distributions where there is no easy canonical multivariate distribution, or one wishes to have flexibility in the mechanism for combination. One of the more interesting marginal distributions is the “Student’s” T distribution. This statistical distribution was published by W. Gosset in 1908.
    [Show full text]
  • Extreme Value Theory and Backtest Overfitting in Finance
    View metadata, citation and similar papers at core.ac.uk brought to you by CORE provided by Bowdoin College Bowdoin College Bowdoin Digital Commons Honors Projects Student Scholarship and Creative Work 2015 Extreme Value Theory and Backtest Overfitting in Finance Daniel C. Byrnes [email protected] Follow this and additional works at: https://digitalcommons.bowdoin.edu/honorsprojects Part of the Statistics and Probability Commons Recommended Citation Byrnes, Daniel C., "Extreme Value Theory and Backtest Overfitting in Finance" (2015). Honors Projects. 24. https://digitalcommons.bowdoin.edu/honorsprojects/24 This Open Access Thesis is brought to you for free and open access by the Student Scholarship and Creative Work at Bowdoin Digital Commons. It has been accepted for inclusion in Honors Projects by an authorized administrator of Bowdoin Digital Commons. For more information, please contact [email protected]. Extreme Value Theory and Backtest Overfitting in Finance An Honors Paper Presented for the Department of Mathematics By Daniel Byrnes Bowdoin College, 2015 ©2015 Daniel Byrnes Acknowledgements I would like to thank professor Thomas Pietraho for his help in the creation of this thesis. The revisions and suggestions made by several members of the math faculty were also greatly appreciated. I would also like to thank the entire department for their support throughout my time at Bowdoin. 1 Contents 1 Abstract 3 2 Introduction4 3 Background7 3.1 The Sharpe Ratio.................................7 3.2 Other Performance Measures.......................... 10 3.3 Example of an Overfit Strategy......................... 11 4 Modes of Convergence for Random Variables 13 4.1 Random Variables and Distributions...................... 13 4.2 Convergence in Distribution..........................
    [Show full text]
  • Approximating the Distribution of the Product of Two Normally Distributed Random Variables
    S S symmetry Article Approximating the Distribution of the Product of Two Normally Distributed Random Variables Antonio Seijas-Macías 1,2 , Amílcar Oliveira 2,3 , Teresa A. Oliveira 2,3 and Víctor Leiva 4,* 1 Departamento de Economía, Universidade da Coruña, 15071 A Coruña, Spain; [email protected] 2 CEAUL, Faculdade de Ciências, Universidade de Lisboa, 1649-014 Lisboa, Portugal; [email protected] (A.O.); [email protected] (T.A.O.) 3 Departamento de Ciências e Tecnologia, Universidade Aberta, 1269-001 Lisboa, Portugal 4 Escuela de Ingeniería Industrial, Pontificia Universidad Católica de Valparaíso, Valparaíso 2362807, Chile * Correspondence: [email protected] or [email protected] Received: 21 June 2020; Accepted: 18 July 2020; Published: 22 July 2020 Abstract: The distribution of the product of two normally distributed random variables has been an open problem from the early years in the XXth century. First approaches tried to determinate the mathematical and statistical properties of the distribution of such a product using different types of functions. Recently, an improvement in computational techniques has performed new approaches for calculating related integrals by using numerical integration. Another approach is to adopt any other distribution to approximate the probability density function of this product. The skew-normal distribution is a generalization of the normal distribution which considers skewness making it flexible. In this work, we approximate the distribution of the product of two normally distributed random variables using a type of skew-normal distribution. The influence of the parameters of the two normal distributions on the approximation is explored. When one of the normally distributed variables has an inverse coefficient of variation greater than one, our approximation performs better than when both normally distributed variables have inverse coefficients of variation less than one.
    [Show full text]
  • Calculating Variance and Standard Deviation
    VARIANCE AND STANDARD DEVIATION Recall that the range is the difference between the upper and lower limits of the data. While this is important, it does have one major disadvantage. It does not describe the variation among the variables. For instance, both of these sets of data have the same range, yet their values are definitely different. 90, 90, 90, 98, 90 Range = 8 1, 6, 8, 1, 9, 5 Range = 8 To better describe the variation, we will introduce two other measures of variation—variance and standard deviation (the variance is the square of the standard deviation). These measures tell us how much the actual values differ from the mean. The larger the standard deviation, the more spread out the values. The smaller the standard deviation, the less spread out the values. This measure is particularly helpful to teachers as they try to find whether their students’ scores on a certain test are closely related to the class average. To find the standard deviation of a set of values: a. Find the mean of the data b. Find the difference (deviation) between each of the scores and the mean c. Square each deviation d. Sum the squares e. Dividing by one less than the number of values, find the “mean” of this sum (the variance*) f. Find the square root of the variance (the standard deviation) *Note: In some books, the variance is found by dividing by n. In statistics it is more useful to divide by n -1. EXAMPLE Find the variance and standard deviation of the following scores on an exam: 92, 95, 85, 80, 75, 50 SOLUTION First we find the mean of the data: 92+95+85+80+75+50 477 Mean = = = 79.5 6 6 Then we find the difference between each score and the mean (deviation).
    [Show full text]
  • Lecture 14: Order Statistics; Conditional Expectation
    Department of Mathematics Ma 3/103 KC Border Introduction to Probability and Statistics Winter 2017 Lecture 14: Order Statistics; Conditional Expectation Relevant textbook passages: Pitman [5]: Section 4.6 Larsen–Marx [2]: Section 3.11 14.1 Order statistics Pitman [5]: Given a random vector (X(1,...,Xn) on the probability) space (S, E,P ), for each s ∈ S, sort the § 4.6 components into a vector X(1)(s),...,X(n)(s) satisfying X(1)(s) 6 X(2)(s) 6 ··· 6 X(n)(s). The vector (X(1),...,X(n)) is called the vector of order statistics of (X1,...,Xn). Equivalently, { } X(k) = min max{Xj : j ∈ J} : J ⊂ {1, . , n} & |J| = k . Order statistics play an important role in the study of auctions, among other things. 14.2 Marginal Distribution of Order Statistics From here on, we shall assume that the original random variables X1,...,Xn are independent and identically distributed. Let X1,...,Xn be independent and identically distributed random variables with common cumulative distribution function F ,( and let (X) (1),...,X(n)) be the vector of order statistics of X1,...,Xn. By breaking the event X(k) 6 x into simple disjoint subevents, we get ( ) X(k) 6 x = ( ) X(n) 6 x ∪ ( ) X(n) > x, X(n−1) 6 x . ∪ ( ) X(n) > x, . , X(j+1) > x, X(j) 6 x . ∪ ( ) X(n) > x, . , X(k+1) > x, X(k) 6 x . 14–1 Ma 3/103 Winter 2017 KC Border Order Statistics; Conditional Expectation 14–2 Each of these subevents is disjoint from the ones above it, and each has a binomial probability: ( ) X(n) > x, .
    [Show full text]
  • Medians and Order Statistics
    Medians and Order Statistics CLRS Chapter 9 1 What Are Order Statistics? The k-th order statistic is the k-th smallest element of an array. 3 4 13 14 23 27 41 54 65 75 8th order statistic ⎢n⎥ ⎢ ⎥ The lower median is the ⎣ 2 ⎦ -th order statistic ⎡n⎤ ⎢ ⎥ The upper median is the ⎢ 2 ⎥ -th order statistic If n is odd, lower and upper median are the same 3 4 13 14 23 27 41 54 65 75 lower median upper median What are Order Statistics? Selecting ith-ranked item from a collection. – First: i = 1 – Last: i = n ⎢n⎥ ⎡n⎤ – Median(s): i = ⎢ ⎥,⎢ ⎥ ⎣2⎦ ⎢2⎥ 3 Order Statistics Overview • Assume collection is unordered, otherwise trivial. find ith order stat = A[i] • Can sort first – Θ(n lg n), but can do better – Θ(n). • I can find max and min in Θ(n) time (obvious) • Can we find any order statistic in linear time? (not obvious!) 4 Order Statistics Overview How can we modify Quicksort to obtain expected-case Θ(n)? ? ? Pivot, partition, but recur only on one set of data. No join. 5 Using the Pivot Idea • Randomized-Select(A[p..r],i) looking for ith o.s. if p = r return A[p] q <- Randomized-Partition(A,p,r) k <- q-p+1 the size of the left partition if i=k then the pivot value is the answer return A[q] else if i < k then the answer is in the front return Randomized-Select(A,p,q-1,i) else then the answer is in the back half return Randomized-Select(A,q+1,r,i-k) 6 Randomized Selection • Analyzing RandomizedSelect() – Worst case: partition always 0:n-1 T(n) = T(n-1) + O(n) = O(n2) • No better than sorting! – “Best” case: suppose a 9:1 partition T(n) = T(9n/10)
    [Show full text]
  • Parametric (Theoretical) Probability Distributions. (Wilks, Ch. 4) Discrete
    6 Parametric (theoretical) probability distributions. (Wilks, Ch. 4) Note: parametric: assume a theoretical distribution (e.g., Gauss) Non-parametric: no assumption made about the distribution Advantages of assuming a parametric probability distribution: Compaction: just a few parameters Smoothing, interpolation, extrapolation x x + s Parameter: e.g.: µ,σ population mean and standard deviation Statistic: estimation of parameter from sample: x,s sample mean and standard deviation Discrete distributions: (e.g., yes/no; above normal, normal, below normal) Binomial: E = 1(yes or success); E = 0 (no, fail). These are MECE. 1 2 P(E ) = p P(E ) = 1− p . Assume N independent trials. 1 2 How many “yes” we can obtain in N independent trials? x = (0,1,...N − 1, N ) , N+1 possibilities. Note that x is like a dummy variable. ⎛ N ⎞ x N − x ⎛ N ⎞ N ! P( X = x) = p 1− p , remember that = , 0!= 1 ⎜ x ⎟ ( ) ⎜ x ⎟ x!(N x)! ⎝ ⎠ ⎝ ⎠ − Bernouilli is the binomial distribution with a single trial, N=1: x = (0,1), P( X = 0) = 1− p, P( X = 1) = p Geometric: Number of trials until next success: i.e., x-1 fails followed by a success. x−1 P( X = x) = (1− p) p x = 1,2,... 7 Poisson: Approximation of binomial for small p and large N. Events occur randomly at a constant rate (per N trials) µ = Np . The rate per trial p is low so that events in the same period (N trials) are approximately independent. Example: assume the probability of a tornado in a certain county on a given day is p=1/100.
    [Show full text]
  • Exponential Families and Theoretical Inference
    EXPONENTIAL FAMILIES AND THEORETICAL INFERENCE Bent Jørgensen Rodrigo Labouriau August, 2012 ii Contents Preface vii Preface to the Portuguese edition ix 1 Exponential families 1 1.1 Definitions . 1 1.2 Analytical properties of the Laplace transform . 11 1.3 Estimation in regular exponential families . 14 1.4 Marginal and conditional distributions . 17 1.5 Parametrizations . 20 1.6 The multivariate normal distribution . 22 1.7 Asymptotic theory . 23 1.7.1 Estimation . 25 1.7.2 Hypothesis testing . 30 1.8 Problems . 36 2 Sufficiency and ancillarity 47 2.1 Sufficiency . 47 2.1.1 Three lemmas . 48 2.1.2 Definitions . 49 2.1.3 The case of equivalent measures . 50 2.1.4 The general case . 53 2.1.5 Completeness . 56 2.1.6 A result on separable σ-algebras . 59 2.1.7 Sufficiency of the likelihood function . 60 2.1.8 Sufficiency and exponential families . 62 2.2 Ancillarity . 63 2.2.1 Definitions . 63 2.2.2 Basu's Theorem . 65 2.3 First-order ancillarity . 67 2.3.1 Examples . 67 2.3.2 Main results . 69 iii iv CONTENTS 2.4 Problems . 71 3 Inferential separation 77 3.1 Introduction . 77 3.1.1 S-ancillarity . 81 3.1.2 The nonformation principle . 83 3.1.3 Discussion . 86 3.2 S-nonformation . 91 3.2.1 Definition . 91 3.2.2 S-nonformation in exponential families . 96 3.3 G-nonformation . 99 3.3.1 Transformation models . 99 3.3.2 Definition of G-nonformation . 103 3.3.3 Cox's proportional risks model .
    [Show full text]