1 Selected Concepts from Probability
Total Page:16
File Type:pdf, Size:1020Kb

Load more
Recommended publications
-
A Test for Singularity 1
STATISTIII & PROBABIUTY Lk'TTERS ELSEVIER Statistics & Probability Letters 40 (1998) 215-226 A test for singularity 1 Attila Frigyesi *, Ola H6ssjer Institute for Matematisk Statistik, Lunds University, Box 118, 22100 Lund, Sweden Received March 1997; received in revised form February 1998 Abstract The LP-norm and other functionals of the kernel estimate of density functions (as functions of the bandwidth) are studied as means to test singularity. Given an absolutely continuous distribution function, the LP-norm, and the other studied functionals, will converge under some mild side-constraints as the bandwidth decreases. In the singular case, the functionals will diverge. These properties may be used to test whether data comes from an arbitrarily large class of absolutely continuous distributions or not. (~) 1998 Elsevier Science B.V. All rights reserved AMS classification." primary 60E05; secondary 28A99; 62G07 Keywords: Singularity; Absolute continuity; Kernel density estimation 1. Introduction In this paper we aim to test whether a given set of data (= {XI ..... X,}), with a marginal distribution #, stems from an absolutely continuous distribution function or from a singular one. A reference to this problem is by Donoho (1988). Consider a probability measure # on (g~P,~(~P)), where ~(~P) is the Borel a-algebra on R p. The measure tt is said to be singular with respect to the Lebesgue measure 2 if there exists a measurable set A such that /~(AC)=0 and 2(A)=0. If we only require #(A)>0 and 2(A)=0, then /~ is said to have a singular part. If tt does not have a singular part, then it is absolutely continuous with respect to 2. -
Vector Spaces
ApPENDIX A Vector Spaces This appendix reviews some of the basic definitions and properties of vectm spaces. It is presumed that, with the possible exception of Theorem A.14, all of the material presented here is familiar to the reader. Definition A.I. A set.A c Rn is a vector space if, for any x, y, Z E .A and scalars a, 13, operations of vector addition and scalar multiplication are defined such that: (1) (x + y) + Z = x + (y + z). (2) x + y = y + x. (3) There exists a vector 0 E.A such that x + 0 = x = 0 + x. (4) For any XE.A there exists y = -x such that x + y = 0 = y + x. (5) a(x + y) = ax + ay. (6) (a + f3)x = ax + f3x. (7) (af3)x = a(f3x). (8) 1· x = x. Definition A.2. Let .A be a vector space and let JV be a set with JV c .A. JV is a subspace of .A if and only if JV is a vector space. Theorem A.3. Let .A be a vector space and let JV be a nonempty subset of .A. If JV is closed under addition and scalar multiplication, then JV is a subspace of .A. Theorem A.4. Let .A be a vector space and let Xl' ••• , Xr be in .A. The set of all linear combinations of Xl' .•• ,xr is a subspace of .A. Appendix A. Vector Spaces 325 Definition A.S. The set of all linear combinations of Xl'OO',X, is called the space spanned by Xl' .. -
Randomized Algorithms
Chapter 9 Randomized Algorithms randomized The theme of this chapter is madendrizo algorithms. These are algorithms that make use of randomness in their computation. You might know of quicksort, which is efficient on average when it uses a random pivot, but can be bad for any pivot that is selected without randomness. Analyzing randomized algorithms can be difficult, so you might wonder why randomiza- tion in algorithms is so important and worth the extra effort. Well it turns out that for certain problems randomized algorithms are simpler or faster than algorithms that do not use random- ness. The problem of primality testing (PT), which is to determine if an integer is prime, is a good example. In the late 70s Miller and Rabin developed a famous and simple random- ized algorithm for the problem that only requires polynomial work. For over 20 years it was not known whether the problem could be solved in polynomial work without randomization. Eventually a polynomial time algorithm was developed, but it is much more complicated and computationally more costly than the randomized version. Hence in practice everyone still uses the randomized version. There are many other problems in which a randomized solution is simpler or cheaper than the best non-randomized solution. In this chapter, after covering the prerequisite background, we will consider some such problems. The first we will consider is the following simple prob- lem: Question: How many comparisons do we need to find the top two largest numbers in a sequence of n distinct numbers? Without the help of randomization, there is a trivial algorithm for finding the top two largest numbers in a sequence that requires about 2n − 3 comparisons. -
MA3K0 - High-Dimensional Probability
MA3K0 - High-Dimensional Probability Lecture Notes Stefan Adams i 2020, update 06.05.2020 Notes are in final version - proofreading not completed yet! Typos and errors will be updated on a regular basis Contents 1 Prelimaries on Probability Theory 1 1.1 Random variables . .1 1.2 Classical Inequalities . .4 1.3 Limit Theorems . .6 2 Concentration inequalities for independent random variables 8 2.1 Why concentration inequalities . .8 2.2 Hoeffding’s Inequality . 10 2.3 Chernoff’s Inequality . 13 2.4 Sub-Gaussian random variables . 15 2.5 Sub-Exponential random variables . 22 3 Random vectors in High Dimensions 27 3.1 Concentration of the Euclidean norm . 27 3.2 Covariance matrices and Principal Component Analysis (PCA) . 30 3.3 Examples of High-Dimensional distributions . 32 3.4 Sub-Gaussian random variables in higher dimensions . 34 3.5 Application: Grothendieck’s inequality . 36 4 Random Matrices 36 4.1 Geometrics concepts . 36 4.2 Concentration of the operator norm of random matrices . 38 4.3 Application: Community Detection in Networks . 41 4.4 Application: Covariance Estimation and Clustering . 41 5 Concentration of measure - general case 43 5.1 Concentration by entropic techniques . 43 5.2 Concentration via Isoperimetric Inequalities . 50 5.3 Some matrix calculus and covariance estimation . 54 6 Basic tools in high-dimensional probability 57 6.1 Decoupling . 57 6.2 Concentration for Anisotropic random vectors . 62 6.3 Symmetrisation . 63 7 Random Processes 64 ii Preface Introduction We discuss an elegant argument that showcases the -
HIGHER MOMENTS of BANACH SPACE VALUED RANDOM VARIABLES 1. Introduction Let X Be a Random Variable with Values in a Banach Space
HIGHER MOMENTS OF BANACH SPACE VALUED RANDOM VARIABLES SVANTE JANSON AND STEN KAIJSER Abstract. We define the k:th moment of a Banach space valued ran- dom variable as the expectation of its k:th tensor power; thus the mo- ment (if it exists) is an element of a tensor power of the original Banach space. We study both the projective and injective tensor products, and their relation. Moreover, in order to be general and flexible, we study three different types of expectations: Bochner integrals, Pettis integrals and Dunford integrals. One of the problems studied is whether two random variables with the same injective moments (of a given order) necessarily have the same projective moments; this is of interest in applications. We show that this holds if the Banach space has the approximation property, but not in general. Several sections are devoted to results in special Banach spaces, in- cluding Hilbert spaces, CpKq and Dr0; 1s. The latter space is non- separable, which complicates the arguments, and we prove various pre- liminary results on e.g. measurability in Dr0; 1s that we need. One of the main motivations of this paper is the application to Zolotarev metrics and their use in the contraction method. This is sketched in an appendix. 1. Introduction Let X be a random variable with values in a Banach space B. To avoid measurability problems, we assume for most of this section for simplicity that B is separable and X Borel measurable; see Section 3 for measurability in the general case. Moreover, for definiteness, we consider real Banach spaces only; the complex case is similar. -
Quantization of Self-Similar Probability Measures and Optimal Quantizers by Do˘Ganc¸¨Omez Department of Mathematics, North Dakota State University
Quantization of self-similar probability measures and optimal quantizers by Do˘ganC¸¨omez Department of Mathematics, North Dakota State University Outline: 1. Five-minute Crash Course in Probability 2. Probability Measures and Distributions 3. Voronoi Partitions 4. Centers and Moments of Probability Distributions 5. Quantization 6. Some Recent Results n Throughout we will deal with R with a norm k k: In particular k kr will be the r-norm for some 1 ≤ r < 1: Some of the information below are adopted from Foundations of Quantization for Probability Distributions, by S. Graf and H. Luschgy, Lecture Notes in Mathematics, # 1730, Springer, 2000. 1. Five-minute Crash Course in Probability. (Ω; F;P ) : a probability space. A measurable function X :Ω ! R is called a random variable. R The expectation (or mean) of X is defined as E(X) := Ω XdP: (i) E(Xk): k-th moment of X; and E(jXjk): k-th absolute moment of X: (ii) E[(X − E(X))k]: k-th central moment of X; and E[jX − E(X)jk]: k-th absolute central moment of X: Note: If k = 1;E[(X − E(X))k] = 0: When k = 2;E[(X − E(X))2] is called the variance of X: If X is a random variable, then it induces a probability measure on Borel subsets of R by −1 PX := P ◦ X : (Sometimes PX is called a law.) The distribution (function) of a r.v. X is a function F : R ! R such that F (x) = PX (−∞; x] = P (f! : X(!) ≤ xg): Observe that F (b) − F (a) = PX (a; b]: Fact. -
Control Policy with Autocorrelated Noise in Reinforcement Learning for Robotics
International Journal of Machine Learning and Computing, Vol. 5, No. 2, April 2015 Control Policy with Autocorrelated Noise in Reinforcement Learning for Robotics Paweł Wawrzyński primitives by means of RL algorithms based on MDP Abstract—Direct application of reinforcement learning in framework, but it does not alleviate the problem of robot robotics rises the issue of discontinuity of control signal. jerking. The current paper is intended to fill this gap. Consecutive actions are selected independently on random, In this paper, control policy is introduced that may undergo which often makes them excessively far from one another. Such reinforcement learning and have the following properties: control is hardly ever appropriate in robots, it may even lead to their destruction. This paper considers a control policy in which It is applicable to robot control optimization. consecutive actions are modified by autocorrelated noise. That It does not lead to robot jerking. policy generally solves the aforementioned problems and it is readily applicable in robots. In the experimental study it is It can be optimized by any RL algorithm that is designed applied to three robotic learning control tasks: Cart-Pole to optimize a classical stochastic control policy. SwingUp, Half-Cheetah, and a walking humanoid. The policy introduced here is based on a deterministic transformation of state combined with a random element in Index Terms—Machine learning, reinforcement learning, actorcritics, robotics. the form of a specific stochastic process, namely the moving average. The paper is organized as follows. In Section II the I. INTRODUCTION problem of our interest is defined. Section III presents the main contribution of this paper i.e., a stochastic control policy Reinforcement learning (RL) addresses the problem of an that prevents robot jerking while learning. -
CONVERGENCE RATES of MARKOV CHAINS 1. Orientation 1
CONVERGENCE RATES OF MARKOV CHAINS STEVEN P. LALLEY CONTENTS 1. Orientation 1 1.1. Example 1: Random-to-Top Card Shuffling 1 1.2. Example 2: The Ehrenfest Urn Model of Diffusion 2 1.3. Example 3: Metropolis-Hastings Algorithm 4 1.4. Reversible Markov Chains 5 1.5. Exercises: Reversiblity, Symmetries and Stationary Distributions 8 2. Coupling 9 2.1. Coupling and Total Variation Distance 9 2.2. Coupling Constructions and Convergence of Markov Chains 10 2.3. Couplings for the Ehrenfest Urn and Random-to-Top Shuffling 12 2.4. The Coupon Collector’s Problem 13 2.5. Exercises 15 2.6. Convergence Rates for the Ehrenfest Urn and Random-to-Top 16 2.7. Exercises 17 3. Spectral Analysis 18 3.1. Transition Kernel of a Reversible Markov Chain 18 3.2. Spectrum of the Ehrenfest random walk 21 3.3. Rate of convergence of the Ehrenfest random walk 23 1. ORIENTATION Finite-state Markov chains have stationary distributions, and irreducible, aperiodic, finite- state Markov chains have unique stationary distributions. Furthermore, for any such chain the n−step transition probabilities converge to the stationary distribution. In various ap- plications – especially in Markov chain Monte Carlo, where one runs a Markov chain on a computer to simulate a random draw from the stationary distribution – it is desirable to know how many steps it takes for the n−step transition probabilities to become close to the stationary probabilities. These notes will introduce several of the most basic and important techniques for studying this problem, coupling and spectral analysis. We will focus on two Markov chains for which the convergence rate is of particular interest: (1) the random-to-top shuffling model and (2) the Ehrenfest urn model. -
Multivariate Scale-Mixed Stable Distributions and Related Limit
Multivariate scale-mixed stable distributions and related limit theorems∗ Victor Korolev,† Alexander Zeifman‡ Abstract In the paper, multivariate probability distributions are considered that are representable as scale mixtures of multivariate elliptically contoured stable distributions. It is demonstrated that these distributions form a special subclass of scale mixtures of multivariate elliptically contoured normal distributions. Some properties of these distributions are discussed. Main attention is paid to the representations of the corresponding random vectors as products of independent random variables. In these products, relations are traced of the distributions of the involved terms with popular probability distributions. As examples of distributions of the class of scale mixtures of multivariate elliptically contoured stable distributions, multivariate generalized Linnik distributions are considered in detail. Their relations with multivariate ‘ordinary’ Linnik distributions, multivariate normal, stable and Laplace laws as well as with univariate Mittag-Leffler and generalized Mittag-Leffler distributions are discussed. Limit theorems are proved presenting necessary and sufficient conditions for the convergence of the distributions of random sequences with independent random indices (including sums of a random number of random vectors and multivariate statistics constructed from samples with random sizes) to scale mixtures of multivariate elliptically contoured stable distributions. The property of scale- mixed multivariate stable -
Random Fields, Fall 2014 1
Random fields, Fall 2014 1 fMRI brain scan Ocean waves Nobel prize 2996 to The oceans cover 72% of John C. Mather the earth’s surface. George F. Smoot Essential for life on earth, “for discovery of the and huge economic blackbody form and importance through anisotropy of the cosmic fishing, transportation, microwave background oil and gas extraction radiation" PET brain scan TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: A The course • Kolmogorov existence theorem, separable processes, measurable processes • Stationarity and isotropy • Orthogonal and spectral representations • Geometry • Exceedance sets • Rice formula • Slepian models Literature An unfinished manuscript “Applications of RANDOM FIELDS AND GEOMETRY: Foundations and Case Studies” by Robert Adler, Jonathan Taylor, and Keith Worsley. Complementary literature: “Level sets and extrema of random processes and fields” by Jean- Marc Azais and Mario Wschebor, Wiley, 2009 “Asymptotic Methods in the Theory of Gaussian Processes” by Vladimir Piterbarg, American Mathematical Society, ser. Translations of Mathematical Monographs, Vol. 148, 1995 “Random fields and Geometry” by Robert Adler and Jonathan Taylor, Springer 2007 Slides for ATW ch 2, p 24-39 Exercises: 2.8.1, 2.8.2, 2.8.3, 2.8.4, 2.8.5, 2.8.6 + excercises in slides Stochastic convergence (assumed known) 푎.푠. Almost sure convergence: 푋푛 X 퐿2 Mean square convergence: 푋푛 X 푃 Convergence in probability: 푋푛 X 푑 푑 푑 Convergence in distribution: 푋푛 X, 퐹푛 F, 푃푛 P, … 푎.푠. 푃 • 푋푛 X ⇒ 푋푛 푋 퐿2 푃 • 푋푛 X ⇒ 푋푛 푋 푃 퐿2 • 푋푛 푋 plus uniform integrability ⇒ 푋푛 X 푃 푎.푠. -
THE DERIVATION of the CHI-Square TEST of GOODNESS of FIT
THE DERIVATION OF THE CHI-sQUARE TEST OF GOODNESS OF FIT Jacqueline Prillaman Submitted to McGill University in partial fulfilment of the requirements for the de6ree of Master of Science, August, 1956 I wish to express my gratitude to Prof. W. Kozakiewicz for his generous assistance and valuable guidance in the preparation of this thesis. TABLE OF CONTENTS Introduction .................................................. 1 Part One: An Intuitive Approach to the Chi-Square Test of Goodness of Fit ............................ 5 I: Presentation of the Test II: The Number of Degrees of Fre?dom III: Greenhood's Derivation of the Chi-Square Test of Goodness of Fit Part Two: A Rigorous De velopment of the Chi- Square Test of Goodness of Fit .......................... 22 I: The Normal Distribution in the Space of n-Dimensions II: The Limiting Distribution of the Chi-Square Variable by the Method of Characteristic Functions Introduction Let ~, ~, . ., x be s independent sample values drawn from 5 a population of normally distributed values with zero mean and unit variance. Then the variable 2 2 2 u = x1 + x 2 + . + xs is said to follow the ~t distribution with parameter s. This para meter is called the number of degrees of freedom for reasons which will be explained later. Using the moment-generating function we shall prove that s s-2 u 2 2- (1) u e 2 u > o. f(u) = r(~) (!) Since the basic variable, x, in the population has the frequency function f(x) = m.- e , we have, for G < l2 , M 2(&) = Mx2(&) = xi +• 2 ~ (1-2&) = 1 e -2 dx rh 1-· 2 1 . -
Random Polynomials Over Finite Fields
RANDOM POLYNOMIALS OVER FINITE FIELDS by Andrew Sharkey A thesis submitted to the Faculty of Science at the University of Glasgow for the degree of . Doctor of Philosophy June 22, 1999 ©Andrew Sharkey 1999 ProQuest Number: 13834241 All rights reserved INFORMATION TO ALL USERS The quality of this reproduction is dependent upon the quality of the copy submitted. In the unlikely event that the author did not send a complete manuscript and there are missing pages, these will be noted. Also, if material had to be removed, a note will indicate the deletion. uest ProQuest 13834241 Published by ProQuest LLC(2019). Copyright of the Dissertation is held by the Author. All rights reserved. This work is protected against unauthorized copying under Title 17, United States Code Microform Edition © ProQuest LLC. ProQuest LLC. 789 East Eisenhower Parkway P.O. Box 1346 Ann Arbor, Ml 4 8 1 0 6 - 1346 m Questa tesi e dedicata ad Alessandro Paluello (1975-1997). Preface This thesis is submitted in accordance with the regulations for the degree of Doctor of Philoso phy in the University of Glasgow. It is the record of research carried out at Glasgow University between October 1995 and September 1998. No part of it has previously been submitted by me for a degree at any university. I should like to express my gratitude to my supervisor, Prof. R.W.K.Odoni, for his ad vice and encouragement throughout the period of research. I would also like to thank Dr. S.D.Gohen for helping out with my supervision, and the E.P.S.R.G.