Methods for Generating Variates From

METHODS FOR GENERATING VARIATES FROM PROBABILITY DISTRIBUTIONS by JS DAGPUNAR, B. A., M. Sc. A thesis submitted in fulfilment-of the requirements for the award of the degree of Doctor of Philosophy. Department of Mathematics & Statistics, Brunel University, May 1983 (i) JS Dagpunar (1983) Methods for generating variates from probability distributions. Ph. D. Thesig, Department 'of Matbematics, Brunel University ABSTRACT Diverse probabilistic results are used in the design of random univariate generators. General methods based on these are classified and relevant theoretical properiies derived. This is followed by a comparative review of specific algorithms currently available for continuous and discrete univariate distributions. A need for a Zeta generator is established, and two new methods, based on inversion and rejection with a truncated Pareto envelope respectively are developed and compared. The paucity of algorithms for multivariate generation motivates a classification of general methods, and in particular, a new method involving envelope rejection with a novel target distribution is proposed. A new method for generating first passage times in a Wiener Process is constructed. This is based on the ratio of two random numbers, and its performance is compared to an existing method for generating inverse Gaussian variates. New "hybrid" algorithms for Poisson and Negative Binomial distributions are constructed, using an Alias implementation, together with a Geometric tail procedure. These are shown to be robust, exact and fast for a wide range of parameter values. Significant modifications are made to Atkinson's Poisson generator (PA), and the resulting algorithm shown to be complementary to the hybrid method. A new method for Von Mises generation via a comparison of random numbers follows, and its performance compared to that of Best and Fisher's Wrapped Cauchy rejection method. Finally new methods are proposed for sampling from distribution tails, using optimally designed Exponential envelopes. Timings are given for Gamma and Normal tails, and in the latter case the performance is shown to be significantly better than Marsaglia's tail generation procedure. (ii) ACKNOWLEDGMENTS It is a pleasure to record my gratitude to Professor P Macdonald for first suggesting I write this thesis, for subsequently supervising it, and for the helpful advice and guidance he has given me. I should like to acknowledge the financial assistance towards the cost of registration fees provided by the Governors of Dundee College of Technology. I should also like to record my appreciation to Mrs C Peters for her careful typing of the thesis. My greatest personal debt is to my wife Bridget, who has been a constant source of encouragement and support. (iii) 'CONUENTS Page PREFACE I CHAPTER I GENERAL METHODS FOR GENERATING RANDQMVARIATES 5 1.1 Inversion Method 5 1.2 Stochastic Model Methods 9 1.3 Rejection Methods 10 1.3.1 Envelope Rejection 10 1.3.2 Band Rejection 13 1.3.3 Ratio of Uniforms Method 16 1.3.4 Comparison of Random Numbers 1-18 1.4 Alias Rejection Method 22 1.5 Polynomial Sampling 26 1.6 Sampling from Discrete Empirical Distributionsý 29 1.6.1 Sequential Search 29 1.6.2 Ordered Sequential Search 30 1.6.3 Marsaglia's Method 31 1.6.4 Method of Norman and Cannon 32 1.6.5 Indexed Search 35 CHAPTER 2 METHODS OF GENERATION FROM SPECIFIC CONTINUOUS 37 DISTRIBUTIONS 2.1 Negative Exponential 37 2.2 Normal Distribution 40 2.3 Gamma Distribution 52 2.4 Beta Distribution 66 2.5 Some other Continuous Distributions 76 CHAPTER 3 METHODS OF GENERATION FROM SPECIFIC DISCRETE 80 DISTRIBUTIONS 3.1 Binomial Distribution 80 3.2 Poisson Distribution 84 3.3 Some other Discrete Distributions 88 3.4 Zeta Distribution 91 Uv) Page CHAPTER 4 GENERATION FROM MULTIVARIATE CONTINUOUS 98 DISTRIBUTIONS 4.1 General Methods 98 4.1.1 Method of Conditional Distributions 98 4.1.2 Transformation to Independent Form 99 4.1.3 Rejection Method 99 4.2 Multivariate Normal Distribution 101 4.3 A Bivariate Exponential Distribution 104 4.4 A Non-standard Bivariate Distribution 106 CHAPTER 5 SIXULATING'FIRST PASSAGE TIMES IN A WIENER'PROCESS 110 5.1 Introduction 110 5.2 A theorem facilitating the determination of an enclosing region in the ratio of uniforms method 113 5.3 An algorithm based on the ratio of uniforms method 118 5.4 Computational Experience 121 CHAPTER 6 SOME GENERATORSFOR T14E POISSON AND NEGATIVE BINOMIAL DISTRIBUTIONS 125 6.1 A hybrid Alias/Envelope Rejection generator for the Poisson distribution 125 6.1.1 Introduction 125 126 6.1.2 A hybrid algorithm 6.1.3 Programming and Numerical Aspects 129 6.1.4 Verification and Validation 131 6.2 Modification to aii envelope rejection procedure for the Poisson distribution 134 6.2 1 Introduction 134 134 6-2: 2 Development of algorithm 6.2.3 Determination of, Sampling Efficiency, M-1 138 6.2.4 Verification, Validation and Analysis of algorithm 140 6.3 Timing Comparisons for the Poisson Generators 142 6.4 A hybrid Alias/Envelope Rejection generator for the Negative Binomial Distribution 146 6.4.1 Development of algorithm 146 6.4.2 Programming and Numerical Aspects 148 6.4.3 Verification and Validation 150 6.4.4 Timing Results for routine NBINOM and conclusions 152 I Page CHAPTER 7 GENERATION FROM THE VON MISES DISTRIBUTION VIA A COMPARISON OF RANDOMNUMBERS 154 7.1 Introduction 154 7.2 A Random Number Comparison Method. 155 7.3 A Probability mixture method using envelope rejection with a piecewise uniform target distribution 159 7.4 Best and Fisher's Wrapped Cauchy Method 162 7.5 Verification and Validation of Algorithms 163 7.6 Timing Experiments and Conclusions 167 CHAPTER 8 SAMPLING VARIATES FROM THE TAIL OF GAHM AND NORMAL DISTRIBUTIONS 1 171 8.1 Introduction 171 8.2 Method 172 8.3 Special Cases 174 8.4 Relative Efficiency 175 8.5 Computational Experience 177 8.6 Tail Generation from the Normal Distribution 178 APPENDIX I PROGRAMLISTINGS OF ZIPINF. FOR, ZIPFRP. FOR 183 APPENDIX 2 TESTS OF UNIFORMITY AND SERIAL CORRELATION ON RANDOM NUMBER GENERATOR RAN(r) 186 APPENDIX 3 PROGRAMLISTINGS OF MUL3. FOR, NULI. FOR 191 APPENDIX 4 PROGRAMLISTINGS OF WALDCH.FORO INGAUS. FOR, INLOG. FOR, INLOGI. FOR 193 APPENDIX 5 PROGRAMLISTINGS OF ALIAS. FOR, MVALUE. FOR, POLOG.FOR, NBINOM. FOR 197 APPENDIX 6 PROGRAMLISTINGS OF VMISES. FOR, UNIVON. FOR, BFISH. FOR, BFISHC. FOR 207 APPENDIX 7 PROGRAMLISTINGS OF GATRUN.FOR, TRUNCN. FOR, TRUNCM.FOR, TRUNCS. FOR 215 REFERENCES 219 PREFACE In the area of Statistical Computing* the generation of observations (random variates) from probability distributions, has, within the last ten years, been the subject of active research. Until the early 701s, apart perhaps from the Normal and Negative Exponential distributions', many experimenters were still'using inversion of the cumulative distribution function, frequently necessitating time consuming numerical work or approximations. More recently; there has been interest in developing fast algorithms, which generate 'ýexact" variates without undue numerical work.. It is perplexing to understand why the period 1950-1970 was such an inactive one. The probabilistic ideas underlying the algorithms now available were all well known, simulation'was a recognised tool in stientific and management science investigations, and computers were beginning to be used. Indeed, it might'have been expected, that the scarce availability of computing power at the beginning'of this period would have motivated the early development of efficient algorithms. The present situation is quite different. There is now a selection of methods available for most standard distributions. New methods continue to be proposed, with the result that'a relatively recent review appearing in Fishman's, "Principles of Discrete Event Simulation" (1978), is now somewhat incomplete and dated. For example, no mention is made in that book of Band rejection, indexed search, or the ratio of uniforms methods. 2 One aim of the thesis therefore, is to fill this gap in the literature, by an extensive review of the current situation., Chapter I classifies general approaches to variate generation, the classifica ased on structural properties rather than historical order. This provides a framework for Chapters 2 and 3 which compare algorithms available for specific univariate distributions. In these first three chapters, emphasis is placed upon the theoretical properties which are likely to affect the speed of the generator. One reason for this is that it allows the designer of algorithms to exclude unpromising approaches at an early stage in the development. A second reason is that experimental measures of performance such as speed or storage requirements vary according to the computer in'use. Thus it is useful to have measures, albeit imperfect ones, which are machine independent. Theoretical analyses provide such measures. A third reason is that theoretical measures provide a basis for explaining and understanding experimental results, and for providing qualitative descriptions of the likely behaviour on other machines. Finally, in reviewing material, it has been found that the theoretical properties in the original sources are sometimes incomplete, and by deriving them here, a firm foundation-is placed for the new algorithms to be developed within the thesis. The remainder of the thesis describes new algoritbms or methods which have been developed. The first of these concerns the Zeta distribution, for which no generation methods could be found in the literature. Two complementary algorithms are developed within Chapter 3, allowing such variates to be generated for all values of the parameter of the distribution. 3 The situation described above is rather different when Multivariate distributions are considered. Although a few specific algorithms have appeared in the literature, the designer of algorithms has no classification of general methods to rely upon, as exists in the univariate case. This deficiency motivated the identification of three general methods in Chapter 4.

Load more