DENSITY ESTIMATION FOR FUNCTIONS OF CORRELATED RANDOM VARIABLES A Thesis Presented to The Faculty of the Fritz J. and Dolores H. Russ College of Engineering and Technology Ohio University In Partial Fulfillment of the Requirement for the Degree Master of Science by Jeffrey P. Kharoufeh June 1997 ACKNOWLEDGMENTS The author would like to express his gratitude and thanks to Dr. Helmut Zwahlen for his assistance and guidance in this work, to Dr. Thomas Lacksonen for his helpful suggestions and encouragement, to Dr. David Keck for his mathematical input, and to Dr. Hollis Chen who served as College representative on the Committee. The author would like to give special thanks to Thomas Schnell and Dr. Richard Gerth for their time, input, and helpful suggestions. Chad Johnson, Chandrasekar Subramanian, and Julie Kocsis also gave helpful suggestions for coding. I especially thank my parents, Towfiq and Janette, for their continued love, support, and patience. CONTENTS . LIST OF TABLES ..................................................................................vii ... LIST OF FIGURES ................................................................................ VIII CHAPTER 1. INTRODUCTION ....................................................................................... 1 1.1. Background ..............................................................................................................1 1.2. Statement of The Problem .......................................................................................4 1.3. Review of the Literature ..........................................................................................7 1.3.1. Functions of Independent Random Variables ................................................... 7 1.3.2. Functions of Dependent Random Variables ............................................... 1 1 1.4. Approach ................................................................................................................15 CHAPTER 2 . ESTIMATING THE JOINT PROBABILITY DENSITY FUNCTION ...17 2.1. Approaches to Density Estimation ......................................................................... 17 2.1.1. The Histogram Estimator ................................................................................18 2.1.2. The Kernel Density Estimator for Univariate Data ......................................... 20 2.2. The Multivariate Kernel Estimator ........................................................................26 2.2.1. Introduction to Multivariate Kernel Estimates ................................................26 2.2.2. Selection of the Appropriate Smoothing Parameter ........................................29 2.2.3. Selection of the Appropriate Kernel Function ................................................30 CHAPTER 3 . FROM THE JOINT DENSITY TO FUNCTIONS OF R.V.'s .................34 3.1. The Transformation of Variables for Discrete Random Variables ........................ 34 3.1. 1. A Small Example .............................................................................................26 3.2. The Transformation of Variables for Continuous R.V.'s .......................................3 8 3.2.1. Derivations for Continuous R.V.'s Using the Transformation of Variables ...40 3.2.2. A Small Example ............................................................................................ 44 CHAPTER 4 . IMPLEMENTATION OF THE APPROACH ...........................................48 4.1. Discretization of the Problem ................................................................................48 4.2. Some Notation and Definitions ..............................................................................50 4.3. Construction of the Joint Density Estimate ........................................................... 51 4.3.1. Selection of the Grid, G ..................................................................................51 4.4. Implementation of the Transformation of Variables ..............................................53 CHAPTER 5 . EMPIRICAL RESULTS AND VALIDATION ........................................ 54 5.1 . Experimental Method ................ .. ........................................................................54 5.1 . 1. Experimental Design ....................................................................................... 55 5.1.2. Experimental Procedure .................................................................................. 57 5.2. Evaluation of Empirical Results ........................................................................... 64 5.2.1 . Tests of Goodness-of-Fit .................................................................................. 64 5.2.2. The Analysis of Variance .................................................................................67 5.2.3. The Estimator Versus Enumeration ................................................................82 CHAPTER 6 . SOFTWARE IMPLEMENTATION .........................................................84 6.1 . Requirements of the Software Implementation .....................................................85 6.2. Using the Software Package ..................................................................................86 6.3. Limitations of the Program ....................................................................................92 CHAPTER 7 . CONCLUSIONS, RECOMMENDATIONS. AND FUTURE WORK ...93 7.1 . Conclusions............................................................................................................ 94 7.2. Recommendations.................................................................................................. 95 7.3. Irnprovements and Future Research ....................................................................... 97 7.3.1. Improvements to Current Research .................................................................97 7.3.2. Future Research ...............................................................................................98 REFERENCES ...............................................................................................................-102 APPENDICES .................................................................................................................106 APPENDIX A .1 Experimental Results .......................................................................107 APPENDIX A.2 Comparison of Cumulative Distribution Functions ........................ 120 APPENDIX A.3 Means Tables and Scheffe Post-Hoc Tests .....................................139 APPENDIX A.4 Source Code ListingISample Input File ...........................................150 LIST OF TABLES 1 . A(K) For Normal and Epanechnikov Kernels ............................................... 30 2 . Factors and Their Respective Levels For the Designed Experiment ..................... 56 3 . ANOVA Table, F-Values Given at the 0.05 Level ......................................... 70 LIST OF FIGURES 1. Univariate Histogram for Old Faithhl Geyser Data with 107 Observations ........... -19 2 . Kernel Estimate with Varying Window Width .............................................. 23 3 . Gaussian Kernel Estimate of Old Faithfbl Geyser Data ....................................24 4 . Comparison of Histogram and Kernel C.D.F.s for Old Faithful Geyser Data ..........24 5. Comparison of C.D.F.s for Two Kernels and Histogram Estimates ..................... 32 6 . Probability Distribution of Z = X + Y. Discrete Transformation of Variables Example ........................................................................................ 38 7 . Probability Distribution of Z = X +Y. Continuous Transformation of Variables Example .............................................................................46 8 . Sample Scatterplot of Correlated Data (X-U(O.l). n = 100)............................. 49 9 . Discretization of the Region Containing R for Construction of G ....................... 52 10. Grid Constructed on Correlated Data From Figure 8 ......................................57 1 1. Sample Input Data File For Evaluation of Estimator ......................................62 12. Sample Ouput Data File output.out From Evaluation Program ..........................63 13. Comparison Between Estimator C.D.F. and Simulated C.D.F. for Product When Xis ~istributed or mall^ and R~ = 0.90 ............................................67 14. Pareto Chart of Mean Squares for Each Effect in ANOVA Model .....................68 15. The Effect of Correlation on Maximum Absolute Error ..................................71 16. The Effect of Distribution Type on Maximum Absolute Error ..........................72 17. Interaction Plot for Correlation and Distribution Type ...................................73 18. Interaction Plot for Correlation and Operation............................................ 74 19. Interaction Plot for Correlation and Number of Observations ...........................75 20 . The Effect of Number of Observations (n)on Maximum Asolute Error ...............76 2 1. Interaction Plot for Number of Observations (n)and Distribution Type ............... 77 22 . Interaction Plot for Number of Observations (n)and Operation ........................ 78 23 . Interaction Plot for Distribution Type and Operation..................................... 79 24 . The Effect of Operation of Maximum Absolute Error .................................... 80 25 . Comparison of MAE C.D.F.s for Estimator and Enumeration (n = 50, xis orm mall^ ~istributed.R~