Efficient simulation of network performance by importance sampling Poul E. Heegaard Submitted to the Norwegian University of Science and Technology in partial fulfilment of the requirements for the degree of Doktor Ingeniør, May 1998. ii iii To Kristine, Helene and Solveig iv Abstract Simulation is a flexible means for assessment of the quality of service offered by a tele- communication system. However, when very strict requirements are put on the quality of service, the simulation becomes inefficient because the performance depends on rare events to occur. A rare event is, for instance, a cell loss or a system breakdown. A simu- lation technique that speeds up the experiments must be added. Various techniques are known from the literature and they should be combined to achieve additional speedups. The most efficient speedup techniques for systems dependent on rare events, are impor- tance sampling and RESTART. The importance sampling technique is very sensitive to the change of the underlying sim- ulation process. This is denoted the biasing of the simulation parameters. In this thesis, explicit expressions of the variance of importance sampling estimates and the likelihood ratio are developed for an M/M/1/N queue. The study of how the variance expressions vary as the biasing changes, demonstrates that the importance sampling is very efficient in a narrow region, and that the variance is unbounded outside. It is also observed that, seemingly, the likelihood ratio and its variance may be used as an indication of the accu- racy of simulation results, in combination with the variance of the estimate itself. Neither importance sampling nor RESTART are easily applied to multidimensional mod- els, e.g. a model of a telecommunication network with a variety of different users. In this thesis, the focus is on how to do importance sampling simulations of telecommunication networks with balanced utilisation of the resources. A network system are described by a multidimensional model. The balanced resource utilisation implies that the system per- formance is not given by a single bottleneck. Hence, previous approaches for importance sampling biasing are no longer efficient. The reason is that they assume that the perform- ance of a single resource significantly constrains the system performance, and under this assumption, the parameters can be biased with respect to the bottleneck resource only. - v - vi A new adaptive biasing technique is defined for dynamically setting up the simulation parameters in the importance sampling experiment. This is the major contribution of this thesis, and it has been successfully applied to several networks. The basic idea is to change the simulation parameters to make the simulation process move toward the parts of the state space where the most important rare events occur. Because this importance depends on the current stateω , the change of parameters is adapted to the state changes in the sim- ˜ ulation process. The networks used for feasibility demonstration are offered traffic from (i) users with dif- ferent resource capacities and traffic parameters, (ii) users with and without alternative routing strategies, and (iii) users with different preemptive priority levels and a network with a link failure. The simulation results are validated by comparison with exact results, rough dimensioning rules, and correctness indicators given by the observed likelihood ratio. Preface May 5th 1898, Poul Heegaard1 defended his doctoral thesis. This was a product of his research during the period ’94-’98. The history is now repeated 100 years later, except for the exact date and, of course, the contents of the thesis. My doctoral study has focused on rare event simulation, and in particular on the applica- tion of importance sampling. In my diploma thesis, nearly 10 years ago, I first investigated this technique. Then everything looked simple. However, after several projects during my time as research scientist at SINTEF, numerous problems with the application of impor- tance sampling were experienced. This motivated me to start an in-depth study on this topic, and when Telenor R&D generously made a 3 year scholarship available, I seized the opportunity. I am indebted to number of peoples for their assistance and encouragements. First of all I want to thank Prof. Bjarne E. Helvik at NTNU for his invaluable contributions to my research. He first introduced me to the exciting topic of rare event simulation when I was writing my masters thesis. Ever since, he has constantly assisted and encouraged me, first as a colleague at SINTEF, and for the last 4 years acting as my supervisor. I thank him for patiently listening to my questions, discussing ideas, raising critical questions, and com- menting on my research papers and reports. During the winter season ’95-’96, I was fortunate to work with Dr. Ragnar Andreassen, at that time a fellow doctoral student. In a joint effort with my supervisor, we succeeded in implementing importance sampling in trace driven simulations of MPEG coded video sources. The result of this research is found in appendix A. 1. Poul Heegaard (1871-1948). Thesis: Forstudier til en Teori for de algebraiske Fladers Sammenheng (in danish). Professor in mathematics at University of Copenhagen 1910-1917, and at Oslo University 1918- 1941. He was my great grandfather. - vii - viii The doctoral study has been conducted at NTNU-Department of Telematics. I would like to thank all my fellow students and faculty staff at this department, and the research sci- entists at SINTEF - Department of Systems Engineering and Telematics, for stimulating discussions. In particular, I would like to thank Arne Folkestad. We have shared office most of the time during the study, and have had an inspiring atmosphere which has helped me through the ups and downs for the past four years. A special thanks to the editors and the anonymous reviewers for their comments and con- structive criticisms of the papers I submitted to AEÜ, Special issue on Rare Event Simulation. Parts of these papers are included my thesis. Last, but not least, I will thank all my family and friends for giving me a social life outside the research community. In particular, my wife and two daughters, to whom I have dedi- cated this thesis, for their love, support and patience. They have constantly reminded me that there are much more important things in life than completion of a doctoral thesis! Poul E. Heegaard Trondheim, 30th April 1998. List of publications The research results from this doctoral study have been partly published in journals and presented at conferences. The following is the complete list of publications in the period 1994-98. Journals Poul E. Heegaard. A Scheme for Adaptive Biasing in Importance sampling. To appear in AEÜ International Journal of Electronics and Communication. Special issue on Rare Event Simulation. Poul E. Heegaard: Speed-up techniques for high-performance evaluation. Telektronikk, vol. 91(2/3), pp 195-207, 1995. Special issue on teletraffic. In conference proceedings Poul E. Heegaard. A survey of Speedup simulation techniques. Workshop on Rare Event Simulation. Aachen, Germany, 28-29 Aug., 1997. Session 1: Opening session & System Reliability. Poul E. Heegaard. A Scheme for Adaptive Biasing in Importance sampling. Workshop on Rare Event Simulation. Aachen, Germany, 28-29 Aug., 1997. Session 4: Tandem Queues and Networks. Poul E. Heegaard. Efficient simulation of network performance by importance sampling. 15th International Teletraffic congress - ITC 15. Washington D.C., USA, 23-27 June, 1997. Session: Simulation, Measurement and Data Analysis Poul E. Heegaard: Adaptive Optimisation of Importance Sampling for Multi-Dimensional State Space Models with Irregular Resource Boundaries. In P. Emstad,B. Helvik, and - ix - x A. Myskja, editors, The 13'th Nordic Teletraffic Seminar (NTS-13), pages 176-189, Trondheim, Norway, 20-22 August 1996. Ragnar Ø. Andreassen, Poul E. Heegaard, Bjarne E. Helvik: Importance Sampling for Speed-up Simulation of Heterogeneous MPEG Sources. In P. Emstad,B. Helvik, and A. Myskja, editors, The 13'th Nordic Teletraffic Seminar (NTS-13), pages 190-203, Trondheim, Norway, 20-22 August 1996. Poul E. Heegaard: Rare event provoking simulation techniques. In proceeding of the Inter- national Teletraffic Seminar (ITS), 28 Nov.-1 Dec. 1995 in Bangkok, Thailand. Session III: Performance Analysis I. Poul E. Heegaard: Comparison of speed-up techniques for simulation. In proceeding of the 12. Nordiske Teletrafikk Seminar (NTS-12), 22-24 August 1995 in Helsinki, Finland. Session 8: Intelligent Network Performance. Bjarne E. Helvik and Poul E. Heegaard. A Technique for Measuring Rare Cell Losses in ATM Systems. STF40 A94083. Reprint from 14th International Teletraffic congress - ITC 14. Antibes Juan-les-Pins, France 6-10 June, 1994. Session D31: B-ISDN Network Man- agement and Design. pp 917-930. Poster and PhD colloquium Poul E. Heegaard: Efficient simulation of network performance. Poster. 1997 Winter Sim- ulation Conference, Atlanta, Dec. 7-10, 1997. Poul E. Heegaard: Adaptive Biasing of Importance Sampling Parameters in Simulation of Telecommunication Network. PhD colloquium. 1997 Winter Simulation Conference, Atlanta, Dec. 7-10,1997. xi Table of contents Abstract ...........................................................................................................................v Preface .......................................................................................................................... vii