Statistical Analysis of Wireless Communication Systems Using Hidden Markov Models Ishtiaq Rouf Thesis submitted to the faculty of the Virginia Polytechnic Institute and State University in partial fulfillment of the requirements for the degree of Master of Science In Electrical and Computer Engineering Dr. William H. Tranter Dr. Jeffrey H. Reed Dr. Tim Pratt July 2, 2009 Blacksburg, Virginia Keywords: Hidden Markov Model, Position Location, Stochastic Games, Network Security Situation Awareness Copyright © Ishtiaq Rouf, 2009 Statistical Analysis of Wireless Communication Systems Using Hidden Markov Models Ishtiaq Rouf ABSTRACT This thesis analyzes the use of hidden Markov models (HMM) in wireless communication systems. HMMs are a probabilistic method which is useful for discrete channel modeling. The simulations done in the thesis verified a previously formulated methodology (1). Power delay profiles (PDP) of twelve wireless receivers were used for the experiment. To reduce the computational burden, binary HMMs were used. The PDP measurements were sampled to identify static receivers and grid- based analysis. This work is significant as it has been performed in a new environment. Stochastic game theory is analyzed to gain insight into the decision-making process of HMMs. Study of game theory is significant because it analyzes rational decisions in detail by attaching risk and reward to every possibility. Network security situation awareness has emerged as a novel application of HMMs in wireless networking. The dually stochastic nature of HMMs is applied in this process for behavioral analysis of network intrusion. The similarity of HMMs to artificial neural networks makes it useful for such applications. This application was performed using simulations similar to the original works. TABLE OF CONTENTS Table of Contents .................................................................................................................................................... iii List of Figures ............................................................................................................................................................ v List of Tables ........................................................................................................................................................... vii Acknowledgments ................................................................................................................................................. viii Chapter 1: Introduction ........................................................................................................................................... 1 1.1 Fundamentals of Markov Theory............................................................................................................... 2 1.2 Stochastic Games using Hidden Markov Models ................................................................................... 2 1.3 Position Location using Hidden Markov Models ................................................................................... 3 1.4 Network Security Situation Awareness using Hidden Markov Models .............................................. 3 Chapter 2: Fundamentals of Markov Theory ...................................................................................................... 5 2.1 Markov Processes .......................................................................................................................................... 5 2.2 Discrete-Time Markov Chains .................................................................................................................... 8 2.2.1 States and Transitions ........................................................................................................................... 9 2.2.2 Transition Matrix ................................................................................................................................. 10 2.2.3 Emission Matrix ................................................................................................................................... 12 2.3 Hidden Markov Models ............................................................................................................................. 14 2.3.1 Parameters of Hidden Markov Models ........................................................................................... 14 2.3.2 Output Sequence Generation in Hidden Markov Models .......................................................... 16 2.3.3 Structures of Hidden Markov Models ............................................................................................. 17 2.3.4 Gilbert and Fritchman Models ......................................................................................................... 19 2.3.5 Variations of Hidden Markov Models ............................................................................................. 19 2.3.6 Examples of Hidden Markov Models ............................................................................................. 22 2.4 Parameter Estimation of Markov Models .............................................................................................. 24 2.4.1 Expectation-Maximization (EM) Algorithm .................................................................................. 24 2.4.2 Forward-Backward Algorithm (FB) ................................................................................................. 25 2.4.3 Baum-Welch Algorithm (BWA) ....................................................................................................... 26 2.5 Distance Measure of Hidden Markov Models ....................................................................................... 27 Chapter 3: Stochastic Games using HMMs ....................................................................................................... 29 3.1 Game Theory ............................................................................................................................................... 29 3.2 Stochastic Games ........................................................................................................................................ 30 3.2.1 Summable Markov Decision Processes .......................................................................................... 30 3.2.2 Stationary Markov Transition Property ........................................................................................... 30 3.2.3 휷-Discounted Markov Decision Model, 횪휷 .................................................................................. 32 iii 3.2.4 Termination Markov Decision Model, 횪흉...................................................................................... 32 3.3 Behavior and Markov Strategies ............................................................................................................... 33 3.3.1 Travelling Inspector Model ............................................................................................................... 35 3.3.2 Duopoly Model .................................................................................................................................... 36 3.3.3 Strike Negotiation Model ................................................................................................................... 37 Chapter 4: Position Location using Hidden Markov Models ........................................................................ 39 4.1 Introduction .................................................................................................................................................. 39 4.2 Related Work ................................................................................................................................................ 39 4.2.1 Angle of Arrival Based Technique ................................................................................................... 39 4.2.2 Time of Arrival Based Technique .................................................................................................... 40 4.2.3 Time Difference of Arrival Based Technique ................................................................................ 41 4.2.4 Hybrid Techniques .............................................................................................................................. 42 4.3 HMM-Based Position Location ................................................................................................................ 43 4.3.1 Fading Multipath Channels................................................................................................................ 43 4.3.2 Ricean Power Delay Profiles ............................................................................................................. 44 4.3.3 Power Delay Profiles .......................................................................................................................... 45 4.3.4 Previously Used Method .................................................................................................................... 46 4.3.5 Floor Map of Previous Test Scenario .............................................................................................. 49 4.4 Simulation Methodology ............................................................................................................................ 50 4.4.1 HMM-Based Location Training ......................................................................................................