Thesis submitted in fulfilment of the requirements for the award of the degree of Doctor of Engineering Sciences (Doctor in de ingenieurswetenschappen) AGGREGATION AND RECOVERY METHODS FOR HETEROGENEOUS DATA IN IOT APPLICATIONS. EVANGELOS K. ZIMOS Supervisors: Prof. Deligiannis Nikos Prof. Munteanu Adrian FACULTY OF ENGINEERING Department of Electronics and Informatics Examining Committee Advisor: Prof. Dr. Ir. Nikos Deligiannis, Vrije Universiteit Brussel. Advisor: Prof. Dr. Ir. Adrian Munteanu, Vrije Universiteit Brussel. Chair: Prof. Dr. Ir. Ann Now, Vrije Universiteit Brussel. Vice Chair: Prof. Dr. Ir. Rik Pintelon, Vrije Universiteit Brussel. Com. Member: Prof. Dr. Ir. Martin Timmerman, Vrije Universiteit Brussel. Secretary: Prof. Dr. Ir. Jan Lemeire, Vrije Universiteit Brussel. Ext. Member: Prof. Dr. Miguel Rodrigues, University College London. Ext. Member: Prof. Dr. Ir. Eli De Poorter, Universiteit Gent. “. Πάντες ἄνθρvωποι το˜υ εἱδέναι ὀρέγονται φύσει. Σημείον δ᾿ ἡ των˜ αἰσθή-σεvων ἀγάπησις: καί γὰρ χωρὶς της˜ χρείας ἀγαπωνται˜ δι᾿ αὑτάς, καὶ μάλιστα των˜ ἄλλων ἡ διὰ των˜ ὀμμάτων. Οὐ γὰρ μόνον ἵνα πράττωμεν ἀλλά καί μηθέν μέλλοντες πράττειν τὸ ὁρ˜αν αἱρούμεθα ἀντὶ πάντων ὡς εἰπε˜ιν των˜ ἄλλων. Αἴτιον δ᾿ ὅτι μάλιστα ποιε˜ι γνωρίζειν ἡμ˜ας αὕτη των˜ αἰσθήσεων καὶ πολλὰς δηλο˜ι διαφοράς ...” – ᾿Αριστοτέλης, Μετά τά Φυσικά Α΄ “. All men naturally desire knowledge. An indication of this is our es- teem for the senses; for apart from their use we esteem them for their own sake, and most of all the sense of sight. Not only with a view to action, but even when no action is contemplated, we prefer sight, generally speaking, to all the other senses. The reason of this is that of all the senses sight best helps us to know things, and reveals many distinctions . ” – Aristotle, Metaphysics I Contents Acknowledgements i Synopsis vii Acronyms viii 1 Introduction 1 1.1 Wireless Sensor Networks . 3 1.1.1 SensorNode ........................... 4 1.1.2 Deployment Phases and Topologies . 7 1.2 Data Aggregation and Challenges . 8 1.3 Contributions............................... 11 1.4 Applications................................ 15 1.4.1 Environmental monitoring . 17 1.5 Outline .................................. 19 2 Copula Functions 23 2.1 Motivation ................................ 24 2.2 Copula Definition . 25 2.2.1 Mathematical Background . 29 2.2.2 Sklar’s Theorem . 33 2.2.3 Fr`echet-Hoeffding Bounds . 34 2.2.4 Copulas and Random Variables . 36 2.2.5 Derivatives and Copula Density . 38 i Contents 2.3 Dependence Concepts . 40 2.3.1 Linear Correlation . 40 2.3.2 Perfect Dependence . 41 2.3.3 Concordance . 41 2.3.4 Kendall’s tau and Spearman’s rho . 44 2.3.5 Tail Dependence . 46 2.4 Copula Families. 49 2.4.1 Elliptical Copulas . 50 2.4.2 Archimedean Copulas . 56 2.5 CopulaFitting .............................. 61 2.5.1 Parametric Copula Estimation . 63 2.5.2 Non-Parametric Copula Estimation . 67 2.5.3 Choosing the Right Copula . 69 2.5.4 Dealing with Discrete Marginal Distributions . 70 3 Data Gathering via Multiterminal Coding and Copula Regression 73 3.1 Introduction................................ 73 3.1.1 Contributions . 75 3.2 Background on MT Source Coding . 77 3.2.1 Gaussian Regression as a Refining Stage . 79 3.3 The Proposed MT Source Code Design . 79 3.4 The Proposed Semi-Parametric Copula Regression . 82 3.4.1 Statistical Modelling for Diverse Sensor Data . 83 3.4.2 The Proposed Copula Regression Method . 85 3.5 Experiments................................ 89 3.5.1 Choice of the Appropriate Kernel Function . 93 3.5.2 Performance Evaluation of DPCM . 97 3.5.3 Performance Evaluation of the Copula Regression Algorithm 99 3.5.4 Overall Performance of the Proposed System . 100 3.5.5 Performance Evaluation for Weaker Intra-Sensor Dependence Structure ............................. 102 3.5.6 Comparison with DSC for Different WSN Topologies . 105 ii 3.6 Conclusion ................................ 107 3.7 Discussions ................................ 108 4 Data Gathering via Compressed Sensing with Side Information 113 4.1 Problem Overview . 115 4.1.1 Application Scenario . 115 4.1.2 PriorWork ............................ 116 4.1.3 Contributions . 117 4.2 Background ................................ 118 4.2.1 Compressed Sensing . 119 4.2.2 Compressed Sensing with Side Information . 121 4.2.3 Distributed Compressed Sensing . 124 4.2.4 Data Gathering with CS . 125 4.3 Proposed Architecture based on Compressed Sensing with Side In- formation ................................. 126 4.3.1 Data Gathering under Noise . 126 4.3.2 Successive Data Recovery . 129 4.3.3 Extension to Tree-Based Topologies . 131 4.4 Statistical Modelling For Diverse Data . 132 4.5 Copula-based Belief Propagation Recovery . 137 4.5.1 Background on Bayesian Inference . 138 4.5.2 Description of the proposed recovery algorithm . 139 4.6 Recovery Algorithm for the Extended ` ` Problem . 142 1 − 1 4.7 Experiments................................ 143 4.7.1 Sparsifying Basis Selection . 144 4.7.2 Performance Evaluation of the Proposed Copula-based Algo- rithm ............................... 145 4.7.3 Evaluation of the System Performance . 147 4.7.4 Evaluation of System Performance under Noise . 149 4.8 Conclusions ................................ 152 5 Large-Scale Data Gathering via Compressive Demixing 157 5.1 PriorArt ................................. 158 iii Acknowledgements 5.2 Contributions............................... 159 5.3 Background ................................ 160 5.3.1 Source Separation . 161 5.3.2 Compressive Demixing . 163 5.3.3 Oracle Problem . 164 5.4 ProposedScheme............................. 166 5.4.1 Joint Data Aggregation. 166 5.4.2 Joint Data Recovery via Compressive Demixing . 168 5.5 Experimental Evaluation . 169 5.5.1 Comparison with State of the Art . 170 5.5.2 Comparison with Successive Reconstruction Architecture . 172 5.6 Conclusions ................................ 173 6 Conclusions & Future Study 175 6.1 FutureWork ............................... 178 6.2 General Directions . 182 iv Acknowledgements Becoming a Doctor of Philosophy requires not only your personal effort and dis- cipline but also the help and support from persons that were (or were meant to be) important in your life. During this Ph.D. journey, I was blessed to meet and cooperate with people that significantly contributed to my scientific evolution and changed my perspective on life. Firstly, I would like to express my gratitude to my advisors, Prof. Nikos Deli- giannis and Prof. Adrian Munteanu, for guiding me through the last four years and giving me the opportunity to accomplish my Ph.D. study. Moreover, I would like to thank the rest of my thesis committee: Prof. Ann Now, Prof. Rik Pintelon, Prof. Martin Timmerman, Prof. Jan Lemeire, Prof. Miguel Rodrigues and Prof. Eli De Poorter for dedicating part of their time to evaluate the work presented in this thesis, as well as their insightful comments and encouragement. I am sincerely grateful to Dr. Dimitris Toumpakaris for our continuous infor- mation theoretic talks, which started during my diploma thesis in the University of Patras and continued during my Ph.D. study. Dimitris was a mentor to me through- out all these years, a person with positive spirit, a great source of inspiration and motivation. My sincere thanks also goes to Prof. Miguel Rodrigues and Dr. Jo˜ao Mota for their significant support and guidance, especially during the last two years of my Ph.D. study. Both were providing strict comments and fruitful scientific ideas that were targeting on high-quality studies on the domain. This Ph.D. journey would not be successfully accomplished without working in a positive and productive environment. My scientific colleagues Athanassia, Ruxan- v Acknowledgements dra, Gabor, Bruno, Beerend, Petar, Jan, and Adriaan were also my dear friends that supported me at difficult times and enriched my life with pleasant memories. More importantly, this experience resulted in broadening my family, as my colleague and friend Andrei Sechelea became my best man. My heartfelt thanks goes to my parents, Kostas and Maria, as well as my beloved sister, Dimitra, for their inherent love, support and guidance. They were always available for listening to my personal problems and wisely advising me. Last but not the least, I would like to thank my wife Sofia, my personal “car- diologist”. She is a beautiful person in every aspect, characterized by endless love, trustworthiness, patience and compassion. She knows that she played the most im- portant role in the whole experience. vi Synopsis Powered by technological advances on sensing hardware, wireless communications, cloud computing and data analysis, wireless sensor networks have become a key tech- nology for the Internet of Things. Wireless sensor systems of various types promise to bring new perspectives in our lives by enabling various applications, such as smart cities, ambient-condition monitoring, air-pollution monitoring, smart retail, wearables, and many more. We propose novel data aggregation and recovery mechanisms that solve the fol- lowing challenges: (i) achieving efficient data sensing and in-network compression so as to increase the power autonomy of the wireless sensor network, and (ii) providing robust data transmission against communication noise. Unlike previous data gath- ering schemes, which exploit underlying correlations among homogeneous sensor data, the proposed designs embody new recovery methods that leverage statistical dependencies among diverse sensor data.