Biologically-Inspired Machine Intelligence Technique for Activity Classification in Smart Home Environments Talal Sarheed Alshammari A thesis submitted in partial fulfillment of the requirement of Staffordshire University for the degree of Doctor of Philosophy May 2019 List of Publications 2 REFEREED JOURNAL PUBLICATIONS OpenSHS: Open Smart Home Simulator May, 2017 Sensors Journal MDPI 2 REFEREED JOURNAL PUBLICATIONS SIMADL: Simulated Activities of Daily Living Dataset. April, 2018 Data Journal MDPI 2 REFEREED CONFERENCE PUBLICATIONS Evaluating Machine Learning Techniques for Activity Classifica- February ,2018 tion in Smart Home Environments World Academy of Science, Engineering and Technology International Journal of Infor- mation and Communication Engineering i Acknowledgements I would like to thank my supervisors Dr. Mohamed Sedky and Chris Howard for encour- aging and supporting me during my Ph.D study. They were very helpful and provided me their assistance throughout my Ph.D journey. It was very good experience to improve my research skills. Due to this experience, I learned new knowledge, found new ideas from your supervision. I appreciate your consideration, efforts and kind cooperation, you deserve all the thanks. I would like to thank to my parents and my wife for their unconditional support that has helped me for achieving this thesis. I would like to thank the Ministry of Education in Saudi Arabia and Hail university for funding and supporting this research. ii Contents List of Publicationsi Acknowledgements ii Table of Contents iii Abstract vii List of Figures ix List of Tables xii Abbreviations xiii 1 Introduction1 1.1 Problem Definition...............................1 1.2 Aim.......................................2 1.3 Objectives....................................2 1.4 Research Questions...............................3 1.5 Scope of the Research.............................3 1.6 Challenges....................................4 1.7 Main Contributions...............................6 1.8 Research Methodology.............................6 1.8.1 Quantitative Research.........................6 1.8.2 Qualitative Research..........................6 1.9 Organisation of the Thesis...........................7 2 Literature Review9 2.1 Introduction...................................9 2.2 Internet of Things (IoT)............................9 2.2.1 Major Issues............................... 10 2.2.2 Enabling Technologies......................... 11 2.2.3 Middleware............................... 11 2.2.4 Context-Awareness........................... 12 2.3 Smart Home................................... 12 2.3.1 Definition................................ 12 2.3.2 Classification.............................. 14 iii 2.3.3 Structure................................ 15 2.3.4 Architecture............................... 18 2.3.5 Projects and Applications....................... 18 2.3.6 ADLs Monitoring............................ 22 2.4 Machine Intelligence Algorithms for Smart Homes.............. 25 2.4.1 Support Vector Machines (SVMs):................... 28 2.4.2 Hidden Markov Model (HMM):.................... 29 2.4.3 Decision Trees (DT):.......................... 29 2.4.4 Stochastic Gradient Descent (SGD):................. 30 2.4.5 AdaBoost:................................ 30 2.4.6 Hierarchal Temporal Memory (HTM):................ 31 2.4.7 Multi-layer Perceptron (MLP):..................... 32 2.4.8 Long Short Term Memory (LSTM):.................. 32 2.4.9 Convolutional Neural Network (CNN):................ 33 2.5 Intelligence................................... 33 2.5.1 What Is the Type of Intelligence That a Smart Home Needs (Re- quires)?................................. 34 2.5.2 Machine Learning Algorithms Requirements............. 34 2.5.3 Research the Gap............................ 34 2.6 Why is CLA used for smart home....................... 35 2.6.1 Online Learning............................ 36 2.6.2 Noise Tolerance............................. 36 2.6.3 Robustness and Generalisation.................... 36 2.6.4 Conclusion............................... 37 3 Hierarchical Temporal Memory 38 3.1 Introduction................................... 38 3.2 Biological Neural Network........................... 39 3.3 History of Neural Networks.......................... 40 3.4 HTM: Overview................................. 41 3.4.1 HTM Concept.............................. 43 3.4.2 Hierarchy................................ 44 3.4.3 Sparse Distributed Representations (SDRs).............. 44 3.4.4 Time................................... 46 3.5 Cortical Learning Algorithm (CLA)...................... 46 3.5.1 Understanding How the Cortical Learning Algorithm Works.... 46 3.5.2 Cell States................................ 47 3.5.3 How the Cortical Learning Algorithm Learns............. 47 3.6 Components of CLA.............................. 47 3.6.1 Encoder................................. 47 3.6.1.1 Characteristics of Encoders................. 48 3.6.2 NuPIC Encoders............................ 48 3.6.2.1 Encoding Numbers (Scalar Encoder)............ 48 3.6.2.2 Category Encoder...................... 51 3.6.2.3 SDR Category Encoder................... 52 3.6.2.4 The Date Encoder...................... 52 3.6.3 Spatial Pooler.............................. 52 3.6.4 Temporal Memory........................... 54 3.6.5 Classifier................................ 55 3.6.5.1 K-nearest Neighbours Classifier............... 55 3.6.5.2 CLA Classifier........................ 56 3.6.5.3 SDR Classifier........................ 57 3.7 Planning for the CLA.............................. 59 3.8 Particle Swarm Optimisation (PSO)...................... 59 3.9 Summary.................................... 59 4 SIMADL: Simulated Activities of Daily Living Dataset 61 4.1 Introduction................................... 61 4.2 Related Work.................................. 63 4.2.1 Real Datasets.............................. 63 4.2.2 Simulation Tools............................ 67 4.3 OpenSHS.................................... 69 4.3.1 OpenSHS Advantages......................... 70 4.4 Methodology.................................. 71 4.4.1 Smart Home Design.......................... 71 4.4.2 The Participants............................ 74 4.4.3 Dataset Aggregation.......................... 74 4.5 SIMADL Description.............................. 75 4.6 Classification ADLs............................... 80 4.6.1 Classification of ADLs in SIMADL Dataset.............. 81 4.7 Usability Study................................. 82 4.8 Conclusions................................... 84 5 Proposed Multi-Region CLA Techniques 86 5.1 Challenges.................................... 86 5.2 Hash SDR Encoder............................... 88 5.2.1 Introduction.............................. 88 5.2.2 Flexible Encoder............................ 88 5.2.3 Problem Definition for Hash Encoder................. 88 5.2.4 How Does the Hash Function Work?................. 89 5.2.5 Analysis................................. 90 5.3 Classifier..................................... 93 5.3.1 Introduction.............................. 93 5.3.2 Analysis................................. 93 5.3.3 Multi-layer Perceptron......................... 94 5.3.4 Backpropagation Algorithm...................... 95 5.3.5 How ANNs Works............................ 96 5.3.6 Activation Functions.......................... 96 5.3.7 Application of the MLP classifiers................... 97 5.4 Multi-Region CLA Techniques......................... 99 5.4.1 Introduction.............................. 99 5.4.2 Biological Short and Long-term Memory............... 99 5.4.3 Capacity of Different Forms of Sequence Memory.......... 100 5.4.4 How Many Times Can the Same Input Represent Different Contexts?100 5.4.5 Multi-Region HTM-Related work................... 101 5.4.6 Proposed Parallel Spatio-Temporal Memory Stream Technique (CLA2) ...................................... 102 5.4.6.1 Analysis............................ 105 5.4.7 Proposed Cascaded Temporal Memories Stream Technique (CLA3) 106 5.4.8 Summary................................ 109 6 Test and Evaluation 110 6.1 Introduction................................... 110 6.2 Evaluation Methodology............................ 110 6.2.1 Real Smart Home Dataset:....................... 111 6.2.2 Synthetic Smart Home SIMADL Dataset............... 111 6.2.3 Experiment Design........................... 111 6.2.4 Performance Metrics.......................... 112 6.2.5 Parameter Selection for the CLA Model................ 113 6.3 Results...................................... 114 6.3.1 Base-line CLAs Versus Proposed Multi-Region CLA Techniques.... 114 6.3.1.1 The selected model parameters by the swarm optimiser.. 117 6.3.2 State-of-the-Art Techniques Versus Proposed Multi-Region CLA Tech- niques.................................. 121 6.4 Discussion.................................... 126 6.5 Summary.................................... 128 7 Conclusions and Future Work 130 7.1 Summary.................................... 130 7.2 Original Contributions............................. 132 7.3 Conclusion................................... 133 7.4 Future Work................................... 134 A Datasets 136 B Tables and Figures of Results 144 B.1 Tables of Results................................ 144 B.2 Prediction accuracy with Ground Truth.................... 149 References 153 Abstract With the widespread adoption of Internet-connected devices and the prevalence of ap- plications in the Internet of things (IoT), devices