Exploiting Sensor Data Semantics for Smartphone-Based Lifelogging: Towards the Development of Digital Prosthetic Memory on Smartphones
Total Page:16
File Type:pdf, Size:1020Kb
EXPLOITING SENSOR DATA SEMANTICS FOR SMARTPHONE-BASED LIFELOGGING: TOWARDS THE DEVELOPMENT OF DIGITAL PROSTHETIC MEMORY ON SMARTPHONES By SHAUKAT ALI Supervised By PROF. DR. SHAH KHUSRO DEPARTMENT OF COMPUTER SCIENCE UNIVERSITY OF PESHAWAR, PESHAWAR, PAKISTAN (SESSION 2009-2010) ii EXPLOITING SENSOR DATA SEMANTICS FOR SMARTPHONE-BASED LIFELOGGING: TOWARDS THE DEVELOPMENT OF DIGITAL PROSTHETIC MEMORY ON SMARTPHONES By SHAUKAT ALI This Thesis is submitted to the Department of Computer Science University of Peshawar in Partial Fulfillment of the Dissertation Requirements for the Degree of Ph.D. in Computer Science DEPARTMENT OF COMPUTER SCIENCE UNIVERSITY OF PESHAWAR, PESHAWAR, PAKISTAN (SESSION 2009-2010) i Certificate of Approval This is to certify that the research work presented in this thesis, entitled "Exploiting Sensor Data Semantics for Smartphone-Based Lifelogging: Towards the Development of Digital Prosthetic Memory on Smartphones" was conducted by Mr. Shaukat Ali under the supervision of Prof. Dr. Shah Khusro. No part of this thesis has been submitted anywhere else for any other degree. This thesis is submitted to the Department of Computer Science, University of Peshawar in partial fulfillment of the requirements for the degree of Doctor of Philosophy in the field of Computer Science. Student Name: Mr. Shaukat Ali Signature: _____________________ Examination Committee a) External Examiner 1: Dr. Jamil Ahmad Signature: _____________________ Chairman/Assistant Professor Department of Computational Sciences Research Center for Modeling & Simulation NUST, Pakistan. b) External Examiner 2: Dr. Engr. Sami ur Rahman Signature: _____________________ Assistant Professor Department of Computer Science & IT University of Malakand, Pakistan. c) Internal Examiner: Prof. Dr. Azhar Rauf Signature: _____________________ Professor Department of Computer Science University of Peshawar, Pakistan. Supervisor Name: Prof. Dr. Shah Khusro Signature: _____________________ Name of Dean/HOD: Prof. Dr. Shah Khusro Signature: _____________________ ii Forward Certificate The research entitled “Exploiting Sensor Data Semantics for Smartphone-Based Lifelogging: Towards the Development of Digital Prosthetic Memory on Smartphones” is conducted under my supervision and the thesis is submitted to University of Peshawar in partial fulfillment for the degree of Ph.D. in Computer Science. ______________________________ Dr. Shah Khusro Professor Department of Computer Science University of Peshawar iii Author's Declaration I, Mr. Shaukat Ali, hereby state that my PhD thesis titled "Exploiting Sensor Data Semantics for Smartphone-Based Lifelogging: Towards the Development of Digital Prosthetic Memory on Smartphones" is my own work and has not been submitted to previously by me for taking any degree from this University of Peshawar or anywhere else in the country/world. At any time if my statement is found to be incorrect even after my graduation the university has right to withdraw my PhD degree. ______________________________ Mr. Shaukat Ali Date: 12/01/2018 iv Rights of Thesis Copyright © 2018 by Shaukat Ali All rights reserved. No part of this publication may be reproduced, distributed, or transmitted in any form or by any means, including photocopying, recording, or other electronic or mechanical methods, without the prior written permission of the researcher. ______________________________ Mr. Shaukat Ali [email protected] v Dedication Dedicated to My loving parents & family vi Acknowledgements First, I would like to present my sincere thanks to Almighty Allah for giving me the strength and courage to complete the tedious and hard course of Ph.D. and coming up with this thesis. I would like to dedicate this thesis to my loving parents and family members who have been a constant source of inspiration for me. I am especially thankful to my wife for her unselfish support and encouragement. Certainly, I could not have survived during my Ph.D. without their support, motivation, and endless love. I acknowledge the inspirational instructions and guidance of Professor Dr. Shah Khusro who is my Ph.D. supervisor. His valuable guidance always remained my strength, which not only gave me the deep knowledge and support of the subject but also straighten my research directions and research methodology. His intelligence and scientific vision empowered me of leaving no stone unturned during my Ph.D. His profound knowledge and innovative guidance always impressed me and saved me from disaster whenever I faced any. I am feeling no doubt in my mind to acknowledge that without his constant support and guidance I could not have accomplished my Ph. D. process. I am thankful to my lab mates Mr. Irfan Ullah, Mr. Akif Khan, and Mr. Nasir Khan for accompanying me in my every day research work. Beyond all word, it was great and full of excitement to work with them. Lastly, I cannot forget my beautiful children. I hope that when they grow up, they will understand that why I could not give them my attention and time during some of important years of their lives. This Ph.D. research work is supported by Higher Education Commission (HEC) of Pakistan under Indigenous Fellowship 5000 Program. I am also thankful to HEC for his support. MR. SHAUKAT ALI vii Abstract The paradigm of lifelogging promises the development of automatic systems for recording users' life events information digitally and develops an electronic prosthetic memory for providing complementary assistance to human biological memory. Several lifelogging systems are invented by assimilating computing and sensory technologies to capture, annotate, and retrieve lifelong information. The wearable technology has gained market traction; however, has several limitations including hard-to-work environment, number of sensors, uni-faceted, etc. Therefore, they are unable to use contextual semantics for organizing lifelog information like in human episodic memory. In addition, the large-scale adaptation of lifelogging is possible, if lifelogging functionality is integrated in devices that are already owned and maintained by users. To bridge the gap, this thesis examines smartphone technology for developing a common understanding of using smartphone as a de-facto lifelogging device. The various contextual semantics from smartphone sensors data and their potential applications in lifelogging are identified. The semantic model (i.e., ontology) is developed and tested for using the contextual semantics to organize, annotate, and relate lifelog information in similar to human episodic memory, and provide enough contextual cues to recall lifelog information like associative recall in human memory. The semantic framework is proposed to unify the research efforts by incorporating smartphone's sensors and processing capabilities, and semantic model to develop a semantically enriched digital prosthetic memory on smartphone. The proof-of-concept application called Semantic Lifelogging (SLOG) is developed to show the practicality of the proposed framework. The empirical evaluation has shown effectiveness of the methodology. In addition, future research directions are highlighted to help researchers in finding research topics. viii Table of Contents Certificate of Approval .................................................................................................................. i Forward Certificate ...................................................................................................................... ii Author's Declaration ................................................................................................................... iii Rights of Thesis ............................................................................................................................ iv Dedication ...................................................................................................................................... v Acknowledgements ...................................................................................................................... vi Abstract ........................................................................................................................................ vii Table of Contents ....................................................................................................................... viii List of Figures ............................................................................................................................. xiii List of Tables ............................................................................................................................... xv List of Abbreviations ................................................................................................................ xvii Chapter 1 : Introduction .............................................................................................................. 1 1.1 Introducing Lifelogging ........................................................................................................ 3 1.2 Lifelogging Types ................................................................................................................. 5 1.3 Terminologies........................................................................................................................ 6 1.3.1 Lifelogging and Personal Information Management (PIM) ........................................... 6 1.3.2 Lifelogging and Context-Awareness .............................................................................