Mobile Phones As Cognitive Systems
Total Page:16
File Type:pdf, Size:1020Kb
Downloaded from orbit.dtu.dk on: Oct 10, 2021 Mobile Phones as Cognitive Systems Stopczynski, Arkadiusz Publication date: 2015 Document Version Publisher's PDF, also known as Version of record Link back to DTU Orbit Citation (APA): Stopczynski, A. (2015). Mobile Phones as Cognitive Systems. Technical University of Denmark. DTU Compute PHD-2014 No. 336 General rights Copyright and moral rights for the publications made accessible in the public portal are retained by the authors and/or other copyright owners and it is a condition of accessing publications that users recognise and abide by the legal requirements associated with these rights. Users may download and print one copy of any publication from the public portal for the purpose of private study or research. You may not further distribute the material or use it for any profit-making activity or commercial gain You may freely distribute the URL identifying the publication in the public portal If you believe that this document breaches copyright please contact us providing details, and we will remove access to the work immediately and investigate your claim. Mobile Phones as Cognitive Systems Arkadiusz Stopczynski Technical University of Denmark Applied Mathematics and Computer Science Building 324, DK-2800 Kongens Lyngby, Denmark Phone +45 45253031, Fax +45 45253031 [email protected] www.compute.dtu.dk PHD: ISSN 0909-3192 Summary (English) Driven by the ubiquitous availability of data and inexpensive data storage, our ability to sense human beings has increased dramatically. Big data has perme- ated the public discourse and led to surprising insights across the sciences and the humanities. This dissertation presents research on expanding our capabil- ities in collecting, handling, processing, and using data collected about human beings to create an integrated view of social systems. The goal of the thesis has been threefold. The first part of the thesis focuses on the need, design, and implementation of large-scale sensor-driven human data collection studies. Social networks can be measured with high resolution and on multiple channels, such as face-to- face meetings, social networks, or phone calls, in order to generate a more comprehensive picture of social systems. The largest study to date measur- ing large-scale social system—the Copenhagen Networks Study—is described, together with motivation and challenges of the deployment. Preliminary results are presented, indicating how a possibly biased and incomplete picture can be generated when data are collected from a single channel and with a low resolu- tion, thus emphasizing the importance of the proposed approach and deployed implementation. The second part of the thesis deals with expanding our capabilities to sense the cognitive and emotional state of the users through development of a sys- tem for mobile brain imaging—the Smartphone Brain Scanner. A developed framework allows for EEG data collection and processing. It also provides the ability to build end-user applications on top of raw data and extracted features using off-the-shelf and custom-built neuroheadsets and mobile devices, thereby ii potentially becoming another channel in integrated human sensing. The motiva- tion for creating such system is presented, advanced data processing—3D source reconstruction—is explained, and applications and use-cases are discussed. In the third part, the privacy issues surrounding the handling of such sensi- tive behavioral and biomedical data are investigated. A comprehensive review of best privacy practices in sensor-driven human data collection is presented and recommendations for practitioners are made. Based on this review and experiences with the Copenhagen Networks Study and the Smartphone Brain Scanner, the concept of Living Informed Consent is presented, which postulates larger participant control over collected data for the benefit of users, researchers, and society at large. The same privacy principles are applied to a personal neu- roinformatics context, resulting in a proposed new approach to sensitive EEG data handling. Resumé (Dansk) Drevet af den øgede tilgængelighed af data samt den lave pris på datalagring, er vores mulighed for at indsamle data om mennesker steget dramatisk. “Big data” har gennemsyret den offentlige diskurs og ført til overraskende indsigter på tværs af naturvidenskab og humaniora. Denne afhandling beskæftiger sig med data indsamlet om mennesket og præsenterer forskning der udvider vores kompetencer indenfor indsamling, håndtering, forarbejdning og anvendelse for at skabe et samlet billede af sociale systemer. Målet med denne afhandling er tredelt. Den første del af denne afhandling fokuserer på sensor-drevet indsamling af data og diskuterer behovet, designet, samt implementeringen af denne type studier. Sociale netværk kan måles med høj opløsning og via flere kanaler, såsom ansigt- til-ansigt møder, online netværk eller telefonopkald. For at få et fyldestgørende billede af mennesket skal man dække alle vore kommunikationskanaler. Udfor- dringer samt motivationen bag den til dato største indsamling af sensor-dreven data, med titlen Copenhagen Networks Study, bliver beskrevet og diskuteret. Foreløbige data bliver præsenteret og indikerer udfordringen i bias og mulighe- den for slutninger baseret på ufuldstændige data fra kun en kanal. Dette un- derstreger betydningen af den beskrevne fremgangsmåde. Den anden del af denne afhandling omhandler et nyt mobilt system, kaldet Smartphone Brain Scanner, der udvider vores nuværende rammer til mobilt at kunne registrere kognitive tilstande hos mennesker. Den udviklede metode gør det muligt at både indsamle og behandle EEG data dynamisk. Yderligere, åbner den muligheder for at brugere selv kan bygge og anvende nye applika- tioner på det indsamlede data. Metoden er både kompatibel med standard- iv komponenter, specialbyggede neuro-headset samt andre mobile enheder og kan dermed potentielt blive en ny kanal i indsamlingen af data om mennesket. Denne del beskriver motivationen bag skabelsen af systemet, forklarer de avancerede databehandlingsmetoder og giver eksempler på hvorledes et sådant system kan være nyttigt. I den tredje del undersøges og diskuteres hånderingen af disse følsomme data med henblik på beskyttelsen af privatlivets fred. En omfattende gennemgang belyser de nuværende fremgangsmåder og giver detaljerede anbefalinger til frem- tidige studier der omhandler anvendelse af denne slags personfølsomme data. Baseret på denne gennemgang og de to projekter: Copenhagen Networks Study og Smartphone Brain Scanner introduceres begrebet Living Informed Consent. Her postuleres at større individuel kontrol over data er til gavn for brugere, forskere og samfundet generelt. De samme principper om privatlivets fred an- vendes i sammenhængen med personlig neuroinformatik og resulterer i en ny tilgang til håndtering af følsomme EEG data. Preface This thesis was prepared at the Department of Applied Mathematics and Com- puter Science at the Technical University of Denmark (DTU) in partial fulfill- ment of the requirements for acquiring the Ph.D. degree in engineering. The thesis consists of a summary report and a collection of one book chap- ter, four published scientific papers, and four upcoming papers. The work was carried out between 2011 and 2014. Lyngby, 31-March-2014 Arkadiusz Stopczynski vi List of Publications Papers included in the thesis [A] Daniel Greenwood, Arkadiusz Stopczynski, Brian Sweatt, Thomas Hardjono, and Alex Pentland. Institutional Controls: The New Deal on Data (Book Chapter). Big Data, Privacy, and the Public Good: Frameworks for Engagement. Edited by Julia Lane, Victoria Stodden, Stefan Bender, and Helen Nissenbaum. Cambridge University Press, 2014. Hardback ISBN 9781107067356; paperback ISBN 9781107637689. [B] Arkadiusz Stopczynski, Vedran Sekara, Piotr Sapiezynski, Andrea Cuttone, Mette My Madsen, Jakob Eg Larsen, and Sune Lehmann. Measuring Large-Scale Social Networks with High Resolution. Under Review in PLOS ONE. arXiv preprint arXiv:1401.7233 (2014). [C] Yves-Alexandre de Montjoye, Arkadiusz Stopczynski, Erez Shmueli, Alex ‘Sandy’ Pent- land, and Sune Lehmann. The Strength of the Strongest Ties in Collaborative Problem Solv- ing. Under Review in Scientific Reports (2014). [D] Arkadiusz Stopczynski, Carsten Stahlhut, Michael Kai Petersen, Jakob Eg Larsen, Camilla Falk Jensen, Marieta Georgieva Ivanova, Tobias S. Andersen, and Lars Kai Hansen. Smartphones as pocketable labs: Visions for mobile brain imaging and neurofeedback. Inter- national Journal of Psychophysiology, Volume 91, Issue 1, January 2014, Pages 54-66, ISSN 0167-8760, doi:10.1016/j.ijpsycho.2013.08.007. [E] Michael Kai Petersen, Carsten Stahlhut, Arkadiusz Stopczynski, Jakob Eg Larsen, and Lars Kai Hansen. Smartphones get emotional: mind reading images and reconstructing the neural sources. In Affective Computing and Intelligent Interaction, pp. 578-587. Springer Berlin Heidelberg, 2011. [F] Arkadiusz Stopczynski, Carsten Stahlhut, Jakob Eg Larsen, Michael Kai Petersen, and Lars Kai Hansen. The Smartphone Brain Scanner: A Portable Real-time Neuroimaging System. PLoS ONE 9(2): e86733. doi:10.1371/journal.pone.0086733. viii [G] Carsten Stahlhut, Hagai Thomas Attias, Arkadiusz Stopczynski, Michael Kai Petersen, Jakob Eg Larsen, and Lars Kai Hansen. An evaluation of EEG scanner’s dependence on the imaging technique, forward model computation method, and array dimensionality. Engineer- ing in Medicine and Biology Society (EMBC), 2012 Annual International Conference