A Framework for Automated Heart and Lung Sound Analysis Using a Mobile Telemedicine Platform by Katherine L. Kuan Bachelor of Science in Electrical Engineering and Computer Science, Massachusetts Institute of Technology (2009) Submitted to the Department of Electrical Engineering and Computer Science in Partial Fulfillment of the Requirements for the Degree of Master of Engineering in Electrical Engineering and Computer Science at the Massachusetts Institute of Technology July 2010 ©2010 Massachusetts Institute of Technology All rights reserved. Author....................................................................... Department of Electrical Engineering and Computer Science July 23, 2010 Certified by................................................................. Dr. Gari D. Clifford Associate Director Centre for Doctoral Training in Healthcare Innovation University of Oxford Thesis Supervisor Certified by................................................................. Prof. P. Szolovits Professor in Health Sciences and Technology and Electrical Enginee ring and Computer Science Thesis Co-Supervisor Accepted by.................................................................. Dr. Christopher J. Terman Chairman, Department Committee on Graduate Theses A Framework for Automated Heart and Lung Sound Analysis Using a Mobile Telemedicine Platform by Katherine L. Kuan Submitted to the Department of Electrical Engineering and Computer Science July 23, 2010 In Partial Fulfillment of the Requirements for the Degree of Master of Engineering in Electrical Engineering and Computer Science ABSTRACT Many resource-poor communities across the globe lack access to quality healthcare, due to shortages in medical expertise and poor availability of medical diagnostic devices. In recent years, mobile phones have become increasingly complex and ubiquitous. These devices present a tremendous opportunity to provide low-cost diagnostics to under-served populations and to connect non-experts with experts. This thesis explores the capture of cardiac and respiratory sounds on a mobile phone for analysis, with the long-term aim of developing intelligent algorithms for the detection of heart and respiratory-related problems. Using standard labeled databases, existing and novel algorithms are developed to analyze cardiac and respiratory audio data. In order to assess the algorithms’ performance under field conditions, a low-cost stethoscope attachment is constructed and data is collected using a mobile phone. Finally, a telemedicine infrastructure and work-flow is described, in which these algorithms can be deployed and trained in a large-scale deployment. Thesis Supervisor: Gari D. Clifford Title: Associate Director, Centre for Doctoral Training, Institute of Biomedical Engineering, Department of Engineering Science, University of Oxford Thesis Co-supervisor: Peter S. Szolovits Title: Head of the Clinical Decision-Making Group, MIT Computer Science and Artificial Intelligence Laboratory (CSAIL), Professor of Computer Science and Engineering, Department of Electrical Engineering and Computer Science (EECS), Professor of Health Sciences and Technology, Harvard/MIT Division of Health Sciences 2 Acknowledgments First and foremost, I would like to express my gratitude to Brian and Tabetha Hinman and the MIT Public Service Center, who generously funded this work through a public service graduate fellowship. Without this funding and their continued encouragement, this thesis would not have been possible. Google also funded the work on the media viewer for OpenMRS and Sana through the Google Summer of Code (GSOC) 2009. Sana, which forms the backbone informatics system described in this thesis, has been funded by Telmex, the MIT PSC, Google, the Institute of Electrical and Electronics Engineers, the MIT International Science and Technology Initiatives, Vodaphone Foundation (through the Wireless Innovation Project), and the mHealth Alliance (formed by the United Nations, Rockefeller, and Vodafone Foundations). In addition, I would like to extend a heartfelt thank you to the following people and organizations: To Gari Clifford for mentoring me throughout the past one-and-a-half years (for GSOC, PSC Fellowship, class term projects, and my Master’s thesis). I learned so much from him about the global health field and realities of healthcare technology adoption in resource-poor areas. He was incredibly helpful on my thesis, always available for questions, and taught me a great deal about the skills needed to become a successful scientific researcher. 3 To Peter Szolovits for his experienced advice and guidance on my thesis work, instruction during the HST Biomedical Computing course, and support of the Sana team. To Tiffany Chen for her heart sound analysis code implementation, as well as time spent helping me capture heart sounds with various cell phones and preparing the paper for conference submission. To Shamim Nemati for his code implementation and explanations on the autoregressive model method and signal purity algorithms (to be applied on lung sounds). To John Guttag, Dorothy Curtis, Zeeshan Syed, and Gina Yi for providing me with access to the heart and lung sound data and files from previous research. To Gina for her code on lung sound analysis as well. To Jose Gomez-Marquez for the opportunity to lead a lab activity in his Spring 2009 D- Lab Health class on building a stethoscope attachment for cell phones with his students. To Hamish Fraser and Partners in Health for the experience of being a GSOC student for OpenMRS. That summer was my first glimpse at how a vibrant open source community can thrive with talented members who are open and willing to help each other out anytime. 4 To the MIT PSC (especially Sally Susnowitz and Alison Hynd) and the U.P. Manila National Telehealth Center (especially Dr. Alvin Marcleo) for the opportunity to do a fellowship in the Philippines during the Summer of 2009. It was undoubtedly one of the most thought-provoking and humbling experiences of my time at MIT. To Sana for being the most inspiring team of developers, doctors, business people, faculty, and public health people. Being part of such a dynamic and passionate team has been such a wonderful experience. Thank you to Leo Anthony Celi, Richard Lu, and the other Sana doctors for always answering my questions about the medical context of this system and for providing feedback on the prototype as it evolved. To RJ Ryan for always being helpful and willing to pass on his incredible knowledge of software development to the other developers and me. To Jhonatan Rotberg for his help on the team, including UROP funding during my first semester on Sana and for providing hardware phone devices. To all the others on the team, especially the other developers, who aided me in many different ways. To my dad, mom, and sister for their tremendous, unwavering support and love throughout this thesis, my MIT undergraduate years, and life as a whole. A big thank you all my family and friends who helped me from brainstorming for device instrumentation (especially my dad) to letting me record their heart and lung sounds. I apologize if I inadvertently left anyone out, but so many people were helpful and I am grateful for their kindness. 5 Several open-source MATLAB toolboxes were used and adapted for this work: Gari Clifford’s (2002) ECG toolbox, Tiffany Chen’s (2010) heart sound segmentation algorithm, Gina Yi’s lung sound analysis toolkit (2004), and Shamim Nemati’s (2010) respiration analysis code. Descriptions and references to the original works are included in this thesis where they have been applied or modified. The mobile phone frequency response curves in Chapter 4 (Figures 4-5 to 4-7 and Table 4-1) were included with permission from the GSMArena.com team with appropriate citations. The percentile charts for evidence-based respiratory and heart rates in children compared to international ranges were also included with permission from Susannah Fleming (2010). These are located in Figures 1-2 and 1-5, as well as Tables 1-1 and 1- 2. The results on heart sound analysis were published earlier this year at the AAAI AI-D Spring Symposium in Stanford, CA from March 23-24, 2009. The article was called “Intelligent Heartsound Diagnostics on a Cellphone using a Hands-free Kit” with authors Tiffany Chen, Katherine Kuan, Leo Celi, and Gari Clifford (2010). The project Sana was also published in the Journal of Health Informatics in Developing Countries in Volume 3, Number 1 in 2009 under Sana’s previous name of Moca. The article was entitled “Mobile Care (Moca) for Remote Diagnosis and Screening” by Leo Anthony Celi, Luis Sarmenta, Jhonatan Rotberg, Alvin Marcleo, and Gari Clifford for the 6 Moca team. The conceptual design of the telemedicine infrastructure for capturing, labeling and transmitting resultant diagnostics back to the patient was developed by a large team of engineers at MIT from the Sana team. The system was designed by Drs. Celi and Clifford of MIT’s Division of Health Sciences and Technology for the Edgerton Center class on Information and Communication Technologies for Developing Countries; ICT4D SP.716 (currently known as Nextlab) in the Spring of 2008 (MIT OpenCourseWare 2008). In the Fall of 2008, the client framework evolved from a nascent Symbian prototype developed by Andres Monroy-Hernandez to become an Android-based software system by the work of students RJ Ryan, Zack Anderson, and Boyuan Zhu in Professor Abelson’s Mobile Programming Course. This system