Muscle fatigue detection using Infrared Thermography: Image segmentation to extract the region of interest from thermograms A thesis submitted to the Graduate School at the University of Cincinnati in partial fulfillment of the requirements for the degree of Master of Science in the Department of Electrical Engineering and Computing Systems, College of Engineering and Applied Science November 2018 By Dhyanesh Ramamoorthy, Bachelor of Engineering in Computer Science and Engineering, Anna University, India, 2010 Thesis Committee: Dr. Dharma Agrawal, Advisor Dr. Wen Ben Jone Dr. Yizong Cheng Abstract A major public health problem which affects people from all walks of life is muscle injury. Muscle fatigue can be defined as a reversible decrease in the contractile strength of the muscle(s) that occurs after long lasting or repetitive muscular activity. Whenever a certain muscle or muscle groups are subjected to repetitive movements and pushed beyond their limits, it leads to muscle fatigue, which results in sub-optimal performance. Muscle fatigue, when not diagnosed and treated in the early stages, leads to muscle injuries which tend to have long-lasting effects. The current technologies which are used to detect and treat muscle fatigue are highly invasive and often require the personnel to visit specialized clinics. The invasiveness and inaccessibility of muscle fatigue detection techniques have contributed to delayed diagnosis which leads to undesirable effects like permanent muscle damage. A non-invasive, cost-efficient method for detecting muscle fatigue is crucial. Whenever a muscle(s) is put to use, the corresponding muscle temperature increases. The human body maintains thermal homeostasis and it is achieved with the help of two processes called vasodilation and vasoconstriction. Vasodilation occurs when the temperature of a certain muscle group increases. The blood flow to this particular region is increased and the heat is absorbed by the blood capillaries and dissipated through the skin. Infrared thermography is a non-radiating and contact-free technology which measures the surface temperature of objects. As skin surface temperature has been proven to be a good i indicator of localized muscle fatigue, Infrared thermography can be an effective non-invasive associative technology which can be used to detect muscle fatigue. In our study, we have conducted experiments which involve test subjects performing repetitive exercises involving the upper body muscle groups. The thermal images of the test subjects were taken before and after the exercise routine. We have reviewed the widely used image segmentation and image classification algorithms used to extract the regions of interest from an image and implemented two algorithms which are suitable for analyzing low signal-to-noise thermal images. The changes in temperature values of the regions of interest have been tabulated and the results indicate a clear correlation between muscle fatigue and increase in temperature. ii iii This work is dedicated to my dear parents Dr. M. Ramamoorthy and Dr. V. Renugadevi & my beloved wife Mrs. Vithusini Senthil Kumar iv Acknowledgements First and foremost, I am deeply grateful and thank my advisor and committee chair, Dr. Dharma P Agrawal for his constant encouragement, guidance and support throughout the duration of my master’s degree. I would like to thank Dr. Wen Ben Jone and Dr. Yizong Cheng for serving on my thesis defense committee. I am also thankful to my colleagues at the center for distributed and mobile computing. Their valuable feedback and suggestions during our lab meetings were instrumental in completing my thesis research. Finally, I must express my gratitude, love and affection to my parents and my wife for believing in me and providing me with constant support and encouragement during all these years of my study. v TABLE OF CONTENTS 1. Introduction.............................................................................................. 1 1.1 Research Motivation ..................................................................................... 1 1.2 Muscle Fatigue .............................................................................................. 1 1.3 Infrared thermal imaging (IRTG) in Medical applications ........................... 3 1.4 Advantages of using IRTG for fatigue analysis ............................................ 4 1.5 Purpose and Scope of this study ................................................................... 6 2. Foundation ............................................................................................... 9 2.1 Heat regulation in Human body .................................................................... 9 2.2 Hypothermia and Muscle Fatigue ............................................................... 10 2.3 Infrared Thermal Imaging ........................................................................... 10 2.4 Thermal Energy and Emissivity.................................................................. 13 2.5 Thermographic Camera .............................................................................. 15 3. Experimentation ..................................................................................... 18 3.1 Intended purpose of the experiments .......................................................... 18 3.2 Guidelines ................................................................................................... 18 3.3 Participant Selection Criteria ...................................................................... 20 3.4 Instruments used for the Experiments ......................................................... 21 3.5 Experimental Results .................................................................................. 23 4. Analysis ................................................................................................. 29 4.1 Thermal Image processing .......................................................................... 29 4.2 Thermal Image processing Algorithms ....................................................... 30 4.3 Image Segmentation.................................................................................... 32 4.4 Image Classification.................................................................................... 44 5. Results & Conclusion ............................................................................ 52 5.1 Analysis of the ROI extraction algorithms ................................................. 52 5.2 Conclusion .................................................................................................. 61 6. Future Work ........................................................................................... 63 7. Bibliography .......................................................................................... 64 vi LIST OF FIGURES Figure 2.1 The Electromagnetic spectrum and the wavelengths of Infrared waves ................... 11 2.2 A simplified block diagram of the workings of a thermal camera ........................... 15 2.3 Process flow diagram involved in capturing a thermal image .................................. 16 3.1 A manual analysis of the first testsubject’s thermograms ........................................ 24 3.2 A manual analysis of the second testsubject’s thermograms .................................... 25 3.3 A manual analysis of the third testsubject’s thermograms ....................................... 26 3.4 A manual analysis of the fourth testsubject’s thermograms ..................................... 27 3.5 A manual analysis of the fifth testsubject’s thermograms ........................................ 28 4.1 The original thermal image before segmentation. .................................................... 35 4.2 A histogram plot of the pixel intensities of the original thermal image ................... 36 4.3 The Gaussian-smoothed thermal image .................................................................... 37 4.4 The thermal image classified using 2-level Otsu segmentation ................................ 38 4.5 The thermal image classified using 3-level Otsu segmentation ................................ 39 4.6 Block diagram of the Canny edge detection algorithm ............................................ 41 4.7 The Canny Edge detector applied to a thermal image .............................................. 43 4.8 An illustration of integral image calculation ............................................................ 47 4.9 Calculating the sum of the pixels within a rectangle using integral image. ............. 48 4.10 Block diagram explaining the functioning of a Cascade classifier ......................... 49 5.1 ROI extraction in Subject 1 and Average temperature calculation. .......................... 53 5.2 ROI extraction in Subject 2 and Average temperature calculation. .......................... 54 5.3 ROI extraction in Subject 3 and Average temperature calculation. .......................... 55 vii LIST OF FIGURES(Cont.) Figure 5.4 ROI extraction in Subject 4 and Average temperature calculation. .......................... 56 5.5 ROI extraction in Subject 5 and Average temperature calculation. .......................... 57 5.6 Average Temperature of ROI from 3-level Otsu segmentation................................ 58 5.7 Average Temperature of ROI from 2-level Otsu segmentation................................ 59 5.8 Average Temperature of ROI from Haar-cascade classification .............................