Acoustic Gait Analysis Using Support Vector Machines Jasper Huang1, Fabio Di Troia1, and Mark Stamp1 1Department of Computer Science, San Jose State University, San Jose, California [email protected], [email protected], [email protected] Keywords: Gait recognition; support vector machine; acoustic analysis; biometric. Abstract: Gait analysis, defined as the study of human locomotion, can provide valuable information for low-cost an- alytic and classification applications in security, medical diagnostics, and biomechanics. In comparison to visual-based gait analysis, audio-based gait analysis offers robustness to clothing variations, visibility issues, and angle complications. Current acoustic techniques rely on frequency-based features that are sensitive to changes in footwear and floor surfaces. In this research, we consider an approach to surface-independent acoustic gait analysis based on time differences between consecutive steps. We employ support vector ma- chines (SVMs) for classification. Our approach achieves good classification rates with high discriminative one-vs-all capabilities and we believe that our technique provides a promising avenue for future development. 1 Introduction has been significantly less focus on acoustic-based gait recognition, information from gait audio has been Biometrics—the analysis of human physiological shown to be useful as well. Audio-based research and behavioral characteristics—has become a leading has yielded promising results using frequency anal- method of person identification, verification, and clas- ysis of both footstep sounds and those produced by sification. Image-based biometric techniques include clothing movement and contact (Geiger et al., 2014). iris, face, and fingerprint analysis. These typically ne- However, one issue that arises with frequency-related cessitate subject cooperation and a highly controlled data is sensitivity to contact surfaces, as variations in environment. For example, current fingerprinting and footwear, clothing, floor surfaces, etc., produce dif- hand geometry systems often require a specific place- ferent sounds that may adversely affect classification ment or orientation of the finger or hand, while facial rates. Time-domain analysis would appears to be recognition algorithms can be complicated by non- a promising avenue of research that could mitigate frontal facial images and uncooperative subjects. One some of these issues. In this research, we consider biometric that addresses many of these limitations is temporal acoustic analysis for gait recognition. human gait, a person’s manner of walking. Gait anal- Although gait information alone might not be ysis is the systematic study of human walking, which sufficient to distinguish each individual in a large can be defined as the method of body locomotion for database, gait analysis could still be advantageous for support and propulsion (Whittle, 2014). In compari- low-cost pre-screening. Such technologies could be son to most other biometrics, gait analysis is less in- used, for example, as part of an anomaly detection trusive, and allows for passive monitoring. system in a smart home, for access control in high- Previous work has demonstrated that people can security buildings, and for surveillance in indoor en- recognize subjects by observing illuminated moving vironments. Our audio-based system can easily be joints (Johansson, 1973), and research has also shown combined with existing techniques for a multimodal that people are able to reliably identify co-workers gait recognition system. by listening to their footstep sounds (Makela et al., Compared to visual systems, acoustic techniques 2003). Human gait analysis has also been applied to are not sensitive to changes in illumination, visibility, problems in medical diagnosis (Murray, 1967). and walking angle. The time-based features we ana- The majority of current gait analysis techniques lyze are also likely to be more robust with respect to are applied in the visual domain—the field of visual changes in clothing, footwear, and floor surface. In gait recognition has been an active area of research for addition, acoustic analysis requires only inexpensive at least the past 15 years (Lee, 2002). Although there sensors with minimal sensor density. Our gait analysis method is also relevant in clin- an instance of the more general problem of sound ical and biomechanical applications. The fields of recognition, which is related to automatic speech gait and human movement sciences have proven use- recognition and speaker identification. Current gen- ful in the analysis, diagnosis, and treatment of vari- eral purpose speech recognition systems often ap- ous afflictions (Lai et al., 2009). For example, gait ply hidden Markov models (HMMs) and other sta- analysis has been applied to the diagnosis of cere- tistical techniques to n-dimensional real-valued vec- bral palsy (Kamruzzaman and Begg, 2006) and age- tors consisting of coefficients extracted from words or related issues (Begg and Kamruzzaman, 2005; Begg phonemes. Gaussian mixture models are commonly et al., 2005). With minor modification, our technique used for text-independent verification (Bimbot et al., has the potential for application in such domains. 2004; Reynolds, 1995). Such models combine com- The organization of the remainder of this paper putational feasibility with statistical robustness. is as follows. In Section2, we provide background Similar methods have been extended to acoustic information, including a brief survey of relevant lit- gait recognition. Footstep detection and identification erature in gait and audio analysis. In Section3, we techniques are presented in (She, 2004) and (Shoji outline our methodology and in Section4 we present et al., 2004), respectively. In (Itai and Yasukawa, experimental results. We conclude the paper in Sec- 2008), dynamic time warping and cepstral informa- tion5 and provide suggestions for future work. tion extracted from acoustic gait data is used for clas- sification, while audio data recorded on a staircase— for the purpose of recognizing home inhabitants— 2 Related Work is considered in (Alpert and Allen, 2010). A recent study (Geiger et al., 2014) extracts mel-frequency Gait has generally been interpreted as a visual cepstral coefficients (MFCCs) as audio features and phenomenon, necessitating that the person be seen uses HMMs with a cyclic topology for dynamic clas- to be recognized or classified. Not surprisingly, the sification. bulk of research in gait analysis has been visual- based, relying on video or image data. In authen- tication applications, gait analysis typically involves side-view silhouette features from the spatiotemporal 3 Methodology domain (Wang et al., 2003). A variety of data anal- ysis techniques have been employed, such as hidden Markov models on sequences of feature vectors cor- The objective of this research is to consider a sim- responding to different postures (Kale et al., 2002; ple method for surface-independent audio based gait Sundaresan et al., 2003). In another comprehensive analysis. To address surface-independence, we use study (Man and Bhanu, 2006), a spatiotemporal “gait time domain analysis, as opposed to frequency anal- energy image” forms the basis for gait analysis. A ysis, which is likely to be much more vulnerable to review of human motion analysis in the field of com- surface variations. puter vision appears in (Aggarwal and Cai, 1997). Gait recognition methods that do not involve During a gait cycle, there are multiple distinct vi- video or audio information include Doppler tech- sual stances that a subject transitions through. In a niques (Kalgaonkar and Raj, 2007; Otero, 2005; similar vein, it is likely that a subject’s gait patterns Tahmoush and Silvious, 2009), floor pressure sen- varies in a way that affects the timing between con- sors (Qian et al., 2008; Middleton et al., 2005), secutive footsteps. Therefore, we consider a sequence and smartphone accelerometers (Sprager and Zazula, of time intervals between successive steps, which rep- 2009; Nishiguchi et al., 2012; Mantyjarvi et al., resents a subject’s characteristic gait cadence. We 2005). In medical applications, many gait analysis extract statistical data from this set to obtain feature techniques are also inherently image-based, relying vectors based on footstep sound recordings. Given on features derived from kinetics and kinematics (Ag- numerous labeled feature vectors, we employ a su- garwal and Cai, 1997), such as hip and pelvis accel- pervised learning technique for classification. Here, eration (Aminian et al., 2002). A survey of computa- we provide preliminary results for the effectiveness tional intelligence in gait research within the medical of this approach, based on a small dataset. field can be found in (Lai et al., 2009). Next, we describe the procedure by which we de- An alternative approach to gait analysis is to cap- termine our input features. Then we briefly discuss ture the sounds of walking, that is, acoustic gait anal- support vector machines (SVM), which are used for ysis. Current approaches typically treat the task as classification in this research. 3.1 Input Features where the xi are the CTI sequence and n is the length of this sequence. To measure audio gait characteristics, a microphone • Standard deviation — The variance measures the is assumed to be in a static position on the floor, spread of the data about the mean and will reveal and only one person’s footsteps are monitored at a 1 the variability in stride times. The standard devi- time. From the sound wave data, we detect foot- ation, which is the square root of the variance, is step peaks as shown in Figure1 and compile a se- computed as quence of times, denoted t1;t2;:::;tn+1, at which a s particular individual’s footsteps are heard. Next, time n 2 ∑ (xi − µ) intervals between successive steps are calculated and s = i=1 n − 1 compiled into a second sequence xi = ti+1 −ti, yield- ing x1;x2;:::;xn. We will refer to the xi as the con- where µ is the mean of the CTI. secutive time interval (CTI) sequence, which repre- sents time-based cadence and serves as the base for • Skewness — The symmetry of a distribution can our analysis.