Wearable Sensor System for Human Localization and Motion Capture

by Shaghayegh Zihajehzadeh M.Sc., Amirkabir University of Technology, 2011 B.Sc., Isfahan University of Technology, 2008

Thesis Submitted in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy

in the School of Mechatronic Systems Engineering Faculty of Applied Sciences

 Shaghayegh Zihajehzadeh 2017 SIMON FRASER UNIVERSITY Spring 2017

Approval

Name: Shaghayegh Zihajehzadeh Degree: Doctor of Philosophy Title: Wearable Sensor System for Human Localization and Motion Capture Examining Committee: Chair: Gary Wang Professor

Edward J. Park Senior Supervisor Professor

Stephen Robinovitch Supervisor Professor

Kevin Oldknow Supervisor Senior Lecturer

Carolyn Sparrey Internal Examiner Associate Professor

Benny Lo External Examiner Lecturer Department of Surgery and Cancer Imperial College London

Date Defended/Approved: April 27, 2017

Ethics Statement

iii

Abstract

Recent advances in MEMS wearable inertial/magnetic sensors and mobile computing have fostered a dramatic growth of interest for ambulatory human motion capture (MoCap). Compared to traditional optical MoCap systems such as the optical systems, inertial (i.e. accelerometer and gyroscope) and magnetic sensors do not require external fixtures such as cameras. Hence, they do not have in-the-lab measurement limitations and thus are ideal for ambulatory applications. However, due to the manufacturing process of MEMS sensors, existing wearable MoCap systems suffer from drift error and accuracy degradation over time caused by time-varying bias. The goal of this research is to develop algorithms based on multi-sensor fusion and machine learning techniques for precise tracking of human motion and location using wearable inertial sensors integrated with absolute localization technologies. The main focus of this research is on true ambulatory applications in active sports (e.g., skiing) and entertainment (e.g., gaming and filmmaking), and health-status monitoring. For active sports and entertainment applications, a novel sensor fusion algorithm is developed to fuse inertial data with magnetic field information and provide drift-free estimation of human body segment orientation. This concept is further extended to provide ubiquitous indoor/outdoor localization by fusing wearable inertial/magnetic sensors with global navigation satellite system (GNSS), barometric pressure sensor and ultra-wideband (UWB) localization technology. For health applications, this research is focused on longitudinal tracking of walking speed as a fundamental indicator of human well-being. A regression model is developed to map inertial information from a single waist or ankle- worn sensor to walking speed. This approach is further developed to estimate walking speed using a wrist-worn device (e.g., a smartwatch) by extracting the arm swing motion intensity and frequency by combining sensor fusion and principal component analysis.

Keywords: inertial/magnetic sensor; orientation estimation; position estimation; walking speed; Kalman filter; Gaussian process regression.

Dedication

I dedicate this work to my caring husband and devoted parents.

Acknowledgements

I would like to thank my senior supervisor, Dr. Edward Park, for providing invaluable mentorship, technical expertise and encouragement throughout my PhD studies. His support, inspiration, professional guidance and patience helped me enjoy every moment of working on this thesis. I would also like to thank my supervisory committee: Dr. Stephen Robinovitch and Dr. Kevin Oldknow for providing expert opinion and evaluating different parts of this research. I am also thankful to Dr. Farid Golnaraghi for his confidence in me and accepting me to this PhD program and guiding me during the first few months of my studies.

I would like to thank the Natural Sciences and Engineering Research Council of Canada (NSERC) for the financial support through Vanier Canada Graduate Scholarship (CGS) program.

I am grateful to my lab colleagues and friends: Dr. Omar Aziz, Dr. Ahmed Arafa, Darrell Loh, Matthew Lee, Paul Yoon, Magnus Musngi, Dr. Majid Shokoufi, Shervin Jannesar, and Behzad Abdi. I am glad I had the opportunity to meet and spend time with these amazing people.

Many thanks go to my lovely friends Nastaran Hajinazar, Amir Pourmand and Rajesh Rao for their unconditional friendship and support, making my life more joyful. Family isn’t always blood; you are my family in Canada.

My special thanks go to my caring husband, Ramin who motivated me in so many ways. There are no words that can express my gratitude and appreciation for all you have done and been for me. Your continued and unfailing love, support and understanding made the completion of this thesis possible. I am thoroughly thankful to my parents and parents-in-law for their encouragement and unwavering belief in me.

Table of Contents

Approval ...... ii Ethics Statement ...... iii Abstract ...... iv Dedication ...... v Acknowledgements ...... vi Table of Contents ...... vii List of Acronyms ...... ix

Chapter 1. Introduction ...... 1 1.1. Background and Motivation ...... 1 1.2. Objectives ...... 2 1.3. Organization of the Dissertation ...... 3

Chapter 2. Literature Review ...... 5 2.1. MoCap of human body segments ...... 5 2.2. Indoor and outdoor localization ...... 7 2.3. Walking speed estimation using wearable IMU ...... 10

Chapter 3. Summary of Contributions ...... 13 3.1. Estimation of Human Body Segment Orientation ...... 13 3.1.1. A cascaded two-step KF for estimation of human body segment orientation using wearable IMU ...... 13 3.2. Tracking of Position and Velocity Trajectories ...... 14 3.2.1. Integration of MEMS inertial and pressure sensors for vertical trajectory determination ...... 14 3.2.2. A cascaded Kalman filter-based GPS/MEMS-IMU integration for sports applications ...... 15 3.2.3. UWB-aided inertial motion capture for lower body 3-D dynamic activity and trajectory tracking ...... 16 3.3. Magnetometer-free Motion Tracking ...... 18 3.3.1. A magnetometer-free indoor human localization based on loosely coupled IMU/UWB fusion ...... 18 3.3.2. A novel biomechanical model-aided IMU/UWB fusion for magnetometer-free lower body motion capture ...... 19 3.4. Regression Model-based Walking Speed Estimation ...... 20 3.4.1. Experimental evaluation of regression model-based walking speed estimation using lower body-mounted IMU ...... 20 3.4.2. Regression model-based walking speed estimation using wrist- worn inertial sensor ...... 21

Chapter 4. Conclusions and Future Work ...... 23 4.1. Conclusions and Contributions ...... 23

vii

4.2. Future Work ...... 25

References ...... 27 Appendix A. A cascaded two-step Kalman filter for estimation of human body segment orientation using MEMS-IMU ...... 35 Appendix B. Integration of MEMS inertial and pressure sensors for vertical trajectory determination ...... 40 Appendix C. A cascaded Kalman filter-based GPS/MEMS-IMU integration for sports applications ...... 52 Appendix D. UWB-aided inertial motion capture for lower body 3-D dynamic activity and trajectory tracking ...... 64 Appendix E. A magnetometer-free indoor human localization based on loosely coupled IMU/UWB fusion ...... 76 Appendix F. A novel biomechanical model-aided IMU/UWB fusion for magnetometer-free lower body motion capture ...... 81 Appendix G. Experimental evaluation of regression model-based walking speed estimation using lower body-mounted IMU ...... 94 Appendix H. Regression model-based walking speed estimation using wrist-worn inertial sensor ...... 99

viii

List of Acronyms

EKF Extended Kalman Filter FD Frequency Domain G-N Gauss Newton GNSS Global Navigation Satellite System GPR Gaussian Process Regression GPS Global Positioning System IMU Inertial Measurement Unit INS Inertial Navigation System KF Kalman Filter KPVs Key Performance Variables MEMS Micro-Electro-Mechanical Systems MoCap Motion Capture NLOS Non-Line-of-Sight O2OQ Optimal Two-Observation Quaternion Estimation PCA Principal Component Analysis PIR Passive Infrared RFID Radio Frequency Identification RTS Rauch-Tung-Striebel SVR Support Vector Regression TD Time Domain TOA Time of Arrival TUG Timed-Up-and-Go UKF Unscented Kalman Filter UWB Ultra-Wideband ZUPT Zero Velocity Update

Chapter 1.

Introduction

1.1. Background and Motivation

Simultaneous localization and motion capture (MoCap) of human body segments is important in numerous applications such as sports science [1]-[2], rehabilitation [3] and health monitoring [4]. For instance, in sports, access to quantitative key performance variables (KPVs) can significantly improve overall performance of the athletes [5]-[6]. In rehabilitation, quantitative evaluation of body segment motion in patients with neurological disorders who suffer from dysfunction of the limbs can expedite their recovery process through customized exercise plans [7]. In health monitoring, walking speed can be used as a precursor to predict mild cognitive impairment [8], multiple sclerosis [9], Parkinson’s disease [10], chronic obstructive pulmonary disease [10] and risk of falls [11]. Currently available video-based or camera- based MoCap approaches provide few quantitative variables [12]. In addition to being costly, they are relatively complex, require careful setup, and are limited to confined areas.

Inertial navigation systems (INS) based on inertial measurement units (IMU), on the other hand, are self-contained and thus can provide unconstrained accessibility to advanced motion and location information for truly ‘ambulatory’ applications. Since its emergence, INS is widely used in various MoCap applications such as ships [13] and aircraft [14]-[15] navigation, fastening tool [16] and pen [17] tracking, in addition to sport analysis [18] to provide navigation information such as attitude, velocity, and position. Recently, with the advances in MEMS technology, MEMS-IMU (typically consisting of a tri-axial gyroscope, a tri-axial accelerometer and a tri-axial magnetometer) has emerged as a wearable MoCap device for human motion tracking in various sports and gaming

applications. Additionally, in health status monitoring applications, the advent of affordable wearable IMU facilitated considerable research to estimate walking speed in an ambulatory fashion [20]. However, due to manufacturing process, the wearable IMU suffer from inherent errors such as bias and scale factor instability which in turn results in accumulated drift error over time when employed for human MoCap [19]. As a result, signal processing techniques including sensor fusion and machine learning have emerged to improve the IMU-based human MoCap. For example, fusion of IMU with global navigation satellite system (GNSS) for outdoor environment and ultra-wideband (UWB) for indoor navigation has received an increased attention recently. Likewise, regression methods such as Gaussian process regression (GPR) have emerged as a preferred methodology for ambulatory walking speed estimation based on a set of inertial features [20].

This Ph.D. thesis works toward alleviating the shortcoming of wearable IMU for ambulatory human motion analysis. The focus of this research is on novel sensor fusion and pattern recognition algorithms based on inertial sensors and absolute localizations technologies to develop a 3-D motion tracking system for true ambulatory applications, specifically aimed at active sports (e.g., skiing), entertainment (e.g., gaming) and health monitoring. In sports and gaming application, this research will bring ambulatory MoCap system to perform realistic MoCap in large indoor/outdoor areas such as soccer fields and ski hills, opening up new opportunities in the area of high performance sports training. For health monitoring applications, this research will provide a basis for longitudinal analysis of human walking speed, which can be used to as an important biomarker to study health status.

1.2. Objectives

The aim of this thesis and my future research is the development of practical, reliable and commercializable algorithms and methods for human motion analysis. The objectives of this dissertation are to:

 To reconstruct 3-D human lower body motion based on segments’ inertial information: An advanced Kalman filter (KF) based fusion algorithm will be

developed to fuse inertial and magnetic information from the IMUs attached to the 7 rigid lower-body segments and estimate the orientation of each body segment. Since human body segments can be treated as a system of rigid links connected by joints, 3-D lower body motion can be reconstructed by knowing the orientation of each body segment.

 To develop an indoor/outdoor localization system: In typical MoCap applications, position tracking which gives access to quantitative KPVs is of key importance. In order to accurately track position of human subjects, INS based on wearable IMU will be developed. This INS will be aided with UWB localization mainly for indoor tracking and GNSS signal for outdoor tracking applications.

 To develop a magnetometer-free localization and lower-body MoCap system: The presence of electromagnetic disturbances in indoor environment affects the accuracy of yaw angle estimation in wearable MoCap system based on IMU. Kinematic constraints of the human body (joints), UWB position measurements from multiple nodes on the body and inertial data from accelerometer and gyroscope will be fused to provide a magnetometer-free estimation of yaw angle and 3-D lower-body motion.

 To develop a walking speed estimation algorithm based on a single body- worn IMU: A regression model will be developed to map the inertial information from ankle, waist and wrist-worn IMU to walking speed. Various inertial variables and feature sets will be studied to develop an accurate walking speed estimation model for each IMU mounting locations.

1.3. Organization of the Thesis

This thesis is comprised of four chapters which are organized as follows:

Chapter 1 provides background and motivation behind the present work. In Chapter 2, a comprehensive literature review is provided on three major aspects of this thesis: (i) systems and methods for capturing motion of human body segments, (ii) human localization in indoor and outdoor settings, and (iii) accurate tracking of walking speed using wearable IMU.

Chapter 3 provides a summary of the main contributions of the thesis. The novel sensor fusion algorithm and its capability for estimating human body segment orientation using a wearable IMU is presented in Sec. 3.1. In Sec. 3.2, the proposed system and method for tracking position and velocity trajectories in indoor and outdoor settings is explained. Sec. 3.3 shows how a biomechanical model of the human body can be used

along with the proposed algorithms in Sec. 3.1 and 3.2 to develop a magnetometer-free lower-body MoCap system, which solves one of the most important challenges in indoor inertial MoCap – the adverse effects of magnetic disturbances. Finally, in Sec. 3.4, the proposed walking estimation method based on a single body-worn IMU is described.

Chapter 4 provides conclusions and future works that may stem from this work.

Chapter 2.

Literature Review

In this chapter, a comprehensive literature review is presented on the following three major aspects of this thesis: MoCap of human body segments, human localization in indoor and outdoor settings, and accurate tracking of walking speed using a wearable IMU.

2.1. MoCap of human body segments

There are a large number of publications on tracking human body segment movement. This section provides a thorough literature survey on the available MoCap devices in addition to some of the conventional and relevant inertial tracking algorithms.

The traditional MoCap devices include optical, magnetic and mechanical systems. The optical MoCap devices are highly accurate, but they are expensive, require clear line of sight and careful setup, and they are limited in range such that their application is restricted to confined areas such as MoCap studios [22]. The magnetic systems are relatively low cost but, similar to optical devices, they are limited in range [23]. They can also be affected by ferromagnetic materials and other sources of magnetic field distortion in surrounding environment, especially in indoors. Mechanical sensors can provide very accurate estimation of body segment orientation free of occlusion but they can be cumbersome to use for monitoring daily activities [23].

Considering the inherent limitations of the above mentioned traditional MoCap devices, recently, the availability of low-cost wearable IMU that can be used to track human movement in and outside of a laboratory, has provided an alternative means to overcome the limitations of other MoCap systems [24].

Using an IMU, 3-D orientation can be obtained by integrating the angular velocities from the tri-axial gyroscope, but this causes unbounded orientation drift due to the gyroscope’s bias instability [2]. In order to compensate for this error, accelerometer and magnetometer are employed as the vertical (the gravity) and horizontal (the Earth’s magnetic field) references, respectively [25]. Several methods have been used so far to estimate orientation using inertial and magnetic data.

Luinge et al. showed that orientation obtained by integrating gyroscope’s rotational rate can be improved by fusing inclination information obtained from accelerometers [26]-[27]. In their system, the tilt angles (orientation around the vertical axis) were obtained by integrating angular rate and gravitational acceleration was used to correct drift. The difference between gyroscope and accelerometer tilt was used as an input to a KF to obtain a more accurate tilt angles [27]. However, the experimental results show that the heading (yaw) angle can still drift over time [27] and the main reason is due to lacking magnetometer.

Roetenberg et al. fused magnetometer with gyroscope and accelerometer in a complementary KF [28]. They introduced a KF that operates on two inputs to estimate 3-D orientation of human body segments. The first input is the difference between inclination obtained from accelerometer and gyroscope. The second input is the difference between the magnetic field measurements from magnetometer and the one that gyroscope estimates. They included gyroscope bias error and orientation error as part of the process model. They also included magnetic disturbances in their model and tried to estimate those disturbances. The filter was tested under static and dynamic conditions with ferromagnetic materials close to the sensor and their results show accurate and drift-free orientation estimates. However, the authors reported that the tracker was tested under controlled and limited conditions, and the accuracy of orientation angle estimates could change with faster and more complex movements. In another study, Roetenberg et al. fused a magnetic system with inertial sensors in which the magnetic system consists of three orthogonal coils, the source, fixed to the body and 3-D magnetic sensors, fixed to remote body segments, which measure the fields generated by the source [29]. Their system shows good position and orientation tracking

accuracies, but the errors were expected to grow if ferromagnetic materials were anywhere close to the magnetic system [29].

More recently, Lee et al. introduced new fusion methods to improve the computational cost of the available algorithms for orientation estimation [25]-[30]. In [25], they used optimal two-observation quaternion estimation method [31] (referred to as O2OQ) in addition to a separate vector selector algorithm in a linear KF algorithm which has the minimum number of states to estimate the full 3-D orientation. The results show that their computationally efficient algorithm will improve the accuracy of the estimated orientation under dynamic conditions and/or magnetic disturbances. In [30], they proposed a fast quaternion-based orientation algorithm which uses conventional Gauss- Newton (G-N) optimization method to remove the orientation drift caused by integration of gyroscope signal. In another subsequent study, they proposed a KF-based algorithm which fuses accelerometer and gyroscope to estimate the tilt angles (i.e. roll and pitch angles) [32]. The proposed algorithm uses an acceleration model-based approach to deal with the estimation problem during dynamic condition. The experimental results show that their proposed algorithm can accurately estimate tilt angles and external accelerations for short accelerated periods; however, for prolonged high external accelerations, the proposed algorithm exhibits gradually increasing errors [32].

2.2. Indoor and outdoor localization

As mentioned earlier, INS based on wearable IMU present a great potential for the purpose of human localization [33]-[34]. However, unlike the tactical grade IMU, these wearable devices suffer from manufacturing imperfections which cause some changes in their characteristics such as bias and scale factor with the change of environmental conditions. Therefore, the inertial navigation solution based on wearable IMU will drift quickly due to the integration of bias over time. The answer to this problem may involve the fusion of an absolute localization system with IMU. This combination has become increasingly popular given certain complimentary characteristics [35]. The absolute localization systems are stable over long periods of time; however, because of the line of sight requirement, their navigation performance can suffer from short-term outages due to occlusion. Conversely, INS is reliable over short time periods, but lacks

long-term stability due to the accumulation of sensor errors. By combining these two technologies, better accuracy is obtained than if either technology is used in isolation. The remainder of this section will provide a literature survey on the available localization systems for indoor and outdoor motion tracking applications in addition to the pertinent fusion algorithms [36].

Waegli et al. compared the performance of two fusion methods for the integration of IMU with low cost global positioning system (GPS) [12] for the purpose of trajectory tracking in sports. These two fusion methods are loosely coupled and tightly coupled algorithms, respectively, based on extended Kalman filter (EKF). In the loosely coupled approach, GPS coordinates and velocities are fed to the filter as measurement updates while in the tightly coupled approach, raw GPS signals (carrier-phase smoothed pseudoranges and Doppler measurements) are fed to the EKF at the update stage. Post processing Rauch-Tung-Striebel (RTS) [37]-[38] smoothing algorithm has also been used for further error reduction. By comparing the estimated trajectory with the one from the reference system consisting of a tactical-grade IMU and a dual frequency GPS receiver, they concluded that the loosely coupled integration strategy provides slightly increased performance over the tightly coupled approach for full or partial satellite constellations [12]. They have also compared the performance of unscented Kalman filter (UKF) with EKF for the same purpose and they found similar accuracies for both methods in sport application but higher computational cost for UKF, which makes it comparatively less interesting [39]. Additionally, Brodie et al. used a combination of GPS, IMU and pressure sensitive insole sensors in a fusion algorithm to capture 3-D kinetics and kinematics of alpine ski racing [40]. More recently, Sadi et al. combined three linear Kalman filters with a complementary EKF to improve the performance of the conventional GPS/IMU fusion based on EKF [5]. They have shown that their method can accurately calculate jump KPVs (key performance variables) with the accuracy of less than 14 cm.

One issue in the above mentioned GPS/IMU fusion approaches is that the consumer grade GPS-derived vertical positional (or altitude) information is much less accurate than the horizontal position [41]. According to [42], GPS altitude measurement accuracies can vary up to 40 m. One potential way of increasing vertical position

information is by using a barometric pressure sensor, which can also derive altitude using barometric pressure measurements. However, height measurements from MEMS barometric pressure sensors suffer from thermal noise and a significant amount of quantization noise [43]. This makes them unsuitable for tracking highly dynamic movements in sports, but may still be used to stabilize the drift-prone IMU for navigation in the vertical direction [43]. The integration of a MEMS barometric pressure sensor with an IMU for applications under dynamics typical for human movements is investigated in [43]-[44].

In addition to outdoor localization, fusion of IMU and an absolute localization system has been used for indoor navigation. Fusion of monocular camera as a vision sensor with a low cost IMU in a vision-aided INS has been thoroughly investigated by Panahandeh et al. in [45]. Another vision-based fused localization system for human tracking based on inertial sensors and wearable monocular camera is introduced in [46]. An indoor tracking system based on inertial sensors, radio frequency (RF) and ultrasound beacons is introduced in [47] and another one based on radio frequency identification (RFID) technology and inertial sensors is proposed in [48]. Among the available radio positioning systems, UWB technology is another promising technology for localization [49].

The original distinguishing feature for the UWB radios is their potential ability to transmit in an unlicensed way with very low power over an ultra-wide portion of the spectrum, allowing this technology to coexist with current and future licensed wireless systems [50]. In fact, UWB technology makes use of sub-nanosecond duration pulses with several GHz of bandwidth which offers the unique possibility of distinguishing the different multipath components, accurate estimation of time of arrival (TOA), and finally accurate estimation of position [50]. In contrast to these advantages, UWB system suffers from non-line-of-sight (NLOS) conditions and often multipath effects which in turn affect the final positioning accuracy. That is why the integration of UWB with IMU has attracted many researchers.

Pittet et al. were among the pioneers who investigated the integration of IMU/UWB for the purpose of indoor human tracking and they reported an accuracy of

about 1m [51]. Hol et al. introduced a tightly coupled fusion algorithm based on EKF to fuse UWB measurements with IMU [52]. This algorithm estimates position as well as orientation of the sensor unit while being reliable in case of multipath effects and NLOS conditions. Corrales et al. investigated the performance of loosely coupled fusion algorithms based on a linear KF, a particle filter and a combination of both for indoor human tracking using UWB/IMU fusion [53]. Their experimental results show that the combined Kalman/particle filter is slightly more accurate that a standalone KF. However, the computational cost of the combined filter is considerably higher than that of the linear KF.

2.3. Walking speed estimation using wearable IMU

Walking speed and its variation over time can be considered as a powerful predictor of hospitalization, disability and survival [54]-[55]. In a clinical setting, different protocols including the 4-meter [56], 10-meter [57], 6-minute walking tests [20] and the timed up and go (TUG) test [58]-[59] have been used as standard tools to evaluate walking speed and gait parameters. However, the short walking tests (e.g., the 10-meter walking test) are subject to bias due to their brevity [60] and the longer tests are less accepted due to the space and time constraints in clinical exams [61]. Additionally, walking speed results of the clinical tests cannot be fully applied to free-living environment [62]. This emphasizes the need for a reliable system/method for longitudinal (and continuous) walking speed measurement in real-world situations.

Aiming at longitudinal walking speed measurement outside the clinical setting, some researchers have used passive infrared (PIR) motion sensors. These PIR sensors can be mounted on ceiling [63] or walls [64] of a residence and can measure the individuals’ walking speed when they are in the field of view of the sensors. However, walking speed measurement based on PIR sensors is limited to confined areas such as hallways. Additionally, such system cannot differentiate between multiple residents, limiting its application to independent-living resident homes. Camera-based systems have also been used in the literature for in-home gait speed measurement [65]. However, camera-based systems can get affected by the lighting conditions and similar

to the PIR sensors, and they are limited to confined areas and hence more suitable for the clinical settings.

Fortunately, with the recent advances in the MEMS sensor technology, wearable IMU can enable walking speed measurement in an ambulatory fashion. Considering that the acceleration data from tri-axial accelerometer in a wearable inertial sensor can be integrated to estimate the velocity, Laudanski et al. have proposed an integration-based approach for speed tracking [21]. However, the main challenge in the integration-based approach is the velocity drift over time that happens as a result of time-varying bias in MEMS-based IMU [66]. To mitigate the drift, Laudanski et al. have proposed the detection of periodic foot stance phases during walking to reset the velocity to zero through a process called zero velocity update (ZUPT) [21],[66]-[69]. However, the need for foot-stance detection requires the wearable sensor to be normally mounted on the leg (ideally on the foot), which is inconvenient for longitudinal walking speed monitoring. Using waist-worn IMU, Hu et al. have modeled the foot swing in walking as an inverted pendulum in order to find a 3-D walking kinematic model for speed estimation [70].

On the other hand, regression models have emerged as a preferred methodology for walking speed estimation based on a set of inertial features. A support vector regression (SVR) model based on acceleration data and a Gaussian process regression (GPR) model based on acceleration and angular velocity data from a waist-mounted IMU are proposed by Schimpl et al. [20] and Vathsangam et al. [71], respectively, to estimate average walking speed. These regression-based approaches for walking speed estimation are based on mapping the inherent pattern of acceleration and rate of turn information corresponding to the hip rotation in a gait cycle to walking speed.

Accuracy of the regression algorithms depends on the set of variables and the extracted features. In inertial walking speed estimation, the variables can be divided into raw variables (the signals collected directly from the inertial sensors) and the processed variables (the signals obtained by manipulation of raw variables using filtering techniques such as sensor fusion). For instance, the external (gravity compensated) acceleration is a processed variable obtained from acceleration and angular velocity using a KF fusion algorithm [2]. Furthermore, the features can be generally divided into

time-domain and frequency-domain features. Due to the periodicity of walking, frequency-domain features of raw inertial variables from a waist-mounted IMU have been extensively used for walking speed estimation [71]. On the other hand, conventional time-domain features (such as minimum, maximum, sum of absolute values, etc.) have also been used in some other studies for the same raw variables in walking speed estimation [20].

Chapter 3.

Summary of Contributions

The main contributions of this Ph.D. thesis are published in five peer-reviewed journal papers and three peer-reviewed conference papers enclosed in Appendices A-H. In this chapter, a summary of each paper is provided followed by a description of the key connections between the papers.

3.1. Estimation of Human Body Segment Orientation

3.1.1. A cascaded two-step KF for estimation of human body segment orientation using wearable IMU

In many ambulatory biomechanical analyses, motion tracking of human body segments by accurate determination of each segment’s orientation is of key importance [30]-[72]. The wearable miniature IMU, together with a sensor fusion algorithm, can be used to estimate accurate orientation of human body segments [25],[30],[73]. To estimate the full 3-D orientation (roll, pitch, and yaw), most researchers have focused on fusing data from the tri-axial accelerometer, gyroscope and magnetometer triplets in a KF together with an optimization algorithm such as O2OQ [25] and G-N [30]. These optimization methods provide the optimal orientation from accelerometer and magnetometer output vector for measurement update step in the KF. In spite of their accuracy, these optimization algorithms have high computational costs.

This work introduces a novel fast two-step cascaded KF for orientation estimation without using optimization. The proposed algorithm uses two linear Kalman filters, consisting of a tilt angle (roll and pitch) KF followed by a yaw (heading) angle KF. The first step uses the accelerometer’s output vector along with an acceleration model to

accurately estimate the tilt angles. The second step extends the tilt angle algorithm to full 3-D orientation by a novel yaw angle estimation method. Using this proposed method, the effect of ferromagnetic disturbances is completely decoupled from the tilt angle estimation. Furthermore, the estimated tilt angles in the first step help to determine the yaw angle in the second step more accurately. Experimental results reveal that the proposed algorithm provides robust orientation estimation in both kinematically and magnetically disturbed conditions.

This orientation tracing method is a building block in the estimation of the position and velocity tracking trajectories described in Sec. 3.2. For additional information, the reader is referred to Appendix A or [2].

3.2. Tracking of Position and Velocity Trajectories

3.2.1. Integration of MEMS inertial and pressure sensors for vertical trajectory determination

INS based on wearable IMU can provide athletes with KPVs allowing them to share quantitative information with coaches, record performance over time, and even get real-time feedback of how they are doing for further improvement. Typically, these systems make use of an IMU and/or an absolute position sensor to capture motion in addition to localization. For outdoor localization, GPS is integrated with the inertial devices [74]. However, poor accuracy (10~20m) of the consumer grade GPS-derived vertical position, directly affects the altitude estimation based on the GPS/IMU integration. Although recent progress in real-time GPS technologies (real-time kinematic GPS or differential GPS) provide higher positional accuracies [75], their prohibitive cost is a limiting factor for the sport consumer electronics market.

For the purpose of accurately determining vertical trajectories in sports, the performance of the wearable IMU integrated with the MEMS barometric pressure sensor is investigated in this work [1]. A cascaded two-step KF consisting of separate orientation and position/velocity subsystems is proposed for this integration. The two- step cascaded KF uses two linear KFs: an orientation KF for determination of tilt angles

followed by a vertical position/velocity KF. The proposed algorithm avoids the use of a magnetometer for attitude determination which makes it robust against electromagnetic disturbances. Additionally, due to its cascaded structure, it avoids the need to propagate additional states, resulting in more computational efficiency needed for small and lightweight battery-powered wearable technologies for sports. Slow human movements in addition to more rapid sport activities such as vertical and step-down jumps can be tracked using the proposed algorithm. The height tracking performance is benchmarked against a reference camera-based motion tracking system and an error analysis is performed. The experimental results show that the vertical trajectory tracking error is less than 28.1 cm. For the determination of jump vertical height/drop, the proposed algorithm has an error of less than 5.8 cm.

For additional information, the reader is referred to Appendix B or [1].

3.2.2. A cascaded Kalman filter-based GPS/MEMS-IMU integration for sports applications

In addition to vertical trajectory tracking explained in Sec. 3.2.1, horizontal position and velocity trajectories are important parameters for quantitative performance analysis of athletes. Although these parameters can be obtained by GPS, the GPS sampling rate is typically too low to detect complex kinematic motions in sports competitions. Additionally, some other factors such as satellite signal attenuation in athletes' environment, and the number of satellites and their positions may also affect the accuracy of GPS position/velocity calculation [76]. Alternatively, an IMU can be integrated with GPS to detect rapid movements and bridge the gap between GPS blockage periods [36]. EKF and UKF are widely used for GPS/IMU integration in sport trajectory determination. However, despite their excellent sensor fusion capabilities in terms of accuracy, the above mentioned nonlinear Kalman filtering methods demand relatively high computational time that is not desirable in low-cost, battery-powered, and lightweight wearable/portable navigation systems worn by athletes. In addition to high computational cost, the available GPS/IMU KF-based fusion approaches rely on GPS observations to correct the otherwise drift prone orientation calculated by the gyroscope [77]. They make use of the fact that errors in the attitude solution of an INS propagate into

errors in velocity. Therefore, errors in attitude can be observed through as independent measurement of velocity under particular motion conditions that include acceleration, which is non-parallel to the velocity vector in the navigation frame [77]. However, the accuracy and speed of attitude correction by this method depends on frequency and magnitude of acceleration maneuvers [67]. Additionally, using GPS position and velocity observations to correct orientation is problematic for long GPS outage periods.

To address the shortcomings of available GPS/IMU fusion algorithms mentioned above, this work [66] introduces a cascaded KF to integrate GPS/IMU for trajectory determination in sports applications. This cascaded KF consists of a separate orientation filter cascaded with a position/velocity filter. Accurate orientation obtained from the orientation KF are fed to a gyroscope error KF and navigation KF to estimate navigation parameters in addition to inertial bias errors. Using cascaded linear Kalman filters result in more computational efficiency by reducing matrix operations. Additionally, the use of a separate orientation filter results in more robust orientation estimation during GPS attenuation/blockage periods. Experimental results for a number of downhill snowboarding runs show that the proposed algorithm is 70% and 80% faster than the two common nonlinear KF-based sensor fusion methods, i.e. the EKF and UKF, respectively without sacrificing the tracking accuracy. Additionally the results show that the proposed method can successfully bridge the frequent GPS outage periods of 5 s in skiing and snowboarding with the horizontal position accuracy of less than 2 m.

For additional information, the reader is referred to Appendix C or [66].

3.2.3. UWB-aided inertial motion capture for lower body 3-D dynamic activity and trajectory tracking

This work [78] extends the application of the method presented in Sec. 3.2.2 to localization in indoor settings. Among the RF-based systems (e.g., RFID, WiFi and Bluetooth) for indoor localization, UWB technology has been used in this work mainly due to its high precision, reasonable cost, large coverage range (about 400 푚2) and robustness against multipath effect [79]-[80]. However, materials like metal and liquid can cause interferences in the UWB radio signals [81]. Additionally, in UWB positioning systems, the UWB-derived vertical position tends to be less accurate than the horizontal

position [52]. One of the major reasons behind this is the geometry of UWB sensors and the fact that they are installed almost on the same horizontal plane [82]. With the limited number of UWB sensors, typically, this geometry helps to cover more horizontal area but suffers from the vertical accuracy. The available UWB/IMU fusion algorithms have only considered the tracking problem in horizontal plane [51],[53]-[83]. Nevertheless, there is a need to accurately track trajectory in vertical direction in activities such as jumping and hopping as well as more complex acrobatic motions. Additionally, the previous works on UWB/IMU fusion have only focused on the tracking accuracies under less dynamic human activities such as walking [52]-[53]. However, one potential benefit that IMU can bring is to improve the tracking accuracy under highly dynamic motions where noisy and low update rate measurements of UWB fail to track dynamics of the motions.

For the purpose of full motion trajectory tracking and MoCap, which include the 3-D tracking of the root joint (i.e. waist) position/velocity trajectory and lower body kinematics under dynamic conditions, the performance of a wearable IMU integrated with an UWB localization system is investigated in this work. A loosely coupled cascaded two-step KF is used to fuse IMU measurements for orientation estimation. The orientation KF is then cascaded with a position/velocity KF for UWB-aided 3-D inertial tracking. The experimental results, which benchmark the system against a reference camera-based motion tracking system, show that the fusion algorithm can significantly improve the 3-D localization accuracy of the UWB system. In fact, under fast dynamic motion, when the UWB system tends to be less accurate due to its low sampling rate, the fused system can significantly improve the localization accuracy of the UWB system by at least 65%. The fusion algorithm shows the accuracies of about 4.2, 3.6 and 4.9 cm in x, y and z directions, respectively. Furthermore, the algorithm can also accurately track joint angles during fast dynamic activities, with the accuracy of about 2.1 degrees in the case of the knee joint.

For additional information, the reader is referred to Appendix D or [78].

3.3. Magnetometer-free Motion Tracking

3.3.1. A magnetometer-free indoor human localization based on loosely coupled IMU/UWB fusion

As mentioned in Sec. 3.2.3, a fused UWB/IMU localization system can provide accurate estimates can be used for accurate 3-D human position estimation. In the existing loosely coupled UWB/IMU fusion algorithms, magnetic data from magnetometer are used to help the estimation of yaw angle and thus the horizontal position and velocity [51],[53]. However, in indoor environments, the Earth’s magnetic field can be easily perturbed by the presence of ferromagnetic objects [4], [84]-[85], necessitating the need for magnetometer-free localization. Although a nonlinear tightly coupled KF-based fusion algorithm for magnetometer-free UWB/IMU integration has previously been proposed in [52], the complexity and computational costs of the nonlinear approach may not be desirable for low cost, battery powered ambulatory analysis in human localization [52].

This work [86] introduces a novel two-step cascaded KF for a loosely coupled magnetometer-free UWB/IMU integration. It investigates the effect of not using magnetometer for indoor human localization and yaw angle estimation. The proposed algorithm uses two linear Kalman filters. At the first step, i.e. the tilt KF which is based on the work explained in Sec. 3.1.1, tilt angles are estimated. At the second step, i.e. the localization KF, the UWB localization system is fused with the IMU’s accelerometer and gyroscope to estimate position, velocity and the yaw angle. Compared to the magnetometer-aided tracking, using the proposed magnetometer-free algorithm, any indoor magnetic disturbances will no longer affect the horizontal position and velocity estimation. However, the experimental results show that about 20 s of motion is required for the yaw angle to converge which is the trade-off for not using the magnetometer. The experimental results show that the proposed method can accurately estimate yaw angle, and the position and velocity solutions match the ones from an optical MoCap system.

For additional information, the reader is referred to Appendix E or [86].

3.3.2. A novel biomechanical model-aided IMU/UWB fusion for magnetometer-free lower body motion capture

This work [87] extends the magnetometer-free location estimation algorithm in Sec. 3.3.1 to include MoCap of lower-body segments as well. In addition to localization, the estimation of 3-D orientation for each body segment is required for 3-D posture tracking in a typical MoCap application. However, the presence of electromagnetic disturbances in indoor environment, highly affects the 3-D orientation estimation for body segment. Although a constant magnetic disturbance can be effectively identified and removed by proper magnetometer calibration [88], indoor magnetic disturbances can be from varying sources and change over time. To deal with these magnetic disturbances, model-based sensor fusion [28], threshold-based switching [89] and vector selector [25] approaches were proposed in the literature. Although these approaches are effective for short periods of time (5-10 푠), they are drift-prone under varying disturbances and during longer periods of time [90].

This work introduces a magnetometer-free algorithm for lower-body MoCap including 3-D localization and posture tracking by fusing inertial sensors with an UWB localization system and a biomechanical model of the human lower-body. Using this novel KF-based fusion algorithm, the UWB localization data aided with the biomechanical model can eliminate the drift in inertial yaw angle estimation of the lower- body segments. This magnetometer-free yaw angle estimation makes the algorithm insensitive to the magnetic disturbances. The algorithm is benchmarked against the optical MoCap system for various indoor activities including walking, jogging, jumping and stairs ascending/descending. The results show that the proposed algorithm outperforms the available magnetometer-aided algorithms in yaw angle tracking under magnetic disturbances. In a uniform magnetic field, the algorithm shows similar accuracies in localization and joint angle tracking compared to the magnetometer-aided methods. The localization accuracy of the proposed method is better than 4.5 cm in a 3-D space and its accuracy for knee angle tracking is about 3.5 and 4.5 degree in low and high dynamic motions, respectively.

For additional information, the reader is referred to Appendix F or [87].

3.4. Regression Model-based Walking Speed Estimation

3.4.1. Experimental evaluation of regression model-based walking speed estimation using lower body-mounted IMU

Using a single body-worn IMU, regression models have been used as a popular methodology for walking speed estimation [20], [71], [91]. Accuracy of the regression algorithms depends on the set of variables (e.g. raw acceleration, gravity compensated acceleration, etc.) and the extracted features (time-domain and frequency-domain features). For each IMU mounting location, the choice of variables and features will affect the ultimate walking speed estimation accuracy based on a regression model. Hypothetically, due to the periodic leg swing during walking, a leg-mounted IMU would be ideal in capturing the periodicity of walking. Thus, frequency-domain features calculated for the inertial variables from a leg-mounted IMU would be better representatives of walking speed compared to the ones from a waist-mounted IMU.

This work [92] provides a comparative experimental study to shed light on the dependency of walking speed estimation accuracy on various variables, features, and regression methods for two IMU mounting locations including waist and ankle. For the variables, external acceleration is compared to raw acceleration from a tri-axial accelerometer. For the features, the effect of using time-domain and frequency-domain features on the ankle and waist inertial data is investigated. For the regression models, the walking speed estimation accuracy of a GPR model is compared to a Lasso regression model. Among the three variables, external acceleration shows the best results. Based on subject trials, a GPR model shows superior performance compared to a Lasso regression model. In terms of features, when using only frequency-domain features, the estimation accuracy for the waist will suffer significantly compared to the case of using the combined time-domain and frequency-domain features. By using both time-domain and frequency-domain features, waist and ankle-mounted sensors result in similar accuracies: 4.4 cm/s for the waist and 4.7 cm/s for the ankle. When using only frequency-domain features, estimation accuracy based on a waist-mounted sensor suffers more compared to the one from ankle.

For additional information, the reader is referred to Appendix G or [92].

3.4.2. Regression model-based walking speed estimation using wrist-worn inertial sensor

For longitudinal health status monitoring, among available state-of-the-art inertial sensing-based wearables, wrist-worn devices are the most user-friendly and compliant that do not limit the freedom of movement and do not require specific dressing style (e.g., wearing a belt in the case of waist-worn sensor). Thus, wrist-worn devices have relatively higher potential to be easily incorporated into daily lifestyle and worn for longer hours. Similar to hip rotation in each gait cycle [20], arm swing motion during walking is a periodic motion pattern that is highly correlated to walking speed: the faster the walking speed, the faster the arm swing motion. However, in walking speed estimation based on regression models, free arm motion necessitates the use of more complex algorithms to manipulate the acceleration and rate of turn information and get a variable that is more representative of the arm swing motion. Although extracting this variable is of high importance (since the accuracy of the regression models depends on the set of chosen variables and the extracted features), it has not been addressed in the existing literature.

By using the arm swing motion in walking, this work [93] proposes a regression model-based method for longitudinal walking speed estimation using a wrist-worn IMU. The arm swing motion is represented by a novel variable called pca-acc, which is highly correlated to walking speed in terms of both temporal and frequency characteristics. To obtain the pca-acc variable, first the tilt angles of the wrist are calculated by applying a KF-based sensor fusion method. Second, principal component analysis (PCA) is employed to find the horizontal acceleration in the direction of arm swing. Experimental results from 15 young subjects showed that using the proposed pca-acc variable will result in significantly better walking speed estimation accuracy compared to the use of raw acceleration variables (푝 <0.01). Using a combined time domain (TD) and frequency domain (FD) features of the pca-acc variable, a generalized GPR model resulted in mean absolute error and standard deviation of about 5.9% and 4.7% respectively. Based on the experimental results, the generalized and subject-specific GPR models tend to perform similar except for slow walking regime (speed < 100 cm/s) where a subject- specific model provided better estimation accuracy. Compared to the generalized least square regression model based on Lasso (LSR-Lasso), the generalized GPR model performed significantly better (푝 <0.01) for wrist-based walking speed estimation.

Experimental results from a 10-minute outdoor walking trial demonstrated the feasibility of using the proposed method for wrist-based walking speed estimation in real-world environment.

For additional information, the reader is referred to Appendix H or [93].

Chapter 4.

Conclusions and Future Work

4.1. Conclusions and Contributions

This thesis was devoted to the development of algorithms for unobtrusive measurement of human body motion and localization using wearable inertial sensors fused with indoor and outdoor localization technologies. The developed algorithms were evaluated through human subjects trials in various environmental settings. The key contributions of this thesis can be summarized as follows:

(i) Development of a sensor fusion algorithm to capture 3-D orientation of human body segments using wearable IMU. This algorithm is based on KF with a cascaded structure resulting in more computational efficiency. At the first step, acceleration and angular velocity data are fused to provide estimation of the tilt angles. At the second step, magnetic field information is fused with the angular velocity and the previously estimated tilt angles to estimate yaw angle. Experimental results revealed that the developed algorithm provides robust orientation estimation in both kinematically and magnetically disturbed conditions.

(ii) Development of a sensor fusion algorithm to capture vertical position and velocity trajectories. This sensor fusion algorithm makes use of the above mentioned orientation estimation method to estimate tilt angles in order to find the gravity- compensated acceleration in the vertical direction. This acceleration data is then fused with barometric pressure information to estimate the vertical position and velocity trajectories. This method can be used to compensate for the poor accuracy of GPS is vertical direction when used for tracking athletes in outdoor sports such as skiing. The experimental results showed that the vertical trajectory tracking error is less than about 28 cm.

(iii) Development of a GPS/IMU integration algorithm for tracking horizontal position and velocity in outdoor sports. The proposed algorithm uses the above mentioned orientation filter, cascaded with a position/velocity filter. By using cascaded linear Kalman filtering, this method avoids the need to propagate additional states, resulting in the covariance propagation to become more computationally efficient for ambulatory human motion tracking. Additionally, the use of this separate

orientation filter helps to retain the orientation accuracy during GPS outages. Results of the field experiments revealed that the proposed algorithm is computationally much faster compared to the available non-linear approaches and demonstrates improved trajectory tracking during GPS outages.

(iv) Development of a UWB/IMU fusion algorithm for simultaneous 3-D trajectory tracking and lower body MoCap under various dynamic activities such as walking and jumping. This method uses wearable inertial sensors fused with UWB localization system using a cascaded KF-based fusion algorithm, which consists of a separate orientation filter cascaded with a position/velocity filter. Additionally, to further improve the joint angle tracking accuracy, anatomical constraints are applied to the knee joint. The results showed that the proposed system can maintain similar accuracies between fast and slow motions in lower body MoCap and 3-D trajectory tracking. The obtained accuracy of the system for 3-D body localization and knee joint angle tracking for fast motions were less than 5 cm and 2.1 degrees, respectively.

(v) Development of a magnetometer free IMU/UWB fusion method for 3-D indoor human localization and lower body MoCap. Using this KF based fusion algorithm, the UWB localization data aided by the biomechanical model can eliminate the drift in inertial yaw angle estimation of the lower-body segments. This magnetometer- free yaw angle estimation makes the algorithm insensitive to the magnetic disturbances. The experimental results showed that this algorithm outperforms the available magnetometer-aided algorithms in yaw angle tracking under magnetic disturbances. In a uniform magnetic field, the algorithm showed similar accuracies in localization and joint angle tracking compared to the magnetometer-aided methods. The localization accuracy of the proposed method is better than 4.5 cm in a 3-D space and its accuracy for knee angle tracking is about 3.5 and 4.5 degrees in low and high dynamic motions, respectively.

(vi) Experimental evaluation of regression model-based walking speed estimation using lower body-mounted IMU. The comparison was based on different sets of variables, features, mounting locations and regression methods. An experimental evaluation was performed on healthy subjects during free walking trials. The results showed better accuracy of GPR compared to Lasso regression. Among the variables, external acceleration tended to provide improved accuracy. By using both time-domain and frequency-domain features, waist and ankle-mounted sensors resulted in similar accuracies: 4.4 cm/s for the waist and 4.7 cm/s for the ankle. When using only frequency-domain features, estimation accuracy based on a waist-mounted sensor suffered more compared to the one from ankle.

(vii) Development of a regression model-based walking speed estimation using a wrist- worn IMU. Time-domain and frequency-domain features of arm swing motion in walking, was used in a regression model for longitudinal walking speed estimation using a wrist-worn IMU. To make the best use of arm swing’s inertial information, a novel kinematic variable was developed that finds the wrist acceleration in the principal axis (i.e. the direction of the arm swing). This variable was obtained by applying sensor fusion on tri-axial accelerometer and gyroscope data to find the tilt

angles followed by the use of PCA. The experimental results showed that using the proposed variable can significantly improve the walking speed estimation accuracy when compared against the use of raw acceleration information. The resulting walking speed estimation mean absolute error and standard deviation were about 5.9% and 4.7%, respectively.

4.2. Future Work

In order to truly realize an ambulatory human motion tracking for sports and health monitoring applications, the work presented in this thesis needs to be progressed further. The future work can be mainly divided into the following three themes.

One of the contributions of this thesis was the development of vertical trajectory tracking algorithm by integration of IMU and MEMS barometric pressure sensor. For evaluation of this algorithm only indoor experimental results were considered which may have some limitations if directly applied to outdoor sports such as skiing and snowboarding. While the lab-based subject trials were suited for the purpose of this thesis, which was the theoretical and proof-of-concept demonstration of the proposed algorithm, the ultimate goal is to expand it for vertical jump trajectory tracking in skiing and snowboarding. In future, field testing should be carried out, in order to generalize the proof-of-concept results to outdoor skiing/snowboarding activities, especially under temperature variations and during high wind or speed conditions when the altitude measurements from the barometric pressure sensor might be disturbed and become unreliable.

A magnetometer-free lower body localization and MoCap system was introduced in this thesis in order to remove the errors caused by indoor magnetic disturbances in indoor environment. Although the ferromagnetic disturbances are mainly caused by the underground electrical equipment that are closer to lower-body (e.g., electrical cables), the ferromagnetic objects around the room may also affect the MoCap accuracy of the upper-body as well. Thus the magnetometer-free lower body MoCap algorithm developed in this thesis should be extended to upper body.

With respect to the wrist-based walking speed estimation, the experimental results presented in this thesis may have some limitations. The proposed method is

based on the assumption of free arm swing during walking. In reality, this assumption would not be satisfied in occasions such as carrying a bag, putting hand in the pocket, walking with walkers, etc. However, while there is no arm swing in these situations, since the arm is now fixed to the trunk, the acceleration profile of the wrist will be similar to that of the trunk. Thus, in future, these situations should be identified using a proper activity classification algorithm and a separate regression model should be trained for walking speed estimation in such cases. Additionally, the true value of walking speed estimation for health monitoring is for older adults when the decline in their walking speed can be considered as an early marker of their neurodegenerative disease. As future work, clinical investigations should be carried out by collecting real-world data from elderly subjects and fine-tune the regression model for longitudinal walking speed estimation in older adults.

References

[1] S. Zihajehzadeh, T. J. Lee, J. K. Lee, R. Hoskinson, and E. J. Park, “Integration of MEMS Inertial and Pressure Sensors for Vertical Trajectory Determination,” IEEE Trans. Instrum. Meas., vol. 64, no. 3, pp. 804–814, 2015.

[2] S. Zihajehzadeh, D. Loh, M. Lee, R. Hoskinson, and E. J. Park, “A Cascaded Two - Step Kalman Filter for Estimation of Human Body Segment Orientation Using MEMS - IMU,” 36th Annu. Int. Conf. IEEE Eng. Med. Biol. Soc., no. 2, pp. 6270– 6273, 2014.

[3] J. Favre, B. M. Jolles, R. Aissaoui, and K. Aminian, “Ambulatory Measurement of 3D Knee Joint Angle,” J. Biomech., vol. 41, no. 5, pp. 1029–35, 2008.

[4] R. Fiducials, S. House, S. Connell, and I. Milligan, “Indoor Localization Using Pedestrian Dead Reckoning Updated with RFID-Based Fiducials,” 33rd Annu. Int. Conf. IEEE Eng. Med. Biol. Soc.,pp. 7598–7601, 2011.

[5] F. Sadi and R. Klukas, “New Jump Trajectory Determination Method Using Low- cost MEMS Sensor Fusion and Augmented Observations for GPS/INS Integration,” GPS Solut., vol. 17, no. 2, pp. 139–152, 2012.

[6] T. J. Lee, S. Zihajehzadeh, D. Loh, R. Hoskinson, and E. J. Park, “Automatic Jump Detection in Skiing/Snowboarding Using Head-mounted MEMS Inertial and Pressure Sensors,” Proc. Inst. Mech. Eng. Part P J. Sport. Eng. Technol., vol. 229, no. 4, pp. 278–287, 2015.

[7] L. Bai, M. Pepper, Y. Yan, S. Spurgeon, M. Sakel, and M. Phillips, “Quantitative Assessment of Upper Limb Motion in Neurorehabilitation Utilizing Inertial Sensors.,” IEEE Trans. Neural Syst. Rehabil. Eng., vol. 4320, no. c, 2014.

[8] A. Akl, B. Taati, and A. Mihailidis, “Autonomous Unobtrusive Detection of Mild Cognitive Impairment in Older Adults,” IEEE Trans. Biomed. Eng., vol. 62, no. 5, pp. 1383–1394, 2015.

[9] C. Heesen, J. Böhm, C. Reich, J. Kasper, M. Goebel, and S. M. Gold, “Patient Perception of Bodily Functions in Multiple Sclerosis: Gait and Visual Function are the Most Valuable,” Mult Scler, vol. 14, pp. 988–991, 2008.

[10] J. E. Graham, G. V. Ostir, S. R. Fisher, and K. J. Ottenbacher, “Assessing Walking Speed in Clinical Research: a Systematic Review,” J. Eval. Clin. Pract., vol. 14, no. 4, pp. 552–562, 2008.

[11] I. Bautmans, B. Jansen, B. Van Keymolen, and T. Mets, “Reliability and Clinical

Correlates of 3D-Accelerometry Based Gait Analysis Outcomes According to Age and Fall-risk,” Gait Posture, vol. 33, no. 3, pp. 366–372, 2011.

[12] A. Waegli and J. Skaloud, “Optimization of Two GPS/MEMS-IMU Integration Strategies with Application to Sports,” GPS Solut., vol. 13, no. 4, pp. 315–326, 2009.

[13] B. Wang, Z. Deng, C. Liu, Y. Xia, and M. Fu, “Estimation of Information Sharing Error by Dynamic Deformation Between Inertial Navigation Systems,” IEEE Trans. Ind. Electron., vol. 61, no. 4, pp. 2015–2023, 2014.

[14] K. D. Sebesta and N. Boizot, “A Real-Time Adaptive High-Gain EKF, Applied to a Quadcopter Inertial Navigation System,” IEEE Trans. Ind. Electron., vol. 61, no. 1, pp. 495–503, 2014.

[15] L. Chen and J. Fang, “A Hybrid Prediction Method for Bridging GPS Outages in High-Precision POS Application,” IEEE Trans. Instrum. Meas., vol. 63, no. 6, pp. 1656–1665, 2014.

[16] S. P. Won, F. Golnaraghi, and W. W. Melek, “A Fastening Tool Tracking System Using an IMU and a Position Sensor With Kalman Filters and a Fuzzy Expert System,” IEEE Trans. Ind. Electron., vol. 56, no. 5, pp. 1782–1792, 2009.

[17] J. S. Wang, Y. L. Hsu, and J. N. Liu, “An Inertial-Measurement-Unit-Based Pen With a Trajectory Reconstruction Algorithm and Its Applications,” IEEE Trans. Ind. Electron., vol. 57, no. 10, pp. 3508–3521, 2010.

[18] C. N. K. Nam, H. J. Kang, and Y. S. Suh, “Golf Swing Motion Tracking Using Inertial Sensors and a Stereo Camera,” IEEE Trans. Instrum. Meas., vol. 63, no. 4, pp. 943–952, 2014.

[19] X. Meng, Z. Q. Zhang, J. K. Wu, W. C. Wong, and H. Yu, “Self-contained Pedestrian Tracking During Normal Walking Using an Inertial/Magnetic Sensor Module.,” IEEE Trans. Biomed. Eng., vol. 61, no. 3, pp. 892–9, 2014.

[20] M. Schimpl, C. Lederer, and M. Daumer, “Development and Validation of a New Method to Measure Walking Speed in Free-living Environments Using the Actibelt® Platform.,” PLoS One, vol. 6, no. 8, p. e23080, 2011.

[21] A. Laudanski, S. Yang, and Q. Li, “A Concurrent Comparison of Inertia Sensor- based Walking Speed Estimation Methods,” 33rd Annu. Int. Conf. IEEE Eng. Med. Biol. Soc., pp. 3484–3487, 2011.

[22] G. Welch, “Motion Tracking : No Silver Bullet , but a Respectable Arsenal,” IEEE Comp. Graph. and Applic., vol. 22, no. 6, 2002.

[23] M. A. El-gohary and J. Mcnames, “Joint Angle Tracking with Inertial Sensors,” Ph.D. Dissertation, Portland State University, 2013.

[24] C. H. Wu and Y. C. Tseng, “Deploying Sensors for Gravity Measurement in a Body-Area Inertial Sensor Network,” IEEE Sens. J., vol. 13, no. 5, pp. 1522–1533, 2013.

[25] J.K. Lee, and E. J. Park, “Minimum-Order Kalman Filter With Vector Selector for Accurate Estimation of Human Body Orientation,” IEEE Trans. Robot., vol. 25, no. 5, pp. 1196–1201, 2009.

[26] H. J. Luinge, P. H. Veltink, and C. T. M. Baten, “Ambulatory Measurement of Arm Orientation,” J. Biomech., vol. 40, no. 1, pp. 78–85, 2007.

[27] H. J. Luinge and P. H. Veltink, “Inclination Measurement of Human Movement Using a 3-D Accelerometer with Autocalibration.,” IEEE Trans. Neural Syst. Rehabil. Eng., vol. 12, no. 1, pp. 112–21, 2004.

[28] D. Roetenberg, H. J. Luinge, C. T. M. Baten, and P. H. Veltink, “Compensation of Magnetic Disturbances Improves Inertial and Magnetic Sensing of Human Body Segment Orientation.,” IEEE Trans. Neural Syst. Rehabil. Eng., vol. 13, no. 3, pp. 395–405, 2005.

[29] D. Roetenberg, P. J. Slycke, and P. H. Veltink, “Ambulatory Position and Orientation Tracking Fusing Magnetic and Inertial Sensing.,” IEEE Trans. Biomed. Eng., vol. 54, no. 5, pp. 883–90, 2007.

[30] J. K. Lee and E. J. Park, “A Fast Quaternion-based Orientation Optimizer via Virtual Rotation for Human Motion Tracking.,” IEEE Trans. Biomed. Eng., vol. 56, no. 5, pp. 1574–82, 2009.

[31] F. L. Markley, “Fast Quaternion Attitude Estimation from two Vector Measurements,” J. Guid. Control. Dyn., vol. 25, no. 2, pp. 411–414, 2002.

[32] J. K. Lee, E. Park, and S. Robinovitch, “Estimation of Attitude and External Acceleration Using Inertial Sensor Measurement During Various Dynamic Conditions,” IEEE Trans. Instrum. Meas., vol. 61, no. 8, pp. 2262–2273, 2012.

[33] P. K. Yoon, S. Zihajehzadeh, and E. J. Park, “Adaptive Kalman Filter for Indoor Localization Using Bluetooth Low Energy and Inertial Measurement Unit,” 37th Annu. Int. Conf. IEEE Eng. Med. Biol. Soc., pp. 825–828, 2015.

[34] D. Loh, S. Zihajehzadeh, R. Hoskinson, H. Abdollahi and E. J. Park, “Pedestrian Dead Reckoning With Smartglasses and Smartwatch,” IEEE Sensors, vol. 16, no. 22, pp. 8132–8141, 2016.

[35] S. P. Kwakkel, “Human Lower Limb Kinematics Using GPS/INS,” MASc Thesis, University of Calgary, 2008.

[36] L. Li, Y. Pan, J. Lee, C. Ren, Y. Liu, D. A. Grejner-brzezinska, and C. K. Toth, “Cart-Mounted Geolocation System for Unexploded Ordnance With Adaptive ZUPT Assistance,” IEEE Trans. Instrum. Meas., vol. 61, no. 4, pp. 974–979, 2012.

[37] D. Simon, Optimal State Estimation. Wiley, 2006.

[38] H. Liu, S. Nassar, and N. El-Sheimy, “Two-Filter Smoothing for Accurate INS/GPS Land-Vehicle Navigation in Urban Centers,” IEEE Trans. Veh. Technol., vol. 59, no. 9, pp. 4256–4267, 2010.

[39] A. Wägli, “Trajectory Determination and Analysis in Sports by Satellite and Inertial Navigation,” Ph.D. Dissertation, Swiss Federal Institute of Technology, 2009.

[40] M. Brodie, A. Walmsley, and W. Page, “Fusion Motion Capture: a Prototype System Using Inertial Measurement Units and GPS for the Biomechanical Analysis of Ski Racing,” Sport. Technol., vol. 1, no. 1, pp. 17–28, 2008.

[41] R. M. Otakar Svabensky, Josef Weigel, “On GPS Heighting In Local Networks,” Acta Geodyn. Geomater., vol. 3, no. 143, pp. 39–43, 2006.

[42] J. Zhang, E. Edwan, J. Zhou, W. Chai, and O. Loffeld, “Performance Investigation of Barometer Aided GPS/MEMS-IMU Integration,” Proc. 2012 IEEE/ION Position, Locat. Navig. Symp., pp. 598–604, 2012.

[43] M. Tanigawa, H. Luinge, L. Schipper, AND P. Slycke, “Drift-Free Dynamic Height Sensor using MEMS IMU Aided by MEMS Pressure Sensor,” in 5th Workshop on Positioning, Navigation and Communication, 2008.

[44] M. Imu, P. D. Groves, G. W. Pulford, C. J. Mather, C. A. Littlefield, D. L. J. Nash, and M. R. Carter, “Integrated Pedestrian Navigation Using GNSS, MEMS IMU, Magnetometer and Baro-altimeter,” In Proc. NAV7, 2007.

[45] G. Panahandeh, “Selected Topics in Inertial and Visual Sensor Fusion : Calibration , Observability Analysis , and Applications,” Ph.D. Dissertation, KTH, Sweden, 2014.

[46] Y. Tian, W. R. Hamel, and J. Tan, “Accurate Human Navigation Using Wearable Monocular Visual and Inertial Sensors,” IEEE Trans. Instrum. Meas., vol. 63, no. 1, pp. 203–213, 2014.

[47] G. Pirkl, D. Munaretto, C. Fischer, C. An, P. Lukowicz, M. Klepal, A. Timm-Giel, J. Widmer, D. Pesch, and H. Gellersen, “Virtual lifeline: Multimodal sensor data fusion for robust navigation in unknown environments,” Pervasive Mob. Comput., vol. 8, no. 3, pp. 388–401, 2012.

[48] A. R. Jimenez Ruiz, F. Seco Granja, J. C. Prieto Honorato, and J. I. Guevara Rosas, “Accurate Pedestrian Indoor Navigation by Tightly Coupling Foot-Mounted IMU and RFID Measurements,” IEEE Trans. Instrum. Meas., vol. 61, no. 1, pp. 178–189, 2012.

[49] Y. Rahayu, T. A. Rahman, R. Ngah, and P. S. Hall, “Ultra Wideband Technology and Its Applications,” 5th IFIP Int. Conf. Wirel. Opt. Commun. Networks (WOCN ’08), pp. 1–5, 2008.

[50] G. Bellusci, “Ultra-Wideband Ranging for Low-Complexity Indoor Positioning Applications,” Ph.D Dissertation, Delft University of Technology, 2011.

[51] S. Pittet, V. Renaudin, B. Merminod, and M. Kasser, “UWB and MEMS Based Indoor Navigation,” J. Navig., vol. 61, no. 3, pp. 369–384, 2008.

[52] J. D. Hol, F. Dijkstra, H. Luinge, T. B. Schön, and T. B. Sch, “Tightly Coupled UWB/IMU Pose Estimation,” IEEE Int. Conf. Ultra-Wideband, pp. 688–692, 2009.

[53] J. a. a. Corrales, F. a. a. Candelas, and F. Torres, “Sensor Data Integration for Indoor Human Tracking,” Rob. Auton. Syst., vol. 58, no. 8, pp. 931–939, 2010.

[54] S. Fritz and M. Lusardi, “Walking Speed: the Sixth Vital Sign,” J. Geriatr. Phys. Ther., vol. 32, no. 2, pp. 1–5, 2009.

[55] S. Studenski, S. Perera, K. Patel, C. Rosano, K. Faulkner, M. Inzitari, J. Brach, J. Chandler, P. Cawthon, E. B. Connor, M. Nevitt, M. Visser, S. Kritchevsky, S. Badinelli, T. Harris, A. B. Newman, J. Cauley, L. Ferrucci, and J. Guralnik, “Gait Speed and Survival in Older Adults.,” JAMA, vol. 305, no. 1, pp. 50–58, 2011.

[56] M. Maggio, G. P. Ceda, A. Ticinesi, F. De Vita, G. Gelmini, C. Costantino, T. Meschi, R. W. Kressig, M. Cesari, M. Fabi, and F. Lauretani, “Instrumental and Non-Instrumental Evaluation of 4-Meter Walking Speed in Older Individuals,” PLoS One, vol. 11, no. 4, p. e0153583, 2016.

[57] Y. Abe, A. Matsunaga, R. Matsuzawa, T. Kutsuna, S. Yamamoto, K. Yoneki, M. Harada, R. Ishikawa, T. Watanabe, and A. Yoshida, “Determinants of Slow Walking Speed in Ambulatory Patients Undergoing Maintenance Hemodialysis,” PLoS One, vol. 11, no. 3, p. e0151037, 2016.

[58] D. A. Robertson, G. M. Savva, B. L. King-Kallimanis, and R. A. Kenny, “Negative Perceptions of Aging and Decline in Walking Speed: A Self-fulfilling Prophecy,” PLoS One, vol. 10, no. 4, P. e0123260, 2015.

[59] B. R. Greene and R. A. Kenny, “Assessment of Cognitive Decline Through Quantitative Analysis of the Timed Up and Go Test,” IEEE Trans. Biomed. Eng., vol. 59, no. 4, pp. 988–995, 2012.

[60] C. Vaney, H. Blaurock, B. Gattlen, and C. Meisels, “Assessing Mobility in Multiple Sclerosis Using the Rivermead Mobility Index and Gait Speed,” Clin. Rehabil., vol. 10, no. 3, pp. 216–226, 1996.

[61] P. L. Enright, M. A. McBurnie, V. Bittner, R. P. Tracy, R. McNamara, A. Arnold, and A. B. Newman, “The 6-min Walk Test A Quick Measure of Functional Status in Elderly Adults,” CHEST J., vol. 123, no. 2, pp. 387–398, 2003.

[62] M. Schimpl, C. Moore, C. Lederer, A. Neuhaus, J. Sambrook, J. Danesh, W. Ouwehand, and M. Daumer, “Association between Walking Speed and Age in Healthy, Free-Living Individuals Using Mobile Accelerometry—A Cross-Sectional Study,” PLoS One, vol. 6, no. 8, p. e23299, 2011.

[63] S. Hagler, D. Austin, T. L. Hayes, J. Kaye, and M. Pavel, “Unobtrusive and Ubiquitous In-Home Monitoring: a Methodology for Continuous Assessment of Gait Velocity in Elders.,” IEEE Trans. Biomed. Eng., vol. 57, no. 4, pp. 813–20, 2010.

[64] P. G. Jacobs, E. A. Wan, E. Schafermeer, F. Adenwala, A. S. Paul, N. Preiser and J. Kaye, “Measuring In-Home Walking Speed using Wall-Mounted RF Transceiver Arrays,” vol. 18, no. 9, pp. 914-917, 2014.

[65] F. Wang, E. Stone, M. Skubic, J. M. Keller, C. Abbott, and M. Rantz, “Toward a Passive Low-Cost In-Home Gait Assessment System for Older Adults,” IEEE J. Biomed. Heal. Informatics, vol. 17, no. 2, pp. 346–355, 2013.

[66] S. Zihajehzadeh, D. Loh, T. J. Lee, R. Hoskinson, and E. J. Park, “A Cascaded Kalman Filter-Based GPS/MEMS-IMU Integration for Sports Applications,” Measurement, vol. 73, pp. 200–210, 2015.

[67] E. Foxlin, “Pedestrian Tracking with Shoe-Mounted Inertial Sensors,” IEEE Comp. Graph. and Applic., VOL. 25, no. 6, pp. 38–46, 2005.

[68] X. Meng, Z. Q. Zhang, J. K. Wu, and W. C. Wong, “Hierarchical Information Fusion for Global Displacement Estimation in Microsensor Motion Capture.,” IEEE Trans. Biomed. Eng., vol. 60, no. 7, pp. 2052–63, 2013.

[69] I. Skog, H. Peter, J. Nilsson, and J. Rantakokko, “Zero-Velocity Detection — An Algorithm Evaluation,” IEEE Trans. Biomed. Eng., vol. 57, no. 11, pp. 2657–2666, 2010.

[70] J. S. Hu, K. C. Sun, and C. Y. Cheng, “A Kinematic Human-Walking Model for the Normal-Gait-Speed Estimation Using Tri-Axial Acceleration Signals at Waist Location,” IEEE Trans. Biomed. Eng., vol. 60, no. 8, pp. 2271–2279, 2013.

[71] H. Vathsangam, B. A. Emken, D. Spruijt-Metz, and G. S. Sukhatme, “Toward Free-Living Walking Speed Estimation Using Gaussian Process-based Regression with On-Body Accelerometers and Gyroscopes,” 4th Int. Conf. on Pervasive Comput. Technol. Healthc, 2010.

[72] H. P. Bruckner, C. Spindeldreier, and H. Blume, “Modification and Fixed-Point Analysis of a Kalman Filter for Orientation Estimation Based on 9D Inertial Measurement Unit Data.,” Conf. Proc. IEEE Eng. Med. Biol. Soc., pp. 3953–6, 2013.

[73] Y. Teruyama and T. Watanabe, “A Basic Study on Variable-Gain Kalman Filter Based on Angle Error Calculated from Acceleration Signals for Lower Limb Angle Measurement with Inertial Sensors.,” Conf. Proc. IEEE Eng. Med. Biol. Soc., pp. 3423–6, Jan. 2013.

[74] H. Kuusniemi, J. Liu, L. Pei, Y. Chen, L. Chen, and R. Chen, “Reliability Considerations of Multi-Sensor Multi-Network Pedestrian Navigation,” IET Radar, Sonar Navig., vol. 6, no. 3, p. 157, 2012.

[75] F. Aghili, and A. Salerno, “Driftless 3-D Attitude Determination and Positioning of Mobile Robots By Integration of IMU With Two RTK GPSs,” IEEE/ASME Trans. Mechatronics, vol. 18, no. 1, pp. 21–31, 2013.

[76] P. D.Groves, Principles of GNSS, Inertial, and Multisensor Integrated. Artech House Publishers, 2008.

[77] D. Dusha and L. Mejias, “Error Analysis and Attitude Observability of a Monocular GPS/Visual Odometry Integrated Navigation Filter,” Int. J. Rob. Res., vol. 31, no.

6, pp. 714–737, 2012.

[78] S. Zihajehzadeh, P. K. Yoon, B. Kang, and E. J. Park, “UWB-Aided Inertial Motion Capture for Lower Body 3-D Dynamic Activity and Trajectory Tracking,” IEEE Trans. Instrum. Meas., vol. 64, no. 12, pp. 3577-87, 2015.

[79] Y. Gu, A. Lo, and I. Niemegeers, “A survey of indoor positioning systems for wireless personal networks,” IEEE Commun. Surv. Tutorials, vol. 11, no. 1, pp. 13–32, 2009.

[80] D. Morche, G. Masson, S. De Rivaz, F. Dehmas, S. Paquelet, A. Bisiaux, O. Fourquin, J. Gaubert, and S. Bourdel, “Double-Quadrature UWB Receiver for Wide-Range Localization Applications with Cub-cm Ranging Precision,” IEEE J. Solid-State Circuits, vol. 48, no. 10, pp. 2351–2362, 2013.

[81] G. Deak, K. Curran, and J. Condell, “A Survey of Active and Passive Indoor Localisation Systems,” Comput. Commun., vol. 35, no. 16, pp. 1939–1954, 2012.

[82] P. S. M. Tanigawa, J.D. Hol, F. Dijkstra, H. Luinge, and P. Slycke “Augmentation of Low-Cost GPS/MEMS INS with UWB Positioning System for Seamless Outdoor/Indoor positioning,” in Proc. 21 st International Technical Meeting of the Satellite Division of the Institute of Navigation, pp. 1804-1811, 2008.

[83] A. Benini, A. Mancini, and S. Longhi, “An IMU/UWB/Vision-based Extended Kalman Filter for Mini-UAV Localization in Indoor Environment using 802.15.4a Wireless Sensor Network,” J. Intell. Robot. Syst., vol. 70, no. 1–4, pp. 461–476, 2013.

[84] Y. Zhang, J. Tan, Z. Zeng, W. Liang, and Y. Xia, “Monocular camera and IMU integration for indoor position estimation,” 36th Annu. Int. Conf. IEEE Eng. Med. Biol. Soc., pp. 1198–1201, 2014.

[85] T. H. Riehle, S. M. Anderson, P. A. Lichter, N. A. Giudice, S. I. Sheikh, R. J. Knuesel, D. T. Kollmann, D. S. Hedin, and A. P. Hardware, “Indoor Magnetic Navigation for the Blind,” 34th Annu. Int. Conf. IEEE Eng. Med. Biol. Soc., pp. 1972–1975, 2012.

[86] S. Zihajehzadeh, P. K. Yoon, and E. J. Park, “A Magnetometer-Free Indoor Human Localization Based on Loosely Coupled IMU/UWB Fusion,” Proc. Annu. Int. Conf. IEEE Eng. Med. Biol. Soc., pp. 3141–3144, 2015.

[87] S. Zihajehzadeh, AND E. J. Park, “A Novel Biomechanical Model-Aided IMU / UWB Fusion for Magnetometer-Free Lower Body Motion Capture,” IEEE Trans. Syst., Man, and Cyber. Syst., doi: 10.1109/TSMC.2016.2521823, 2016.

[88] S. A. H. Tabatabaei, A. Gluhak, and R. Tafazolli, “A Fast Calibration Method for Triaxial Magnetometers,” IEEE Trans. Instrum. Meas., vol. 62, no. 11, pp. 2929– 2937, 2013.

[89] A. M. Sabatini, “Quaternion-Based Extended Kalman Filter for Determining Orientation by Inertial and Magnetic Sensing.,” IEEE Trans. Biomed. Eng., vol. 53,

no. 7, pp. 1346–56, Jul. 2006.

[90] M. El-Gohary, and J. McNames, “Human Joint Angle Estimation with Inertial Sensors and Validation with A Robot Arm,” IEEE Trans. Biomed. Eng., vol. 62, no. 7, 2015.

[91] F. Benedikt, F. Dadashi, and K. Aminian, “Instantaneous Walking Speed Estimation for Daily Life Activity Monitoring Based on Wrist Acceleration,” in 4th International Conference on Ambulatory Monitoring of Physical Activity and Movement (ICAMPAM), 2015.

[92] S. Zihajehzadeh, and E. J. Park, “Experimental Evaluation of Regression Model - Based Walking Speed Estimation Using Lower Body - Mounted IMU,” Proc. Annu. Int. Conf. IEEE Eng. Med. Biol. Soc., pp. 243–246, 2016.

[93] S. Zihajehzadeh and E. J. Park, “Regression Model-Based Walking Speed Estimation Using Wrist-Worn Inertial Sensor,” PLoS One, vol. 11, no. 10, p. e0165211, 2016.

Appendix A.

A cascaded two-step Kalman filter for estimation of human body segment orientation using MEMS-IMU

(Reproduced with permission from IEEE)

35 - -

S. Zihajehzadeh, D. Loh, M. Lee, R. Hoskinson, and E.J. Park1, Senior Member, IEEE

Abstract! Orientation of human body segments is an and G-N [1], provide the optimal orientation from important quantity in many biomechanical analyses. To get accelerometer and magnetometer output vector for robust and drift-free 3-D orientation, raw data from miniature measurement update step in the Kalman filter. To use these body worn MEMS-based inertial measurement units (IMU) optimization methods, it should be assumed that constant should be blended in a Kalman filter. Aiming at less reference vector measurements (i.e. the gravity from the computational cost, this work presents a novel cascaded two- accelerometer and Earth magnetic field from the step Kalman filter orientation estimation algorithm. Tilt angles are estimated in the first step of the proposed cascaded Kalman magnetometer) are available, which cannot be guaranteed filter. The estimated tilt angles are passed to the second step of [5]. To deal with this problem, most authors use a threshold- the filter for yaw angle calculation. The orientation results are based switching approach [7] or a vector selector algorithm benchmarked against the ones from a highly accurate tactical [5] to ignore the perturbed acceleration and/or magnetic field grade IMU. Experimental results reveal that the proposed measurement in the Kalman filter. In spite of their accuracy, algorithm provides robust orientation estimation in both these optimization algorithms have high computational kinematically and magnetically disturbed conditions. costs. On the other hand, without employing such I. INTRODUCTION optimization, another method to estimate orientation in kinematically and/or magnetically disturbed conditions is to In many ambulatory biomechanical analyses, motion include acceleration model and magnetic field model in the tracking of human body segments by accurate determination orientation estimation Kalman filter. One complementary -% $ "' 0$&+$,1<0 -/($,1 1(-, (0 -% key importance [1]-[2]. Kalman filter approach is presented in [8] for 2-D The diverse application of body segment motion tracking orientation estimation and is extended to 3-D orientation in ranges from rehabilitation and physical medicine to sports [9]. science [3]-[4]. Recently, with the advances in MEMS This paper introduces a novel fast two-step cascaded technology, miniature inertial measurement units (IMU) Kalman filter for orientation estimation without using have emerged for wearable motion capture technology. optimization. The proposed algorithm uses two linear These wearable miniature IMU, which consist of Kalman filters, consisting of a tilt angle (roll and pitch) accelerometers, gyroscopes and magnetometers, together Kalman filter followed by a yaw (heading) angle Kalman with a sensor fusion algorithm, can be used to estimate filter. The first step is based on our previous algorithm in accurate orientation of human body segments to monitor a [10], which uses the ""$*$/-+$1$/0< -21.21 3$"1-/ *-,& .$/0-,<0.'60(" * ctivities in daily life environment over an with an acceleration model to accurately estimate the tilt extended time period [5]-[6]. A 3-D orientation estimate can angles. The second step extends our tilt angle algorithm to be obtained by integrating the angular velocities from the tri- full 3-D orientation by a novel yaw angle estimation method. axial gyroscope, but, this causes unbounded orientation drift Using this proposed method, the effect of ferromagnetic due to the &6/-0"-.$<0-21put noise. In order to compensate disturbances is completely decoupled from the tilt angle for this error, accelerometer and magnetometer are estimation. Furthermore, the estimated tilt angles in the first employed as the vertical (the gravity) and horizontal (the step help to determine the yaw angle in the second step more /1'<0+ &,$1("%($*#/$%$/$,"$0/$0.$"1(3$*6 [5]. accurately. To estimate the full 3-D orientation (roll, pitch, and yaw), II. METHOD most researchers have focused on fusing data from the above A. Problem Definition sensor triplets in a Kalman filter together with an The orientation of the sensor frame (the frame fixed to optimization algorithm. The widely used optimization sensor) with respect to an inertial frame (the frame pointing algorithms for this purpose, such as QUEST [4], O2OQ [5] to local East, North and Up directions) can be represented by a rotation matrix which maps a vector from the sensor frame This work is supported by the Natural Sciences and Engineering to the inertial frame: Research Council of Canada (NSERC) and the experimental protocol (No, Q' xQ "'S (1) 2013s0790) is approved by the Office of Research Ethics of Simon Fraser H S H University. where ' is an arbitrary w vector and left superscripts S. Zihajehzadeh, D. Loh, M. Lee and E.J. Park are with the School of Q and represent the inertial and sensor frame respectively. S" Mechatronic Systems Engineering, Simon Fraser University, 250-13450 102nd Avenue, Surrey, BC, Canada, V3T 0A3 (email: [email protected], is the w rotation matrix expressed as [10]: [email protected], [email protected], [email protected]) / v 9 /<9=/> u 9<9> R. Hoskinson is with Recon Instruments Inc., 1050 Homer Street, Q (2) Vancouver, BC, Canada, V6B 2W9 (email: S" x|9 u / 9<9=/> v /<9>} [email protected]) v9= /=9> /=/>

978-1-4244-7929-0/14/$26.00 ©2014 IEEE 6270 36 S M where / and 9 are abbreviations for /79 and 946 matrix of 'H; H#Z is the external acceleration error in the respectively; < (yaw), = (pitch) and > (roll) are the rotation sensor frame; $P and $O are the gyroscope and angles about the .-, --, and ,- axes of the inertial frame accelerometer measurement noise vectors, which are respectively. Note that the last row of the rotation matrix is assumed to be uncorrelated and zero-mean white Gaussian; the unit gravity vector expressed in the sensor frame and is the superscripts u and 01 ,#%-/1'$8a posteriori: ,#1'$ independent from yaw angle [10]. By estimating this row, 8a priori:$01(+ 1$0(,1'$ *+ ,%(*1$//$0.$"1(3$*6/_ is a roll and pitch (tilt) angles can be determined: dimensionless constant between 0 and 1 that determines the cut-off frequency in the external acceleration model o" M"o >x MH ~p nm , =x MH ~ p nl (3) (S# x/ # vS uE , with E being the time- o" o" zXVW k H _ H p nn p nm varying error of the external acceleration process model); th th where Q" represents the i row and j column in rotation S L S ab H#H v is the gravity compensated external acceleration; Q matrix S". Additionally, by estimating the first row in and is the norm of gravity vector. rotation matrix and having the tilt angles, yaw angle (<) can The measurement vector in Eq. (6) is calculated by [10]: be readily determined by: ) x( v / #S L v (13) Q Q H O _ H H v/>S "HI u9>"S HJ S L L ) axial gyroscopes and tri-axial accelerometers are used in a and pitch (=) angles are calculated using (3): Kalman filter to estimate the normalized gravity vector in Y MY the sensor frame (i.e., the third row of the rotation matrix > x MH ~ ls , = x MH ~ lr (17) Q Ylt YlsXVW k S") using the following system model equations [10]: As illustrated in Fig. 1, these two tilt angles are the inputs to 'H xH v 'H v u&H v (5) the second step, i.e. the yaw Kalman filter. )H x H 'H u%H (6) B. Yaw Kalman Filter In Eqs. (5) and (6), ' x v9=/=9>/=/>^ is the H The yaw angle Kalman filter allows accurate w state vetor for tilt angles Kalman filter at step ; H is the state transition matrix and & is the process model noise determination of yaw angle under temporary ferromagnetic H disturbances. In this Kalman filter, the measurements from vector)H is the measurement vector (i.e. the normalized measured gravity vector in the sensor frame) is the w tri-axial gyroscopes and tri-axial magnetometers and the H known tilt angles are used to estimate the first row of the observation matrix; and %H is the measurement model noise Q vector. The matrices in (5) and (6) can be calculated using rotation matrix S". The system model equations are the following equations [10]: represented by: ' x v' v u& v (18) H v x v(P v (7) I I I I ) x ' u% (19) &H v x v'H $P (8) I I I I where ' x / v 9/<9=/> u 9<9>^ is H x J (9) I S M the w state vetor for heading angle Kalman filter at step %H xvH #Z u$O (10) S M S M S and I, &I and I can be calculated as: H#Z x#H H v# H (11) S M S L I v x v(P v (20) H#H x/_ H#H v (12) &I v x v'I $P (21) where (P is the w skew symmetric matrix of tri-axial I x J (22) gyroscope measurements'H is the w skew symmetric 6271 37 S L v Gyroscope Tilt Yaw Kalman Kalman AccelerometerAccele rometer filter filter

S L HH I MagnetometerM a gnetometer ValidationVal idation TestTest

Fig. 1. Overview of the proposed algorithm structure Fig. 2. Experimental setup: Xsens MTi-G-700 attached to reference IMU-FSAS I To calculate the measurement vecor )I , roll, pitch and where A] is the magnetometer noise variance in each axis. @àe yaw angles at each step are required. However, at each is the current dip angle (i.e. the angle formed by the gravity step, roll and pitch are already known from the tilt Kalman vector and current magnetic field vector) and @àe is its initial filter. Since these tilt angles are used in calculation of )I value. ?] and ?àe are the threshold values. for the measurement update of yaw Kalman filter, they are Finally, !I is defined similar to !H : " **$#;+$ 02/$#1(*1 ,&*$0< ,##$,-1$#by =d and >d. To I !I v xv 'I v DP'I v (27) &$11'$;+$ 02/$#6 4 ,&*$< u9> S kqjq < x MH Ih Ii (28) Q Ig/= "k j x S q q The structure of the proposed two-step algorithm is shown in = = d d (23) Fig. 1. | }| > v > } d d III. EXPERIMENTAL SETUP v =d =d >d >d In order to evaluate the performance of the proposed 2) The output vector of the magnetometer, (R, is rotated algorithm, raw inertial and magnetic data from Q using the horizontal rotation matrix S"kqjq. The rotated Xsens MTi-G-700 at the rate of 100 Hz are used. The facf vector is expressed by(F R. The horizontal component of reference system uses IMU-FSAS (tactical grade IMU with the rotated vector is located in the horizontal plane (East- fiber optic gyroscopes and servo accelerometers) integrated North plane) of the navigation frame. in Novatel SPAN system with inertial explorer post processing software. Orientation accuracy of the reference (facf xQ " ( F R S >5=5 R (24) system is 0.008 012 for the roll and pitch angles and 0.023 012 for the yaw angle. The MTi-G-700 is attached on top of 3) The measured yaw angle, d v9d/9d MI /_ are set to , and 0.3, respectively. However, under temporary magnetic disturbances, the To examine the robustness of the algorithm against magnetometer output vector cannot be trusted as reliable temporary ferromagnetic disturbance and medium to large information. To avoid the effect of disturbed magnetic field human body accelerations, three different tests were on the estimated yaw angle, a threshold-based switching performed. To be consistent with the output data of the approach similar to the method in [8] is employed. In the reference system yaw angle is converted to heading angle. threshold-based switching method, the measured magnetic IV. EXPERIMENTAL RESULTS AND DISCUSSION %($*#(01$01$#%-/0(&,(%(" ,1#$3( 1(-,0%/-+1'$*-" *$ /1'<0 magnetic field. This validation test excludes the disturbed A. Experiment 1: Ferromagnetic disturbance measurement by assigning a large measurement noise value In this experiment, MTi-G-700 is left stationary for 120 s in the yaw Kalman filter. As a result, "I , the and the magnetic field is temporarily disturbed using an iron measurement noise covariance matrix for the yaw Kalman disk for 5 s, 10 s and 15 s intervals. The normalized filter is defined as: magnitude of the magnetic field and the calculated heading "I x angle with and without the threshold-based switching is I A]BJ (R vC y?] @àe v@àefNGy?àe (26) shown in Fig. 3. As it can be seen in this figure, the x { +:318;491 proposed algorithm with threshold-based switching rejects

6272 38 C. Experiment 3: Medium acceleration with long duration To evaluate performance of the proposed algorithm for a typical motion with low to medium level of acceleration, the subject is asked to rollerblade outdoors while carrying the backpack. The orientation results presented in Fig. 5 show that the estimated orientation is reasonably good during the entire test duration and no orientation drift is observed.

V. CONCLUSION This work proposes the use of a cascaded two-step Kalman filter for human body orientation determination using MEMS inertial/magnetic sensors. In the first step, Fig. 3. Top: heading angle with and without threshold-based switching, ""$*$/-+$1$/0<# 1 *-,&4(1' , ""$*$/ 1(-,+-#$*(020$# Bottom: normalized magnetic field norm in a Kalman filter to accurately estimate tilt angles. In the the disturbances and the estimated heading angle remains second step, the estimated tilt angles along with stable during temporary ferromagnetic disturbances. A + &,$1-+$1$/0< # 1 /$ 2sed in a Kalman filter along small heading drift can also be observed during the accurately track the yaw angle. The yaw Kalman filter disturbance which is due to reliance of the algorithm on employs threshold-based switching method to deal with gyroscopes data. short-term magnetic disturbances. In comparison to the most popular orientation estimation algorithms, the proposed B. Experiment 2: Large acceleration with short duration method does not use optimization for orientation estimation, To produce large acceleration, in this experiment, the which makes it computationally more efficient. Additionally, subject is asked to wear the backpack and perform a vertical because of its two-step structure, the tilt angles would not be jump. Roll, pitch and heading angles along with the norm of affected by a perturbed magnetic field while the estimated tilt the tri-axial accelerometer output vector during a typical angles help to determine yaw angle more accurately. vertical jump is shown in Fig. 4. As it can be seen in this Experimental results show that the proposed algorithm can figure, the acceleration norm is reached to a maximum of 40 cover various ranges of human body motions and is robust I 59 . However, the algorithm can accurately track against temporary magnetic disturbances. orientation even in the presence of the large acceleration. REFERENCES [1] $$ ,# /)8 % 01 200-Newton optimizer for $01(+ 1(,&'2+ ,!-#6-/($,1 1(-,:Conf. Proc. IEEE Eng. Med. Biol. Soc., Vancouver, British Columbia, Canada, Aug. 2008, pp. 1679782. [2] /2"),$/ .(,#$*#/$($/ ,# *2+$8-#(%(" 1(-, ,# fixed-point analysis of a Kalman filter for orientation estimation based on 9- (,$/1( *+$ 02/$+$,12,(1# 1 :Conf. Proc. IEEE Eng. Med. Biol. Soc., Osaka, Japan, Jul. 2013, pp. 395376. [3] 3/$ -**$0 (00 -2( ,# +(,( ,8 +!2* 1-/6 measurement of 3D knee joint angle:J. Biomech., vol. 41, no. 5, pp. 1029735, Jan. 2008. [4] 2, ,# "'+ ,,8 $0(&,+.*$+$,1 1(-, ,# Experimental Results of a Quaternion-Based Kalman Filter for Human -#6-1(-,/ ")(,&:IEEE Trans. Robot., vol. 22, no. 6, pp. 12167 1227, Dec. 2006. Fig. 4. Roll, pitch and heading angles during a typical vertical jump [5] J.K. Lee and E.J Park8(,(+2+-Order Kalman Filter With Vector $*$"1-/%-/ ""2/ 1$ 01(+ 1(-,-%2+ , -#6/($,1 1(-,:IEEE Trans. Robot., vol. 25, no. 5, pp. 119671201, Oct. 2009. [6] $/26 + ,# 1 , !$8 ! 0("012#6-,3 /( !*$-gain Kalman filter based on angle error calculated from acceleration signals %-/*-4$/*(+! ,&*$+$ 02/$+$,14(1'(,$/1( *0$,0-/0:Conf. Proc. IEEE Eng. Med. Biol. Soc., Osaka, Japan, Jul. 2013, pp. 342376. [7] ! 1(,(82 1$/,(-,-based extended Kalman filter for #$1$/+(,(,&-/($,1 1(-,!6(,$/1( * ,#+ &,$1("0$,0(,&:IEEE Trans. Biomed. Eng., vol. 53, no. 7, pp. 1346756, Jul. 2006. [8] 2(,&$ ,#$*1(,)8$ 02/(,&-/($,1 1(-,-%'2+ , body 0$&+$,1020(,&+(,( 12/$&6/-0"-.$0 ,# ""$*$/-+$1$/0:Med. Biol. Eng. Comput., vol. 43, no. 2, pp. 2737282, Mar. 2005. [9] D. Roetenberg, H. J. Luinge, C. T. M. Baten, and P. H. Veltink, 8 -+.$,0 1(-,-%+ &,$1("#(012/! ,"$0(+./-3$0(,$/1( * ,# magnetic 0$,0(,&-%'2+ ,!-#60$&+$,1-/($,1 1(-,:IEEE Trans. Neural Syst. Rehabil. Eng., vol. 13, no. 3, pp. 3957405, Sep. 2005. [10] $$ /) ,#-!(,-3(1"'8 01(+ 1(-,-% 11(12#$ ,# Fig. 5. Roll, pitch and heading angles during a rollerblading experiment external acceleration using inertial sensor measurement during various (The discontinuities in the heading angle curve is due to its range being #6, +(""-,#(1(-,0: / ,0,01/2+$ 03-*,-.. from 0 to 360 deg) 226272273, Aug. 2012.

6273 39 Appendix B.

Integration of MEMS inertial and pressure sensors for vertical trajectory determination

(Reproduced with permission from IEEE)

40 804 IEEE TRANSACTIONS ON INSTRUMENTATION AND MEASUREMENT, VOL. 64, NO. 3, MARCH 2015 Integration of MEMS Inertial and Pressure Sensors for Vertical Trajectory Determination Shaghayegh Zihajehzadeh, Tien Jung Lee, Jung Keun Lee, Member, IEEE, Reynald Hoskinson, and Edward J. Park, Senior Member, IEEE

Abstract— Integration of a low-cost global positioning sys- suitable for motion capture of outdoor sports such as skiing, tem (GPS) with a microelectromechanical system-based inertial snowboarding, rollerblading, and biking, which take place over measurement unit (MEMS-IMU) is a widely used method that large distances. Inertial navigation systems (INSs), on the other takes advantage of the individual superiority of each system to get a more accurate and robust navigation performance. However, hand, are self-contained and thus can provide unconstrained because of poor observations as well as multipath effects, the accessibility to advanced motion and location information. GPS has low accuracy in the vertical direction. As a result, Since its emergence, INS is widely used in various motion the navigation accuracy even in an integrated GPS/MEMS-IMU capture (MOCAP) applications such as ships [2] and system is more challenged in the vertical direction than the aircraft [3]–[6] navigation, fastening tool [7], and pen [8] horizontal direction. To overcome this problem, this paper investigates the integration of a MEMS barometric pressure sen- tracking, as well as sport analysis [9] to provide navigation sor with the MEMS-IMU for vertical position/velocity tracking information such as attitude (roll and pitch angles), velocity, without the GPS that has applications in sports. A cascaded and position. two-step Kalman filter consisting of separate orientation and Recently, miniature microelectromechanical system position/velocity subsystems is proposed for this integration. Slow (MEMS) inertial devices have become commonplace, resulting human movements in addition to more rapid sport activities such as vertical and step-down jumps can be tracked using in the emergence of INS for human body motion tracking the proposed algorithm. The height-tracking performance is and attracting many researchers toward wearable MOCAP benchmarked against a reference camera-based motion-tracking technology. Typically, wearable MOCAP devices make system and an error analysis is performed. The experimental use of a MEMS inertial measurement unit (MEMS-IMU) results show that the vertical trajectory tracking error is less than and/or an absolute position sensor to capture motion 28.1 cm. For the determination of jump vertical height/drop, the proposed algorithm has an error of less than 5.8 cm. in addition to indoor/outdoor localization. For instance, Index Terms— Barometric pressure sensor, inertial in [10]–[12], wearable inertial devices have been used for measurement unit (IMU), inertial navigation system (INS), accurate human body MOCAP. A camera system can be jump trajectory, Kalman filter (KF), microelectromechanical added to inertial devices as an absolute position sensor for system (MEMS), sports, vertical trajectory, wearable technology. more accurate human body indoor localization [13], [14]. For I. INTRODUCTION outdoor localization, the global positioning system (GPS) is OR athletes, access to quantitative key performance vari- integrated with the inertial devices [15]. Fables (KPVs) can significantly improve overall perfor- As a result of its more accurate and robust per- mance, allowing them to share information with coaches, formance for outdoor MOCAP, the GPS/MEMS-IMU record performance over time, and even real-time feedback integration has received an increased attention recently for of how they are doing. Despite this need, currently available trajectory determination in sports. In [16], the performance of video- or camera-based motion capture approaches provide the two widely used Kalman filtering methods, the unscented few quantitative variables [1]. Additionally, these techniques Kalman filter (UKF) and extended Kalman filter (EKF), for are restricted to the use of preset-confined areas that are not GPS/MEMS-IMU integration in sport trajectory determination are compared, finding the performance of the two algorithms Manuscript received June 30, 2014; revised September 3, 2014; accepted comparable but the UKF with a higher computational cost. September 6, 2014. Date of publication October 14, 2014; date of current ver- In [17], the EKF is used for jump trajectory determination in sion February 5, 2015. This work was supported by the Natural Sciences and Engineering Research Council of Canada (NSERC) and Recon Instruments skiing and mountain biking applications. Inc. The Associate Editor coordinating the review process was Dr. Salvatore One issue in the aforementioned GPS/MEMS-IMU Baglio. integration approaches is that the consumer-grade GPS-derived S. Zihajehzadeh, T. J. Lee, and E. J. Park are with the School of Mechatronic Systems Engineering, Simon Fraser University, Surrey, BC V3T 0A3, Canada vertical positional (or altitude) information is much less accu- (e-mail: [email protected]). rate than the horizontal [18]. According to [19], GPS altitude J. K. Lee is with the Department of Mechanical Engineering, Hankyong measurement accuracies can vary up to 40 m (10–20 m being National University, Anseong 456-749, Korea. R. Hoskinson was with Recon Instruments Inc., Vancouver, BC V6B 3W9, common) and the most important reason behind this is satellite Canada, and is now with the School of Mechatronic Systems Engineering, visibility (i.e., few visible satellites over the horizon) as well as Simon Fraser University, Surrey, BC V3T 0A3, Canada. multipath effects. Although recent progress in real-time GPS Color versions of one or more of the figures in this paper are available online at http://ieeexplore.ieee.org. technologies (real-time kinematic GPS or differential GPS) Digital Object Identifier 10.1109/TIM.2014.2359813 provide higher positional accuracies [20], their prohibitive 0018-9456 © 2014 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information. 41 ZIHAJEHZADEH et al.: INTEGRATION OF MEMS INERTIAL AND PRESSURE SENSORS 805 cost is a limiting factor for the sport consumer electronics Thus, it is important for the INS to accurately estimate market. external acceleration during dynamic conditions by subtracting One potential way of increasing vertical position infor- the gravitational acceleration from the accelerometer signal. mation is by using a barometric pressure sensor, which To realize this, an accurate estimation of orientation is needed. can also derive altitude using pressure measurements. There In this paper, a cascaded Kalman filter (CKF) consisting of a is a long history of using barometric pressure sensors for separate orientation KF followed by a vertical position/velocity aviation and vehicular navigation [21], [22]. Nevertheless, KF is proposed. owing to ergonomic and cost constraints, the traditional, Our reason for choosing the CKF is based primarily on its bulky, and expensive high-quality barometric altimeters are computational efficiency. By removing the calculations related not applicable in miniature devices worn by sport athletes. to orientation states in the position/velocity KF, the cascade The advent of state-of-the-art MEMS pressure sensors has formulation has much less computational overhead than a opened up new cost-effective opportunities to bring baro- global KF [26]. Although the cascaded filter is suboptimal, metric pressure sensors into the consumer electronics mar- the performance is comparable with the global KF [26], [27]. ket. However, height measurements from MEMS barometric In addition to the improved computational cost, the cascaded pressure sensors suffer from thermal noise and a significant implementation allows for increased flexibility and easier amount of quantization noise [23]. This makes them unsuitable implementation and tuning with relatively small performance for tracking highly dynamic movements in sports, but may loss. The structure of the individual filters, that is, the orien- still be used to stabilize the drift-prone MEMS-IMU for tation KF and position/velocity KF is presented below. In the navigation in the vertical direction [23]. The integration of following equations, subscripts 1 and 2 refer to the vari- a MEMS barometric pressure sensor with a MEMS-IMU ables of the orientation KF and vertical position/velocity KF, for applications under dynamics typical for human move- respectively. ments is investigated in [23] and [24]. However, the potential use of the MEMS barometric pressure sensor and A. Orientation Kalman Filter the achievable accuracy in vertical trajectory determination under highly dynamic conditions in sport-related activities It should be noted that for motion tracking in the vertical such as ski and rollerblade jumps require further investiga- direction, the tilt angles with respect to the vertical axis tion. There is a potential of positional accuracy degradation (i.e., the roll and pitch angles) are sufficient for compensation when highly dynamic movements introduce pressure changes of the gravitational acceleration and the full 3-D orientation is that can be mistakenly considered as height changes in the not required. Herein, the tilt angles are estimated using a KF output of the pressure sensor. on the basis of the acceleration model concept presented in our For the purpose of accurately determining vertical trajecto- previous work [28], which allows accurate determination (less ries in sports, the performance of the MEMS-IMU integrated than 2° accuracy) of the roll and pitch angles under dynamic with the MEMS barometric pressure sensor using a cascaded conditions. In this method, the measurements from triaxial two-step Kalman filter (KF) is investigated in this paper. The gyroscopes and triaxial accelerometers are used in a KF to novelty lies in the use of the magnetometer-less IMU for estimate the normalized gravity vector in the sensor frame by tilt angles determination, which makes the algorithm robust using the following system model equations [28]: against magnetic disturbances. Additionally, the use of the x (k) = A (k − 1)x (k − 1) + w (k − 1) (1) cascaded KF avoids the need to propagate additional states, 1 1 1 1 ( ) = ( ) ( ) + ( ). improving the computational efficiency, which makes the z1 k C1 k x1 k v1 k (2) algorithm suitable for small and lightweight battery-powered In (1) and (2), x (k) is the 3 × 1 state vector for orientation electronic devices worn by athletes. 1 filter at step k, which is the normalized gravity vector in The remainder of this paper is organized as follows. the sensor frame, A is the state transition matrix and w is In Section II, the theory behind the cascaded two-step KF for 1 1 the process model noise vector, z is the measurement vector vertical trajectory determination using the barometric aided 1 (i.e., the measured gravity vector in the sensor frame), C is IMU is introduced. Additionally, the initialization process 1 the 3 × 3 observation matrix, and v is the measurement model and trajectory smoothing algorithm are discussed. The exper- 1 noise vector. The matrices in (1) and (2) are calculated using imental setup and experimental protocol are explained in the following equations [28]: Section III. The experimental results consisting of vertical trajectory determination during slow motion and continuous A1(k − 1) = I3−tyG(k − 1) (3) jumping activities as an example of highly dynamic motion are w1(k − 1) = t(−x1(k))nG (4) presented in Section IV. Additionally, from the trajectory, two ( ) = jump-specific KPVs, that is, jump height and drop are derived C1 k gI3 (5) S − and presented. Finally, Section V concludes this paper. v1(k) =− aε (k) + nA (6) S − S − S aε (k) = a (k)− a(k) (7) II. THEORETICAL METHOD S − S + a (k) = ca a (k − 1) (8) In an INS, the navigation parameters, that is, position and velocity, are estimated by solving strapdown inertial navigation where I3 is a 3 × 3 identity matrix, yG is the 3 × 3skewsym- equations through integration of the external acceleration [25]. metric matrix of triaxial gyroscope measurements, x1 is 3 × 3 42 806 IEEE TRANSACTIONS ON INSTRUMENTATION AND MEASUREMENT, VOL. 64, NO. 3, MARCH 2015

Fig. 1. Proposed CKF for vertical trajectory determination. skew symmetric matrix of x1,andnG is the gyroscope mea- the following equations: surement noise vector, which is assumed to be uncorrelated S − ( − ) =− 2 ( − ) ( − ) and zero-mean white Gaussian, aε (k) is the external accel- Q1 k 1 t x1 k 1 G x1 k 1 (11) eration error in sensor frame, nA is the gyroscope measure- R1(k) = acc+A (12) ment noise vector, which is assumed to be uncorrelated and zero-mean white Gaussian, the superscripts + and − stand for where G is the covariance matrix of the gyroscope’s mea- [ T ] the a posteriori and the aprioriestimates in the KF, respec- surement noise which is deﬁned as E nG nG . By assuming tively, the superscript S denotes that the quantity is in the that the gyro noise variances are equal to σ G in the three axes, σ 2 sensor frame, ca is a dimensionless constant between 0 and 1 G is set to G I3. Similar to G , A which is the covariance σ 2 that determines the cutoff frequency in the external accelera- matrix of accelerometer’s measurement noise, is set to AI3. S S tion model ( a(k) = ca a(k − 1) + ε(k), with ε(k) being the acc is the covariance of the acceleration model and is set −1 2S +( − )2 time-varying error of the external acceleration process model), to 3 ca a k 1 . Sa+(k − 1) is the gravity compensated external acceleration, Using the estimated normalized gravity vector in the sensor +( ) =[ ]T and g is the norm of gravity vector. Next, the measurement frame x1 k x1,x x1,y x1,z from the orientation KF, the vector in (2) is calculated by [28] desired roll (γ) andpitch(β) angles are calculated by the : ( ) = ( )− S +( − ) following equations z1 k yA k ca a k 1 (9) + + S ( ) = ( ) − ( ) − x1,y − −x1,x a k yA k gx1 k (10) γ = tan 1 ,β = tan 1 . (13) x1,z x1,y/ sin γ where yA(k) is the bias compensated output vector of the accelerometer. As shown in Fig. 1, these two tilt angles become the The process and measurement noise covariance matrices in inputs to the second step of the proposed CKF, that is, the the orientation KF, Q1(k − 1) and R1(k), are calculated using position/velocity KF. 43 ZIHAJEHZADEH et al.: INTEGRATION OF MEMS INERTIAL AND PRESSURE SENSORS 807

B. Vertical Position/Velocity Kalman Filter The objective of this ﬁlter is to estimate the relative height measurement without the use of the GPS. This relative height can then be calibrated to the absolute height by using the GPS [28]. The following system model equations are used in a KF to estimate vertical position and vertical velocity:

x2(k) = A2(k−1)x2(k−1) + B2(k − 1)u2(k−1) + w2(k−1) (14) z2(k) = C2(k)x2(k) + v2(k) (15) T where x2(k) =[h(k) v(k)] is the state vector for the position/velocity subsystem, A2(k − 1) and B2(k − 1) are, respectively, the state transition and input matrices for the position/velocity subsystem, u2(k − 1) is the input vector, being the vertical component of gravity compensated acceleration in the navigation frame, w2(k − 1) is the 2 × 1 vector of process noise, z2(k) is the relative height (hbaro),which is calculated from the barometric pressure data, and C2(k) is the observation matrix and v2(k) is the measurement noise. These variables are calculated as follows: 1 t A (k − 1) = (16) 2 01 1 2 ( − ) = t Fig. 2. Experimental setup: Xsens IMU (MTi-G-700) attached to a ski goggle B2 k 1 2 (17) and Qualisys MOCAP as a reference. t ( − ) = ([ ]) · n S +( − ) u2 k 1 ⎡001 s Rγ,β a⎤ ⎡k 1 (18)⎤ tool for limiting the drift error in the integrated navigation cos β 0 sin β 1 0 0 solution, has been used. It consists of forcing the velocity n = ⎣ ⎦ ⎣ ⎦ s Rγ,β 0 1 0 0 cos γ − sin γ measurement to zero when still phases are detected. In this − sin β 0 cos β 0 sin γ cos γ paper, still phases are detected by setting a threshold on the (19) norm of acceleration signal measured by the accelerometers. 1 2 This threshold value is the maximum of the measured acceler- ( − ) = t σ w2 k 1 2 A (20) ation norm during the stationary initialization. Fig. 1 shows the t overall structure of the proposed cascaded KF for the vertical 0.19 P trajectory determination. z2(k) hbaro = 44330 1 − − hinit (21) P0 ( ) = . C2 k [1 0] (22) C. Stationary Initialization n In (18) and (19), s Rγ,β is the rotation matrix that aligns During the stationary initialization step, initial attitude and the z-axis of sensor frame to the navigation frame. In (21), sensor biases are calculated. The triaxial accelerometer and hbaro is the relative height with respect to the initial triaxial gyroscope data are used in the preceding orientation S + height (hinit) at the start of the experiment, hinit is calculated KF to calculate the average value of initial tilt angles. a (k), using the initial pressure data at the start of the experiment, the external acceleration in the sensor frame, is the byproduct and P0 is the standard pressure equal to 101, 325 Pa. Because of the attitude KF. However, since the external acceleration of the signiﬁcant quantization noise present in the barometric should be zero during the stationary initialization step, the pressure sensor, a rolling average ﬁlter should be used to average value of Sa+(k) is considered as the accelerometer increase the accuracy of the barometric height measurement bias vector in the sensor frame. Assuming that MEMS gyro- and its remaining residual errors should be considered in the scopes are not accurate enough to measure earth’s rotation measurement noise (v2(k)). The process and measurement rate, the gyro bias vector is calculated as the mean value of noise covariance matrices in the KF, Q2(k − 1) and R2(k), the triaxial gyroscope measurement during this stationary ini- are calculated as follows: tialization step. These bias values are assumed to be constant 1 1 T during the short period of each experiment. The estimated t2 2 t2 Q (k − 1) = 2 σ 2 (23) 2 t A t accelerometer and gyroscope biases are then subtracted from the measured accelerometer and gyroscope signals for vertical R (k) = σ 2 (24) 2 baro trajectory estimation. During the stationary initialization step, σ 2 where baro is the barometer noise variance. velocity is set to zero and hinit in the barometric pressure In addition to the position measurement in the KF, an auto- sensor is calibrated by initial height obtained from a reference matic zero velocity update (ZUPT) [29] detector, an important system at the start of the experiment. This reference system 44 808 IEEE TRANSACTIONS ON INSTRUMENTATION AND MEASUREMENT, VOL. 64, NO. 3, MARCH 2015

Fig. 3. Norm of the accelerometer and gyroscope measurements from the Xsens IMU. (a) Typical slow motion experiment. (b) Typical vertical jump experiment. can provide the actual ellipsoid height for absolute height indoor experiments. The Xsens MTi-G-700 consists of triaxial tracking (such as the one obtained from GPS) or relative height gyroscopes, triaxial accelerometers, triaxial magnetometers, with respect to an arbitrary coordinate system for relative a GPS receiver, and a barometric pressure sensor. Because height tracking (such as the one obtained from camera-based the proposed algorithm only uses an accelerometer, gyroscope, MOCAP system). and barometric pressure sensor, the GPS and magnetometer data are ignored in this paper. The raw IMU and barometric D. Trajectory Smoothing pressure data are provided at the rate of 100 and 50 Hz, respectively. A Qualisys optical MOCAP system, with the In applications where real-time data processing is not positional accuracy of 0.25 mm, is then used as a reference required, the navigation performance can be further improved motion tracking system for the purpose of algorithm perfor- by smoothing, which combines the forward and backward mance veriﬁcation. processed data [30]. In this paper, the Rauch–Tung–Striebel Our ultimate goal is to implement the proposed algorithm (RTS) smoothing algorithm, a widely used algorithm in navi- into active sports goggles, having builtin onboard processor gation applications, is used to smooth out position and velocity and inertial/barometric pressure sensors, to determine an ath- trajectories [30]. RTS-smoother consists of one forward data- lete’s vertical trajectory in skiing, snowboarding, or biking. processing and one backward data-processing part. The for- Recon Instruments, Inc., a Vancouver (BC, Canada) company, ward processing is the aforementioned position/velocity KF has recently developed and commercialized such active sports algorithm in which the smoother stores the estimate and its goggles. Thus, as shown in Fig. 2, the IMU sensor is attached covariance. Then, it recursively updates the smoothed estimate on top of a ski goggle worn by the subjects during the and its covariance in a backward sweep by using the following experiments. A marker is attached on top of the IMU sensor : equations [31] as shown in Fig. 2, to track its motion using the reference + − −1 MOCAP system. To detect the start (when the subject’s foot Ks(k) = P (k)A2(k)[P (k + 1)] (25) 2 2 leaves the ground) and end point (when the subject’s foot hits ( ) = +( ) + ( ) ( + ) − −( + ) ( )T Ps k P2 k Ks k Ps k 1 P2 k 1 Ks k the ground) of the jumps during the jumping experiments, a (26) marker is attached to the subject’s foot to track foot motion + − xs(k) = x (k) + Ks (k)[xs(k + 1) − x (k + 1)] (27) using the reference MOCAP system. 2 2 σ 2 + − The gyroscope noise variance G and the accelerometer where P and P are the a posteriori and the apriori σ 2 2 2 + − noise variance A are obtained from static measurements and −4 2 2 −4 2 4 covariance estimates, Ks is the smoother gain, x2 and x2 are, are set to 10 rad /s and 10 m /s , respectively. The respectively, the a posteriori and the aprioristate estimates, σ 2 2 barometer noise variance baro issetto1m. and xs is the smoothed state vector. All of the aforementioned variables are for the vertical position/velocity KF. B. Experimental Protocol The algorithm should be able to determine vertical trajectory III. EXPERIMENTAL SETUP AND PROTOCOL during typical sport activities with moderate dynamics and A. Experimental Setup also during featured activities such as jumping that involve To evaluate the performance of the proposed algorithm, an high dynamics. Our lab-based experiments are designed to Xsens MTi-G-700 is used as the primary MEMS-IMU in our simulate these two types of motions. The ﬁrst type is simulated 45 ZIHAJEHZADEH et al.: INTEGRATION OF MEMS INERTIAL AND PRESSURE SENSORS 809

Fig. 4. Vertical position trajectory during a typical slow motion experiment. Fig. 5. Estimated roll and pitch angles during a typical slow motion experiment. by asking the subjects to move the IMU sensor slowly by hand over an arbitrary vertical trajectory in the visible range of the reference MOCAP system. This experiment is referred to as the slow-motion experiment in the rest of this paper. The highly dynamic activities are simulated by asking the subjects to perform continuous vertical jumps and step-down jumps (jumping from a step to the ground) during the trials. Each trial starts with about 1 min of stationary initialization following by about 1 min of continuous motion. The raw angular velocity and acceleration during a typical slow motion and vertical jump experiment collected by Xsens IMU are shown in Fig. 3.

C. Participants Eleven young healthy subjects (seven men and four women), aged between 20 and 35 years, participated in the experiment. All participants were students at Simon Fraser University, recruited through advertisements posted on university notice Fig. 6. Effect of RTS-smoother on position trajectory. boards. Each provided an informed written consent, and the experimental protocol was approved by the Research Ethics Board at Simon Fraser University. Figs. 7 and 8 shows the comparison the calculated vertical position/velocity trajectories with the reference trajectories IV. EXPERIMENTAL RESULTS AND DISCUSSION for a typical vertical and step-down jumping experiment, This section is divided as follows. In Section IV-A, the respectively. The calculated tilt angles profile during these accuracy of the proposed algorithm in vertical position/ experiments is also shown in Figs. 7 and 8. Fluctuation of tilt velocity tracking for the three types of experiments: angles during stationary initialization in Figs. 7(b) and 8(b) is slow-motion, vertical jump, and step-down jump, is obtained. due to slight inevitable movement of subjects’ head during In Section IV-B, the accuracy of jump KPVs determination for this period. However, the acceleration threshold of ZUPT the two types of jumping experiments, vertical and step-down detector is set such that ZUPT is active in this period resulting jumps, is discussed. in zero velocity and constant height during initialization. Because the stationary initialization for slow motion experiment is performed by leaving the sensor stationary on the A. Vertical Position and Velocity Tracking Accuracy ground, the aforementioned fluctuations have not been seen Fig. 4 shows the comparison of vertical position trajectory in Fig. 5. The deviation of the calculated vertical position of the slow motion experiment obtained by the proposed trajectory from the reference trajectory is more evident as algorithm with the one obtained by the reference system. shown in Figs. 7 and 8, in comparison with Fig. 4. This can Fig. 5 shows the calculated tilt angles profile during the same be justified by the highly dynamic nature of the two jumping experiment. Fig. 6 shows the effect of the RTS-smoother experiments in comparison with the slow motion experiment in smoothing out the discontinuities caused by the pressure and the effect of this dynamics on the performance of the pres- sensor’s measurement update in KF. sure sensor. In fact, the highly dynamic movements introduce 46 810 IEEE TRANSACTIONS ON INSTRUMENTATION AND MEASUREMENT, VOL. 64, NO. 3, MARCH 2015

Fig. 7. (a) Vertical position and velocity trajectories. (b) Tilt angles proﬁles during vertical jump experiment.

Fig. 8. (a) Vertical position and velocity trajectories. (b) Tilt angles proﬁles during step-down jump experiment.

TABLE I MEAN AND ONE STANDARD DEVIATION OF POSITION AND VELOCITY TRACKING WITH THE PROPOSED VERTICAL POSITION/VELOCITY KALMAN FILTER (KF)

pressure changes, which are mistakenly considered as height the proposed algorithm with and without the smoother. These changes in the output of pressure sensor. mean RMSE and standard deviation values for each type of Table I presents the mean and one standard deviation of experiment are averaged out over the 11 trials performed by the root-mean-square error (RMSE) in vertical position and the 11 subjects. It can be clearly noticed in Table I that velocity tracking for the three types of experiments using the vertical position and velocity tracking in the slow-motion

47 ZIHAJEHZADEH et al.: INTEGRATION OF MEMS INERTIAL AND PRESSURE SENSORS 811

With the above definition of jump height/drop, these two KPVs for vertical and step-down jumps are shown in Figs. 11 and 12 and the accuracy of the proposed trajectory- tracking algorithm in determination of these two parameters is investigated in this section. Table II presents the average KPVs and their error values for the two types of jumps Fig. 9. Ollie jump and definition of height and drop. as obtained from the proposed algorithm and the reference MOCAP system. The reported values in this table are averaged out over the 80 vertical jumps and 50 step-down jumps performed by the 11 subjects. Because jump height and drop have the same value for vertical jump, the reported values for vertical jump in Table II are averaged over these two parameters. The reported values for step-down jump are with respect to drop variable. To demonstrate the advantage of the RTS-smoother, two results are shown for the proposed algorithm in Table II. Fig. 10. Cliff drop jump and definition of height and drop. For each type of jump, one set of KPVs are calculated by the application of the proposed algorithm excluding RTS-smoother, whereas the second set is calculated by the experiment is more accurate than those of the vertical and complete application of the proposed algorithm including the step-down jump experiments. This observation also verifies RTS-smoother. Table II shows that the proposed algorithm can the more accurate performance of the barometric pressure determine jump KPVs with 22% error for the vertical jump sensor for less dynamic motions. Table I also shows that the and 33% error for the step-down jump. Additionally, it shows RTS-smoother is effective in reducing the RMSE error: that these error values can be significantly reduced by using an between 10.9% and 17.3% for vertical position and between RTS-smoother. A smoother can effectively reduce the average 33.3% and 47.7% for vertical velocity RMSE reduction. This KPV error to 2.9 cm (64.2% error reduction) and 5.8 cm achieved vertical position trajectory-tracking accuracy is a (57.3% error reduction) for vertical jump and step-down jump, significant improvement over the 40 m accuracy of GPS in respectively. However, the smoother is only applicable when vertical direction [19]. real-time operation is not required and incurs the cost of a higher computational load. B. Jump KPVs Determination Accuracy Lastly, the comparison between the height-tracking error Jump height and drop are two important KPVs in in Table I and the calculated jump KPV error in Table II jumping [17]. Accurate determination of these variables is reveals that the proposed algorithm is more accurate in jump useful both in entertainment situations (for general interest KPV determination, because the erroneous pressure change in during recreational sport), or as a tool to aid in training dynamic motion appears as a vertical shift in position, which for competitions. These two KPVs are defined as [17]: is almost eliminated in the differential calculation of the jump 1) height—total vertical distance achieved from the point of KPV values. takeoff to the apex of the jump and 2) drop—total vertical distance achieved from the apex to the point of landing. The definition of these two KPVs for two basic C. Limitations of the Results ski/snowboard jumps, that is, ollie and cliff drop are shown The indoor experimental results presented in this paper may in Figs. 9 and 10, respectively. have some limitations if directly applied to outdoor sports such Vertical trajectories of a subject’s head and foot during as skiing and snowboarding. While our lab-based subject trials a typical vertical jump and step-down jump are shown in were suited for the purpose of this paper, which was the the- Figs. 11 and 12, respectively. The foot trajectories in these oretical and proof-of-concept demonstration of the proposed two figures show that in our lab-based jumping experiments, MEMS inertial/pressure sensing-based vertical KF algorithm, the vertical jump trajectory [Fig. 11(b)] is indeed similar to our ultimate goal is to expand it for vertical jump trajectory the ollie jump trajectory as shown in Fig. 9 and the step-down tracking in skiing and snowboarding. Since no field testing jump trajectory [Fig. 12(b)] is similar to the cliff-drop jump was carried out, the proof-of-concept results presented herein trajectory as shown in Fig. 10. may not yet be generalizable to show robustness for outdoor However, as shown in Fig. 11(a), instead of having skiing/snowboarding activities, especially during high wind or monotonically increasing height from the start of the jump to speed conditions when the altitude measurements from the the apex and monotonically decreasing trajectory from apex barometric pressure sensor might be disturbed and become to the end of jump, two additional minimums are seen in the unreliable. trajectory, because the attached sensor to the head captures There are technical and logistic challenges that prohibit the inevitable squatting, which happens right before and after us from performing field experiments for this paper. each jump. The local minimums of the two captures shown in Both the proper installation and calibration of our reference Fig. 12(a) also have the same cause. camera-based MOCAP system at ski jump platforms on actual 48 812 IEEE TRANSACTIONS ON INSTRUMENTATION AND MEASUREMENT, VOL. 64, NO. 3, MARCH 2015

Fig. 11. (a) Head vertical trajectory during a typical vertical jump. (b) Foot vertical trajectory for the same vertical jump obtained by the referencesystem.

Fig. 12. (a) Head vertical trajectory during a typical step-down jump. (b) Foot vertical trajectory for the same step-down jump obtained by the reference system.

TABLE II JUMP KPV DETERMINATION ERROR WITH THE PROPOSED VERTICAL POSITION/VELOCITY KF

ski hills are very difﬁcult due to the conditions (e.g., lighting, reference system, a Novatel SPAN system consisting of temperature, weather, and lack of power) of the location. SPAN receiver with real-time kinematic GPS capability, and Alternatively, we have previously conducted ﬁeld experiments a tactical grade IMU, for another study involving motion by using a highly accurate inertial motion-tracking tracking in rollerblading [32] and also general 3-D trajectory

49 ZIHAJEHZADEH et al.: INTEGRATION OF MEMS INERTIAL AND PRESSURE SENSORS 813 tracking of expert skiers/snowboarders. However, although [6] D. A. Grejner-Brzezinska, C. K. Toth, H. Sun, X. Wang, and C. Rizos, such a system can be carried by the participants in a large “A robust solution to high-accuracy geolocation: Quadruple integration of GPS, IMU, pseudolite, and terrestrial laser scanning,” IEEE Trans. backpack, it is too cumbersome for them to perform jumps Instrum. Meas., vol. 60, no. 11, pp. 3694–3708, Nov. 2011. properly and safely. With our recent work [33], we will [7] S.-H. P. Won, F. Golnaraghi, and W. W. Melek, “A fastening tool tracking be developing an accurate biometric jump measurement system using an IMU and a position sensor with Kalman filters and a fuzzy expert system,” IEEE Trans. Ind. Electron., vol. 56, no. 5, system on the basis of high-definition video footage through pp. 1782–1792, May 2009. calibration of the scenes and homographic transformations. [8] J.-S. Wang, Y.-L. Hsu, and J.-N. Liu, “An inertial-measurement-unit- based pen with a trajectory reconstruction algorithm and its applica- V. C ONCLUSION tions,” IEEE Trans. Ind. Electron., vol. 57, no. 10, pp. 3508–3521, Oct. 2010. This paper proposes the use of a cascaded two-step [9]C.N.K.Nam,H.J.Kang,andY.S.Suh,“Golfswingmotiontracking KF combined with an RTS-smoother for loosely coupled inte- using inertial sensors and a stereo camera,” IEEE Trans. Instrum. Meas., vol. 63, no. 4, pp. 943–952, Apr. 2014. gration of a MEMS-IMU with a MEMS barometric pressure [10] J. K. Lee and E. J. Park, “Minimum-order Kalman filter with vector sensor. Applications of the proposed algorithm include vertical selector for accurate estimation of human body orientation,” IEEE Trans. trajectory determination in sports such as rollerblading, skiing, Robot., vol. 25, no. 5, pp. 1196–1201, Oct. 2009. snowboarding, or biking. The two-step cascaded KF uses two [11] G. Panahandeh, N. Mohammadiha, A. Leijon, and P. Handel, “Continu- ous hidden Markov model for pedestrian activity classification and gait linear KFs: an orientation KF for determination of tilt angles analysis,” IEEE Trans. Instrum. Meas., vol. 62, no. 5, pp. 1073–1083, followed by a vertical position/velocity KF. The proposed May 2013. algorithm avoids the use of a magnetometer for attitude [12] G. X. Lee, K. S. Low, and T. Taher, “Unrestrained measurement of arm motion based on a wearable wireless sensor network,” IEEE Trans. determination which makes it robust against electromagnetic Instrum. Meas., vol. 59, no. 5, pp. 1309–1317, May 2010. disturbances. Additionally, because of its cascaded structure, [13] A. Colombo, D. Fontanelli, D. Macii, and L. Palopoli, “Flexible indoor it avoids the need to propagate additional states, resulting localization and tracking based on a wearable platform and sensor data fusion,” IEEE Trans. Instrum. Meas., vol. 63, no. 4, pp. 864–876, in more computational efficiency needed for small and Apr. 2014. lightweight battery-powered wearable technologies for sports. [14] Y. Tian, W. R. Hamel, and J. Tan, “Accurate human navigation using The accuracy of the proposed method is benchmarked wearable monocular visual and inertial sensors,” IEEE Trans. Instrum. Meas., vol. 63, no. 1, pp. 203–213, Jan. 2014. against a highly accurate camera-based MOCAP system for [15] H. Kuusniemi, J. Liu, L. Pei, Y. Chen, L. Chen, and R. Chen, “Reliability slow and highly dynamic motion which involves jumping. The considerations of multi-sensor multi-network pedestrian navigation,” experimental results show that the vertical trajectory-tracking IET Radar, Sonar Navigat., vol. 6, no. 3, pp. 157–164, Mar. 2012. [16] A. Waegli and J. Skaloud, “Optimization of two GPS/MEMS-IMU error is about 26.9, 27.2, and 28.1 cm for slow motion, vertical integration strategies with application to sports,” GPS Solutions, vol. 13, jump, and step-down jumps vertical trajectory tracking, no. 4, pp. 315–326, Apr. 2009. respectively. Significant improvement in vertical trajectory- [17] F. Sadi and R. Klukas, “New jump trajectory determination method using tracking accuracy was observed with the use of an low-cost MEMS sensor fusion and augmented observations for GPS/INS integration,” GPS Solutions, vol. 17, no. 2, pp. 139–152, Jun. 2012. RTS-smoother. Additionally, according to the experimental [18] O. Švábenský, J. Weigel, and R. Machotka, “On GPS heighting in results, the average jump KPVs (height/drop) determination local networks,” Acta Geodyn. Geomater, vol. 3, no. 143, pp. 39–43, error with the use of an RTS-smoother is about 2.9 and 5.8 cm Jun. 2006. [19] J. Zhang, E. Edwan, J. Zhou, W. Chai, and O. Loffeld, “Performance for vertical jump and step-down jump, respectively (compared investigation of barometer aided GPS/MEMS-IMU integration,” in Proc. with the ∼40 m error of the GPS in some spots in the vertical IEEE/ION Position Location Navigat. Symp. (PLANS), Apr. 2012, direction). This achieved accuracy should be sufficient for pp. 598–604. [20] F. Aghili and A. Salerno, “Driftless 3-D attitude determination and recreational purposes where the magnitude of jump height is positioning of mobile robots by integration of IMU with two RTK small. On the other hand, the magnitude of jump height in GPSs,” IEEE/ASME Trans. Mechatronics, vol. 18, no. 1, pp. 21–31, athletic sport jumps is much higher than the one in recreational Feb. 2013. [21] S.-S. Jan, D. Gebre-Egziabher, T. Walter, and P. Enge, “Improving GPS- jumps. As a result, the achieved jump KPVs determination based landing system performance using an empirical barometric altime- accuracy of about 2.9–5.8 cm should also be adequate for ter confidence bound,” IEEE Trans. Aerosp. Electron. Syst., vol. 44, outdoor measurement purposes using wearable technologies. no. 1, pp. 127–146, Jan. 2008. [22] N. Hayashi, “Augmentation of GPS with a barometer and a heading rate gyroscope for urban vehicular navigation,” M.S. thesis, Dept. Geomatics, REFERENCES Univ. Calgary, Calgary, AB, Canada, 1996. [1] A. Wägli, “Trajectory determination and analysis in sports by satel- [23] M. Tanigawa, H. Luinge, L. Schipper, and P. Slycke, “Drift-free dynamic lite and inertial navigation,” Ph.D. dissertation, Archit. Construit Lab. height sensor using MEMS IMU aided by MEMS pressure sensor,” in Topométrie, École Polytechnique Fédérale de Lausanne, Lausanne, Proc. 5th Workshop Positioning, Navigat. Commun., Hanover, Germany, Switzerland, 2009. Mar. 2008, pp. 191–196. [2] B. Wang, Z. Deng, C. Liu, Y. Xia, and M. Fu, “Estimation of informa- [24] P. Groves et al., “Integrated pedestrian navigation using GNSS, MEMS tion sharing error by dynamic deformation between inertial navigation IMU, magnetometer and baro-altimeter,” in Proc. NAV, London, U.K., systems,” IEEE Trans. Ind. Electron., vol. 61, no. 4, pp. 2015–2023, Nov. 2007, pp. 822–865. Apr. 2014. [25] J. L. Crassidis, “Sigma-point Kalman filtering for integrated GPS and [3] K. D. Sebesta and N. Boizot, “A real-time adaptive high-gain EKF, inertial navigation,” IEEE Trans. Aerosp. Electron. Syst., vol. 42, no. 2, applied to a quadcopter inertial navigation system,” IEEE Trans. Ind. pp. 750–756, Apr. 2006. Electron., vol. 61, no. 1, pp. 495–503, Jan. 2014. [26] D. B. Kingston and R. W. Beard, “Real-time attitude and position [4] L. Chen and J. Fang, “A hybrid prediction method for bridging GPS estimation for small UAVs using low-cost sensors,” in Proc. AIAA 3rd outages in high-precision POS application,” IEEE Trans. Instrum. Meas., Unmanned Unlimited Tech. Conf. Work. Exhibit, Chicago, IL, USA, vol. 63, no. 6, pp. 1656–1665, Jun. 2014. Sep. 2004, pp 489–497. [5] F. Jiancheng and Y. Sheng, “Study on innovation adaptive EKF for in- [27] Z. Lendek, R. Babuška, and B. De Schutter, “Distributed Kalman flight alignment of airborne POS,” IEEE Trans. Instrum. Meas., vol. 60, filtering for cascaded systems,” Eng. Appl. Artif. Intell., vol. 21, no. 3, no. 4, pp. 1378–1388, Apr. 2011. pp. 457–469, Apr. 2008. 50 814 IEEE TRANSACTIONS ON INSTRUMENTATION AND MEASUREMENT, VOL. 64, NO. 3, MARCH 2015

[28] J. K. Lee, E. J. Park, and S. N. Robinovitch, “Estimation of attitude Jung Keun Lee (M’11) received the B.Sc. and and external acceleration using inertial sensor measurement during M.Sc. degrees in mechanical engineering from various dynamic conditions,” IEEE Trans. Instrum. Meas., vol. 61, no. 8, Hanyang University, Seoul, Korea, in 1997 and pp. 2262–2273, Aug. 2012. 1999, respectively, and the Ph.D. degree in mecha- [29] F. Höflinger, J. Müller, R. Zhang, L. M. Reindl, and W. Burgard, tronic systems engineering from the School of Engi- “A wireless micro inertial measurement unit (IMU),” IEEE Trans. neering Science, Simon Fraser University, Surrey, Instrum. Meas., vol. 62, no. 9, pp. 2583–2595, Sep. 2013. BC, Canada, in 2010. [30] H. Liu, S. Nassar, and N. El-Sheimy, “Two-filter smoothing for accurate He was a Naval Officer with the Republic of INS/GPS land-vehicle navigation in urban centers,” IEEE Trans. Veh. Korea Navy, Jinhae, Korea, and then a Research Technol., vol. 59, no. 9, pp. 4256–4267, Nov. 2010. Engineer with the Powertrain Research and Devel- [31] D. Simon, Optimal State Estimation: Kalman, H Infinity, and Nonlinear opment Centre, Hyundai Motor Company, Seoul, Approaches. New York, NY, USA: Wiley, 2006. Korea. Since 2012, he has been with Hankyong National University, Anseong, [32] S. Zihajehzadeh, D. Loh, M. Lee, R. Hoskinson, and E. J. Park, Korea, where he is currently an Assistant Professor with the Department “A cascaded two-step Kalman filter for estimation of human body of Mechanical Engineering. His current research interests include inertial- segment orientation using MEMS-IMU,” in Proc. IEEE EMBC, Chicago, sensing-based human motion tracking, biomechatronics, wearable sensor IL, USA, Aug. 2014, pp. 6270–6273. applications, dynamics and control, and automotive engineering. [33] F. Firmani, S. N. Robinovitch, and E. J. Park, “Biometric system for measuring gait and fall characteristics captured on video,” ASME J. Biomech. Eng., vol. 136, no. 7, May 2014, Art. ID 071005. Reynald Hoskinson received the B.A. degree from McGill University, Montreal, QC, Canada, the M.Sc. degree in computer science and the Ph.D. degree in electrical and computer engineering from the University of British Columbia, Vancouver, BC, Canada, in 2002 and 2010, respectively. He was a R&D Manager of Recon Instruments Inc., Vancouver, BC, Canada, from 2008 to 2014. He is currently a Research Associate with the Shaghayegh Zihajehzadeh received the B.Sc. degree in electrical engineer- School of Mechatronic Systems Engineering, Simon Fraser University, Surrey, ing/control systems from the Isfahan University of Technology, Isfahan, Iran, BC, Canada. He has been actively involved in wearable technology since 2008. in 2008, and the M.Sc. degree in mechatronics from the Amirkabir University of Technology, Tehran, Iran, in 2011. She is currently pursuing the Ph.D. degree with the School of Mechatronic Systems Engineering, Simon Fraser University, Surrey, BC, Canada. Her research focuses on the development of an accurate and ubiquitous Edward J. Park (M’03–SM’13) received the motion capture system using wearable technology. B.A.Sc. degree from the University of British Columbia, Vancouver, BC, Canada, in 1996, and the M.A.Sc. and Ph.D. degrees from the University of Toronto, Toronto, ON, Canada, in 1999 and 2003, respectively, all in mechanical engineering. He was an Assistant Professor with the Depart- ment of Mechanical Engineering, University of Vic- toria, Victoria, BC, Canada, from 2003 to 2008. Tien Jung Lee received the B.A.Sc. degree He is currently a Professor with the School of in mechatronics from Simon Fraser University, Mechatronic Systems Engineering, Simon Fraser Surrey, BC, Canada, in 2012, where he is currently University, Surrey, BC, Canada, where he is also the Director of the pursuing the M.A.Sc. degree in mechatronics. Biomechatronic Systems Laboratory. His current research interests include His current research interests include feature wearable technologies, biomechatronics and biomedical technologies for extraction, classification, and wearable technology. life sciences, rehabilitation and medicine, and mechatronics applied to next generation vehicular, robotic, and space systems. He has authored or co-authored close to 100 journal and conference papers in these areas. He is also an Associate Dean of Faculty of Applied Sciences and an Associate Member of Faculty Health Sciences.

Appendix C.

A cascaded Kalman filter-based GPS/MEMS-IMU integration for sports applications

(Reproduced with permission from Measurement)

52 Measurement 73 (2015) 200–210

Contents lists available at ScienceDirect

Measurement

journal homepage: www.elsevier.com/locate/measurement

A cascaded Kalman ﬁlter-based GPS/MEMS-IMU integration for sports applications

Shaghayegh Zihajehzadeh a,b, Darrell Loh a,b, Tien Jung Lee a, Reynald Hoskinson a, ⇑ Edward J. Park a, a School of Mechatronic Systems Engineering, Simon Fraser University, 250-13450 102nd Avenue, Surrey, BC V3T 0A3, Canada b Recon Instruments Inc., 1050 Homer Street, Vancouver, BC V6B 2W9, Canada article info abstract

Article history: Nonlinear Kalman filtering methods are the most popular algorithms for integration of a Received 25 September 2014 MEMS-based inertial measurement unit (MEMS-IMU) with a global positioning system Received in revised form 10 May 2015 (GPS). Despite their accuracy, these nonlinear algorithms present a challenge in terms of Accepted 15 May 2015 the computational efficiency for portable wearable devices. We introduce a cascaded Available online 25 May 2015 Kalman filter for GPS/MEMS-IMU integration for the purpose of trajectory determination in sports applications. The proposed algorithm uses a novel orientation filter, cascaded Keywords: with a position/velocity filter. By using cascaded linear Kalman filtering, this method Kalman filter avoids the need to propagate additional states, resulting in the covariance propagation GPS/IMU integration Inertial navigation system to become more computationally efficient for ambulatory human motion tracking. Sports Additionally, the use of this separate orientation filter helps to retain the orientation accuracy during GPS outage. Results of the field experiments reveal that the proposed algorithm is computationally much faster compared to the available non-linear approaches and demonstrates improved trajectory tracking during GPS outages. Ó 2015 Elsevier Ltd. All rights reserved.

1. Introduction alternative for chronometry or video recordings. A number of commercial products are available on the market, for For athletes, access to quantitative performance vari- example, Ripxx, and ShadowBox, provide performance ables can signiﬁcantly improve overall performance, allow- parameters derived from satellite-based position [2]. ing them to share information with coaches, record However, the GPS sampling rate is typically too low to performance over time, and even provide real-time feed- detect complex kinematic motions in sports competitions. back of how they are doing. Currently available Additionally, some other factors such as satellite signal video-based or camera-based motion capture approaches attenuation in athletes’ environment, and the number of provide few quantitative variables and are limited by mete- satellites and their positions may also affect the accuracy orological conditions [1]. Additionally, these techniques are of GPS position/velocity calculation [3]. restricted to the use of pre-set conﬁned areas that are not Alternatively, an inertial navigation system (INS) can be suitable for motion capture of outdoor sports such as skiing, integrated with GPS to detect rapid movements and bridge snowboarding and mountain biking that take place over the gap between GPS blockage periods [4]. An INS is a nav- large distances. igation system based on an inertial measurement unit The global positioning system (GPS), which provides (IMU), which consists of accelerometers, gyroscopes and position and velocity trajectories, might be a good sometimes magnetometers to calculate orientation, position and velocity via dead reckoning [5]. In fact, since its ⇑ Corresponding author. Tel.: +1 778 782 8662. emergence, INS is widely used in various motion capture E-mail address: [email protected] (E.J. Park). (MOCAP) applications such as pedestrian [6], ship [7,8] http://dx.doi.org/10.1016/j.measurement.2015.05.023 0263-2241/Ó 2015 Elsevier Ltd. All rights reserved.

53 S. Zihajehzadeh et al. / Measurement 73 (2015) 200–210 201 aircraft [9–11] land vehicle [12,13] navigation, hand-held operations and more computationally efficient covariance surgical devices [14], fastening tools [15] and pen [16] propagation. Additionally, the use of a separate orientation tracking, in addition to robotics [17–19], to provide naviga- filter results in more robust orientation estimation during tion information such as orientation, velocity, and position. GPS attenuation/blockage periods. The proposed orienta- Recently, miniature MEMS inertial devices have become tion filter uses two linear Kalman filters consisting of a tilt commonplace, resulting in the emergence of INS for angle Kalman filter followed by a yaw angle Kalman filter. human body motion tracking using wearable MOCAP Accurate orientation obtained from the orientation Kalman devices [20]. These devices typically make use of a MEMS filter are fed to a gyroscope error Kalman filter and a nav- inertial measurement unit (MEMS-IMU) and a GPS receiver igation Kalman filter to estimate navigation parameters in for localization. As a result of its more accurate and robust addition to inertial bias errors. MOCAP performance, the GPS/MEMS-IMU integration The remainder of the paper is organized as follows. In algorithms have received increased attention recently for Section 2, theory behind the proposed GPS/IMU fusion trajectory determination in sports. In [1], the performance algorithm for trajectory determination in sports applica- of the two widely-used nonlinear Kalman filtering meth- tions is introduced. The experimental setup and experi- ods, the unscented Kalman filter (UKF) and extended mental protocol are explained in Section 3. In Section 4, Kalman filter (EKF), for GPS/MEMS-IMU integration in the proposed method is assessed based on downhill snow- sport trajectory determination is compared, finding the boarding experiments and benchmarking the results performance of the two algorithms comparable but the against those obtained from tactical grade IMU fused with UKF incurring a higher computational cost. In [2], the EKF real-time kinematic GPS using commercial software. The is utilized for jump trajectory determination in ski and performance of the algorithm under GPS outage intervals mountain biking applications. In [21], a combination of and the computational efficiency of the proposed method GPS, MEMS-IMU and pressure sensitive insoles sensors are also investigated in this section. Finally, Section 5 con- has been used in a fusion algorithm to capture 3-D kinetics cludes the paper. and kinematics of alpine ski racing. Despite their excellent sensor fusion capabilities in 2. Theoretical method terms of accuracy, the above mentioned nonlinear Kalman filtering methods demand relatively high compu- In an INS, the navigation parameters, i.e. position and tational time that is not desirable in low-cost, velocity, are estimated by solving strapdown inertial navi- battery-powered, and lightweight wearable/portable navigation equations through integration of the external accel- gation systems worn by athletes. The model complexity eration [25]. Thus, it is important for the INS to accurately and the large amount of computation required for the estimate external acceleration during dynamic conditions available inertial MOCAP methods are also reported in by subtracting the gravitational acceleration from the [22]. To reduce this high computational cost, [22] proposes accelerometer signal. To realize this, an accurate estima- a two-step Kalman filter for orientation estimation in iner- tion of orientation is needed. Additionally, due to their tial MOCAP applications. However, it does not address the light weight and their fabrication process, MEMS sensors navigation and tracking issues that are highly relevant in have large bias instability and noise, which consequently sports applications. affect the obtained navigation accuracy. Therefore estimat- In addition to high computational cost, the available ing the variations in inertial biases is extremely important GPS/IMU Kalman filter-based fusion approaches rely on in a MEMS-based INS. GPS observations to correct the otherwise drift prone ori- To accurately determine navigation parameters includ- entation calculated by the gyroscope [23]. They make use ing orientation, position and velocity in addition to inertial of the fact that errors in the attitude solution of an INS bias errors, a cascaded Kalman filter fusion algorithm is propagate into errors in velocity. Therefore, errors in atti- proposed here. The cascaded structure simplifies the math- tude can be observed through as independent measure- ematical model by removing the calculations related to ori- ment of velocity under particular motion conditions that entation states in the position/velocity Kalman filter. Thus, include acceleration, which is non-parallel to the velocity the cascade formulation has much less computational vector in the navigation frame [23]. However, the accuracy overhead than a global Kalman filter [26]. Even though and speed of attitude correction by this method depends the cascaded filter is suboptimal, the accuracy is compara- on frequency and magnitude of acceleration maneuvers ble to the global Kalman filter [26]. In addition to the [24]. Additionally, using GPS position and velocity observa- improved computational cost, the cascaded implementa- tions to correct orientation is problematic for long GPS out- tion allows for increased flexibility and easier implementa- age periods. tion and tuning with relatively small performance loss. The To address the shortcomings of available structure of these filters is explained in this section. GPS/MEMS-IMU fusion algorithms mentioned above, this paper introduces a cascaded Kalman filter to integrate GPS/MEMS-IMU for trajectory determination in sports 2.1. Orientation filter applications. This cascaded Kalman filter consists of a separate and novel orientation filter cascaded with a posi- In the orientation filter, Euler angles and the gyro- tion/velocity filter. By using cascaded linear Kalman scope’s bias error are estimated using three cascaded filtering, the proposed method avoids the need to propa- Kalman filters: tilt Kalman filter, yaw Kalman filter and gate additional states, resulting in reduced matrix gyroscope bias error Kalman filter. The orientation of the

54 202 S. Zihajehzadeh et al. / Measurement 73 (2015) 200–210

body frame with respect to a navigation frame (the frame state transition matrix and w1 is the process model noise pointing to local East, North and Up directions) can be rep- vector; z1 is the measurement vector (i.e. the normalized resented by a rotation matrix which maps a vector from measured gravity vector in the sensor frame); C1 is the the body frame to the navigation frame: 3 3 observation matrix; and v1 is the measurement model noise vector. The matrices in (5) and (6) can be cal- n n b x ¼ bR x ð1Þ culated using the following equations [27]: where x is an arbitrary 3 1 vector and the left superscript A1ðk 1Þ¼I33 Dty~Gðk 1Þð7Þ n and subscript b represent the navigation and body frame respectively. nR is the 3 3 rotation matrix expressed as b w1ðk 1Þ¼Dtðx~1ðkÞÞnG ð8Þ [27]: 2 3 cacb casbsc sacc casbcc þ sasc C1ðkÞ¼gI33 ð9Þ 6 7 n 4 5 bR ¼ sacb sasbsc þ cacc sasbcc casc ð2Þ S sb cbsc cbcc v1ðkÞ¼ae ðkÞþnA ð10Þ where c and s are abbreviations for cos and sin respectively; S S S ae ðkÞ¼ a ðkÞ aðkÞð11Þ a (yaw), b (pitch) and c (roll) are the rotation angles about the Z-, Y-, and X- axes of the navigation frame, respectively. SaðkÞ¼c Saþðk 1Þð12Þ Note that the last row of the rotation matrix is the unit a gravity vector expressed in the sensor frame and is inde- where y~G is the 3 3 skew symmetric matrix of bias com- pendent from yaw angle [27]. By estimating this row, roll pensated tri-axial gyroscope measurements; I is identity and pitch (tilt) angles can be determined: matrix; x~1 is the 3 3 skew symmetric matrix of x1; ! ! S n n ae ðkÞ is the external acceleration error in the sensor bR bR c ¼ tan1 3;2 ; b ¼ tan1 3;1 ð3Þ frame; n and n are the gyroscope and accelerometer nR nR = sin c G A b 3;3 b 3;2 measurement noise vectors, which are assumed to be where nR represents the ith row and jth column in rota- uncorrelated and zero-mean white Gaussian; the super- b i;j scripts + and stand for the ‘‘a posteriori’’ and the ‘‘a priori’’ tion matrix nR. Additionally, by estimating the first row b estimates in the Kalman filter, respectively; c is a dimen- in rotation matrix and having the tilt angles, yaw angle a sionless constant between 0 and 1 that determines the (a) can be readily determined by: ! cut-off frequency in the external acceleration model n n S S cc R þ sc R ( aðkÞ¼ca aðk 1ÞþeðkÞ, with eðkÞ being the 1 b 1;2 b 1;3 a ¼ tan n ð4Þ time-varying error of the external acceleration process bR1;1=cb model); Saþðk 1Þ is the gravity compensated external In this work, a two-step Kalman filter is used to deter- acceleration; and g is the norm of gravity vector. mine full 3-D orientation including tilt angles and yaw The measurement vector in Eq. (6) is calculated by [27]: angle. In the first step, which is the tilt Kalman filter, gyro- S þ scope and accelerometer data are used along with an accel- z1ðkÞ¼yAðkÞca a1 ðk 1Þð13Þ eration model to estimate the last row of the orientation S þ þ matrix in order to calculate the tilt angles. In the second a ðkÞ¼yAðkÞgx1 ðkÞð14Þ step, gyroscope and magnetometer data are used along where y (k) is the bias compensated output vector of the with the estimated tilt angles from the first step to accu- A accelerometer. rately estimate the first row of the rotation matrix to deter- The process and measurement noise covariance matri- mine the yaw angle. ces in the tilt Kalman filter, Q1(k 1) and M1(k), are calculated using the following equations [27]: 2.1.1. Tilt Kalman filter The tilt Kalman filter is based on the algorithm pre- T Q 1ðk 1Þ¼E½w1ðk 1Þw1ðk 1Þ sented in our previous work [27], which allows accurate t2x~ k 1 R x~ k 1 15 determination of the roll and pitch angles under dynamic ¼D 1ð Þ G 1ð ÞðÞ conditions. In this method, the measurements from the T tri-axial gyroscopes and tri-axial accelerometers are used M1ðkÞ¼E½v1ðkÞv1ðkÞ ¼Racc þ RA ð16Þ in a Kalman filter to estimate the normalized gravity vector in the sensor frame (i.e. the third row of the rota- where RG is the covariance matrix of the gyroscope’s mea- T n surement noise which is defined as E½nGnG. By assuming tion matrix b R) using the following system model equa- 2 tions [27]: that the gyroscope noise variances are equal to rG in the 2 three axes, RG is set to rGI3. Similar to RG, RA which is x1ðkÞ¼A1ðk 1Þx1ðk 1Þþw1ðk 1Þð5Þ the covariance matrix of accelerometer’s measurement 2 noise, is set to r I3. Racc is the covariance of the accelera- z k C k x k k 6 A 1ð Þ¼ 1ð Þ 1ð Þþv1ð ÞðÞ 1 2 S þ 2 tion model and is set to 3 ca k a ðk 1Þk . T In Eqs. (5) and (6), x1ðkÞ¼½sb cbsc cbcc is the 3 1 Using the ‘‘a posteriori’’ estimate of normalized gravity þ T state vector for the tilt Kalman filter at step k; A1 is the vector in the sensor frame, x1 ðkÞ¼½x1;x x1;y x1;z , the best

55 S. Zihajehzadeh et al. / Measurement 73 (2015) 200–210 203

+ + estimate of roll (c ) and pitch (b ) angles are calculated To calculate the measurement vector z2(k), roll, pitch using (3): and yaw angles at each step k are required. However, ! ! at each step, roll and pitch are already known from the xþ xþ + þ 1 1;y þ 1 1;x tilt Kalman filter. Since these tilt angles (c (k) and c ðkÞ¼tan þ ; b ðkÞ¼tan þ ð17Þ + x1;z x1;y= sin c b (k)) are used in calculation of z2(k) for the measurement update of yaw Kalman filter, they are called ‘mea- As illustrated in Fig. 1, these two tilt angles are the sured tilt angles’ and are denoted by bm and cm. To get inputs to the second step, i.e. the yaw Kalman filter. the ‘measured yaw angle’, am, for complete calculation of z2(k), magnetometer data is used in the following 2.1.2. Yaw Kalman filter algorithm: The yaw angle Kalman filter allows accurate determination of yaw angle under temporary ferromagnetic distur- (1) The rotation matrix with respect to local horizontal bances. In this Kalman filter, the measurements from n plane, b Rc ;b , is calculated using the known tilt tri-axial gyroscopes and tri-axial magnetometers and the m m angles: known tilt angles are used to estimate the first row of 2 32 3 the rotation matrix nR. The system model equations are cosb 0 sinb 1 0 0 b 6 m m 76 7 represented by: nR ¼ 4 0 1 0 54 0 cosc sinc 5 b cm;bm m m x2ðkÞ¼A2ðk 1Þx2ðk 1Þþw2ðk 1Þð18Þ sinbm 0 cosbm 0 sincm coscm ð23Þ z2ðkÞ¼C2ðkÞx2ðkÞþv2ðkÞð19Þ (2) The calibrated output vector of the magnetometer, T where x2ðkÞ¼½cacb casbsc sacc casbcc þ sasc is the yM, is rotated using the horizontal rotation matrix 3 1 state vector for heading angle Kalman filter at step nR . The rotated vector is expressed by tilty . b cm;bm M k and A2, w2 and C2 can be calculated similar to the ones The horizontal component of the rotated vector is for tilt Kalman filter as: located in the horizontal plane (East–North plane) of the navigation frame. A2ðk 1Þ¼I33 Dty~Gðk 1Þð20Þ tilty ¼ nR y ð24Þ M b cm;bm M w2ðk 1Þ¼Dtðx~2ðkÞÞnG ð21Þ (3) The measured yaw angle, am, is calculated as the tilt angle between the horizontal component of yM C2ðkÞ¼I33 ð22Þ and that of the Earth’s magnetic field.

Orientaon ﬁlter Gyro bias error KF1: Tilt Kalman

Gyroscope Subtract Time update bias error Measurement KF3: Gyro bias error Subtract update Accelerometer Kalman ﬁlter bias error Time update

KF2: Yaw Kalman Measurement update Accelerometer bias error Time update

Measurement Magnetometer Validaon update Rotaon matrix Correct for calculaon bias and scale

posion/velocity ﬁlter

Fig. 1. Overview of the proposed orientation ﬁlter.

56 204 S. Zihajehzadeh et al. / Measurement 73 (2015) 200–210

observation vector in the gyroscope bias error estimation Then, z2(k) can be calculated as: Kalman filter. This observation vector is calculated by the T z2ðkÞ¼½camcbm camsbmscm samccm camsbmccsamscm following steps: ð25Þ (1) x1 ðkÞ and x2 ðkÞ which are ‘‘a priori’’ estimates in tilt However, under temporary magnetic disturbances, the and yaw Kalman filters respectively, are calculated magnetometer output vector cannot be trusted as reliable using the system model Eqs. (5) and (18). information. To avoid the effect of disturbed magnetic field However, instead of bias corrected output of gyro- on the estimated yaw angle, a threshold-based switching scope yG, the bias contaminated output vector of approach similar to the method in [26] is employed. In gyroscope yG,b (refer to Fig. 1), is used in Eqs. (7) the threshold-based switching method, the measured and (20) to calculate A1 and A2. magnetic field is tested for significant deviations from (2) Using Eqs. (17) and (28), and substituting x1 ðkÞ and the local earth’s magnetic field. This validation test xðkÞ, a(k), b(k) and c(k) are calculated. excludes the disturbed measurement by assigning a large 2 (3) z3(k) is calculated as: measurement noise value in the yaw Kalman filter. As a T result, M2(k), the measurement noise covariance matrix hðkÞ¼½cðkÞ bðkÞ aðkÞ ; for the yaw Kalman filter is defined as: T ð33Þ ( hþðkÞ¼ cþðkÞ bþðkÞ aþðkÞ 2 rMI3 kyMðkÞhk < eM \jhdip hdip;t¼0j < edip M2ðkÞ¼ 1 Otherwise hðkÞhþðkÞ z3ðkÞ¼ ð34Þ ð26Þ Dt where r2 is the magnetometer noise variance in each axis. M As shown in Fig. 1, the estimated gyroscope bias error at h is the current dip angle (i.e. the angle formed by the dip step k 1 is used to compensate gyroscopes’ output vector gravity vector and current magnetic field vector) and at step k. The dotted lines in Fig. 1 represent the states h is its initial value. e and e are the threshold values dip,t=0 M dip from step k 1, used in step k. determined by experiment. Finally, Q (k) is defined similar to Q (k): 2 1 2.2. Position/velocity filter 2 Q 2ðk 1Þ¼Dt x~2ðk 1ÞRGx~2ðk 1Þð27Þ Position, velocity and accelerometers’ bias error are Using the ‘‘a posteriori’’ estimate of x2(k) from yaw n estimated in this subsystem using rotation matrix bR from þ T Kalman filter, x2 ðkÞ¼½x2;xx2;yx2;z , the best estimate for the orientation filter, accelerometer output vector, GPS + yaw angle (a ) can be calculated as: position and velocity data. The following model equations ! are used in design of the position/velocity Kalman filter: ccxþ þ scxþ aþðkÞ¼tan1 2;y 2;z ð28Þ þ x4ðkÞ¼A4ðk 1Þx4ðk 1ÞþB4ðk 1Þu4ðk 1Þþw4ðk 1Þ x2;x=cb ð35Þ The structure of the proposed orientation algorithm is shown in Fig. 1. As illustrated in Fig. 1, estimated Euler z4ðkÞ¼C4ðkÞx4ðkÞþv4ðkÞð36Þ angles in the orientation filter becomes an input to the n n T gyroscope bias error Kalman filter and position/velocity where x4ðkÞ¼½r ðkÞ v ðkÞ bAðkÞ is the 9 1 state vec- Kalman filter. tor containing position and velocity vector in the navigation frame in addition to accelerometer bias error. 2.1.3. Gyroscope bias error Kalman filter A4(k 1) and B4(k 1) are the state transition and The gyroscope’s bias error is modeled as a random walk input matrices for the position/velocity subsystem; process [25] using the following system model equations: u4(k 1) is the input vector, being the gravity compensated acceleration in the navigation frame; w4(k 1) is x3ðkÞ¼A3ðk 1Þx3ðk 1Þþw3ðk 1Þð29Þ the 9 1 vector of process noise; z4(k) is the measurement vector from GPS; C4(k) is the observation matrix z3ðkÞ¼C3ðkÞx3ðkÞþv3ðkÞð30Þ and v4(k) is the measurement noise. These variables are calculated as follows: 2 3 where x3(k) is the 3 1 vector of gyroscope bias error at 1 I33 DtD 033 step k; w3(k) and v3(k) are process and measurement noise 6 7 A ðk 1Þ¼4 ~ n ~ n n 5 vectors; A3 and C3 are: 4 033 I33 Dtð2xie þ xneÞDtbR

033 033 I33 A3ðk 1Þ¼I33 ð31Þ ð37Þ 2 3 C3ðkÞ¼I33 ð32Þ 1 ðNþhÞcosu 00 The difference between the corrected orientation vector 1 6 7 D ¼ 4 0 1 0 5 ð38Þ (output of tilt and yaw Kalman ﬁlter) and the one calcu- Mþh lated from gyroscopes’ output vector is used as the 001

57 S. Zihajehzadeh et al. / Measurement 73 (2015) 200–210 205 2 3 1 Dt2I MTi-G-700 (MEMS-IMU) at the rate of 100 Hz. Since the 6 2 33 7 Xsens sensor does not output raw position/velocity mea- B4ðk 1Þ¼4 DtI33 5 ð39Þ surements from its built-in GPS receiver, the raw GPS data 03 3 is collected from TI WL1281 (L1-GPS having positioning accuracy of less than 15 m) at the rate of 1 Hz. The u ðk 1Þ¼nRyðkÞþgn ð40Þ 4 b A built-in GPS in the Xsens sensor is used for synchronization of the IMU and L1-GPS. GPS GPS T z4ðkÞ¼½r ðkÞv ðkÞ ð41Þ The reference system was a Novatel SPAN system consisting of IMU-FSAS (tactical grade IMU with fiber optic I33 033 033 gyroscopes and servo accelerometers), SPAN SE receiver C4ðkÞ¼ ð42Þ 033 I33 033 and L1/L2 antenna. The highest level of accuracy in the reference system is achieved by the use of real-time kine- where M and N are the principle ellipsoidal radii of curva- matic (RTK) corrections and post processing with inertial ture, h is the altitude and u the geographic latitude. xn is ie explorer (IE) software. The reference system provides posi- the rotation rate of the Earth expressed in the navigation tion, velocity and orientation data at the rate of 100 Hz. n frame and xne the rotation rate of the local geographic n Orientation accuracy of the reference system is 0.008 frame with respect to the Earth fixed frame and g stands degrees for the roll and pitch angles and 0.023 degrees for gravity in the navigation frame. for the yaw angle and its positioning accuracy is 1 cm. The process and measurement noise covariance matri- The MTi-G-700 is attached on top of the IMU-FSAS ces in the Kalman filter, Q4(k 1) and M4(k), are as follows using double-sided adhesive tape and the whole system [1,2]: 2 3 is fixed to an aluminum frame (Fig. 3). This frame goes into a backpack that is worn by subjects. hiRr 033 033 T 6 7 Eventually, our goal is to implement the proposed algo- Q 4ðk 1Þ¼E w4ðk 1Þw4ðk 1Þ ¼ 4 033 Rv 033 5 rithm into active sports goggles, having built-in onboard 0 0 R 33 33 b processor and inertial, magnetic and GPS sensors, in order ð43Þ to determine an athlete’s motion trajectory in skiing, snowboarding, or biking. Recon Instruments Inc., a Rr:GPS 033 Vancouver (British Columbia, Canada) company, has M4ðkÞ¼ ð44Þ 033 Rv:GPS recently developed and commercialized such active sports goggles. where Rr, Rv and Rb are diagonal covariance matrices of position, velocity and accelerometer bias error respectively. Rr,GPS and Rv,GPS are the covariance matrices of 3.2. Experimental protocol GPS position and velocity measurement. The structure of the position/velocity Kalman filter is shown in Fig. 2.As Nine downhill snowboarding runs were collected at it is illustrated in this figure, the estimated accelerometer Cypress Mountain, British Columbia, Canada. Trajectory bias error is fed back to the orientation Kalman filter. of a sample run collected by the reference system is shown More explanation on the above mentioned navigation equation can be found in [1,2,24].

3. Experimental setup and protocol

3.1. Experimental setup

In order to evaluate the performance of a low-cost GPS/IMU system with the proposed integration strategy, raw inertial and magnetic data is collected from an Xsens

Orientaon Filter

Posion/velocity ﬁlter Accelerometer bias KF4: Ineral navigaon error Kalman ﬁlter

Accelerometer Time update

GPS Measurement update Posion, Velocity

Fig. 2. Overview of the proposed position/velocity ﬁlter. Fig. 3. Experimental setup.

58 206 S. Zihajehzadeh et al. / Measurement 73 (2015) 200–210 on Google Earth in Fig. 4. Each run takes approximately 1.5 min.

3.3. Participants

Three young healthy subjects (all male), ranging in age between 20 and 30 years, participated in the experiment. All participants were students at Simon Fraser University, recruited through advertisements posted on university notice boards. Each provided an informed written consent, and the experimental protocol was approved by the Research Ethics Board at Simon Fraser University.

4. Experimental results and discussion

This section is divided into three parts. In part (a), the accuracy of the proposed method for orientation tracking Fig. 5. Orientation results during a typical run. during the snowboarding experiment is investigated. Part (b) and (c) examine the accuracy of the low-cost GPS/MEMS-IMU system using the proposed fusion algo- roll and pitch angles and less than 3° for the heading angle. rithm in position and velocity tracking under full GPS The higher error observed in the heading angle compared reception and GPS blockage, respectively. Finally, part (d) to the roll and pitch angles is due to the fact that, in addi- evaluates the computational efﬁciency of the proposed tion to other error sources, the heading is determined by method. using both the roll and pitch angles, thus it gets affected by their respective errors. Less accuracy for yaw angle esti- 4.1. Orientation tracking accuracy mation comparing to the roll and pitch angles has been reported in [22] as well. Comparing to the latest orienta- The accuracy of the proposed orientation algorithm tion algorithm in the literature [22], our proposed orienta- against electromagnetic disturbances and also large exter- tion method provides similar accuracies. However, the nal accelerations has already been investigated in our pre- proposed algorithm has the advantage of retaining its vious work [28]. Thus, the main purpose of this section is accuracy during temporary electromagnetic disturbances to evaluate the orientation tracking accuracy speciﬁcally [28]. during our targeted sport activities. The estimated roll, pitch and heading angles during the snowboarding exper- 4.2. Position and velocity tracking under full GPS reception iment along with the reference angles are shown in Fig. 5. The orientation results presented in Fig. 5 suggest that the To accurately track position and velocity, the external estimated orientation is reasonably accurate during the acceleration (i.e. the gravity compensated acceleration) entire test duration and no orientation drift is observed. needs to be determined accurately. Figs. 7 and 8 show Fig. 6 presents the orientation root-mean-square how accurately the algorithm can track the external accel- error (RMSE) for the nine downhill snowboarding runs. eration. The horizontal trajectory and East–North velocities The resulting orientation accuracy is less than 2° for the of a sample run are shown in Figs. 9 and 10, respectively. The position and velocity RMSE averaged over the nine

Fig. 4. Position trajectory of a typical run shown on Google earth. Fig. 6. Orientation RMSE and standard deviation.

59 S. Zihajehzadeh et al. / Measurement 73 (2015) 200–210 207 downhill snowboarding runs are shown in Table 1. The reported accuracies in Table 1 are close to the ones reported in [12] (horizontal position and velocity error of about 3 m and 0.2 m/s, respectively), which employed a nonlinear Kalman ﬁlter. The achieved position and velocity tracking accuracy without satellite masking is mainly determined by the accuracy of the L1 GPS, and the IMU and sensor fusion algorithm are less effective in providing an accuracy improvement. Thus, further improvements in accuracy could be obtained with a more accurate GPS system such as the real-time kinematic (RTK) GPS [12]. For further comparison purposes, the vertical trajectory of the same run is shown in Fig. 11. The vertical trajectory error over the nine downhill snowboarding runs is 3.86 m, which is more than twice larger than the errors in the horizontal direction. This is due to the fact that the GPS-derived vertical position information is much less accurate than the horizontal [29]. The most important rea- Fig. 9. East and North velocity. son behind this is satellite visibility (i.e. few visible satellites over the horizon). In order to achieve higher accuracies in the vertical direction, the integration of a MEMS barometric pressure sensor into the

Fig. 10. Vertical position trajectory.

Fig. 7. External acceleration during a typical run. Table 1 Position and velocity RMSE with full GPS reception.

Position (m) Velocity (m/s) East North East North 1.53 1.27 0.35 0.21

GPS/MEMS-IMU system in sports applications was investigated in our very recent study [30].

4.3. Performance under GPS blockage

Since a barometric pressure sensor can completely bridge the GPS blockage gaps in the vertical direction, the performance under GPS blockage is only evaluated in the horizontal direction. To investigate performance of the proposed algorithm under GPS blockage, a 30 s interval of a complete GPS blockage is simulated in the middle of the runs and a typical horizontal position tracking result Fig. 8. Position trajectory in the East–North plane. is shown in Fig. 11. The trajectory depicted in Fig. 11

60 208 S. Zihajehzadeh et al. / Measurement 73 (2015) 200–210

Fig. 13. Comparison of processing time for 16,000 data points using the proposed cascaded Kalman ﬁlter, EKF and UKF. Fig. 11. Position trajectory in the East–North plane with 30 s of GPS outage.

outage of 5 s remains less than 2 m. Thus, our investigation here reveals that the proposed filter is capable of bridging typical outages without a significant loss of the navigation accuracy. As a result, it can be concluded that MEMS-IMU measurements can be integrated with consumer-grade GPS to obtain sufficiently accurate results during GPS outages up to a few seconds. There are some recently published artificial intelligence-based GPS/IMU fusion methods for land vehicle navigation [32], which provide higher accuracies during GPS outages than the achieved accuracy using the proposed method. However, the non-holonomic constraints in the land vehicle navigation (i.e. the lateral and vertical velocities are zero if the vehicle is not skidding) help to reduce drift during GPS outage [31], which is not applicable to trajectory tracking in highly dynamic sport activities. Nonetheless, our target application is outdoor sports such as skiing, snowboarding and mountain biking, which mostly happens in open mountainous areas for which Fig. 12. Position RMSE versus GPS outage duration. GPS signal blockage/attenuation of longer than 5 s is less probable. Thus, the computational efficiency required for light weight battery powered wearable sensors is deemed illustrates that even in the presence of 30 s of GPS outage more important than their tracking performance under (shown by green dots), the trajectory does not diverge prolonged GPS outage durations. considerably. Fig. 12 shows the position RMSE for various satellite 4.4. Computational cost outage durations (from 5 s to 30 s). It also illustrates that with a complete GPS blockage of up to 30 s, the position In this part, the computational efficiency of the pro- tracking accuracy remains less than about 6 m and 5 m posed cascaded linear Kalman filter is compared against for East and North directions, respectively. This is a slight the two well-known nonlinear Kalman filtering methods improvement over the reported RMSE of about 10 m and for navigation, i.e. EKF and UKF. The dynamic model and 5 m in East and North directions obtained using a nonlin- system equations used for EKF and UKF are the ones pro- ear extended Kalman filter method during 30 s of GPS out- posed in [24]. The device used to process the recorded sen- age [1]. This improvement is due to the use of a separate sor data is a desktop computer with Intel Core i7 processor. orientation filter, which helps to retain the orientation Fig. 13 compares the processing time (which is an indica- accuracy during short GPS outages. tion of the power consumption) of the three sensor fusion According to [31], GPS outages of up to 5 s are frequent methods for 16,000 data points. This figure shows that the in sport applications, and indeed, the most outages are less computational time of the UKF is longer than the EKF’s, than 15 s (e.g., in alpine skiing through satellite masking which agrees with the observation made in [1]. due to the changing surrounding). According to Fig. 12, Moreover, however, it shows that the proposed method the accuracy of horizontal trajectory tracking for GPS is 70% and 80% faster than the EKF and UKF, respectively.

61 S. Zihajehzadeh et al. / Measurement 73 (2015) 200–210 209

This can be physically justified by the linear structure of [3] Paul D. Groves, Principles of GNSS, Inertial, and Multisensor the propose method for the GPS/MEMS-IMU integration, Integrated, Artech House Publishers, 2008. [4] L. Li, Y. Pan, J. Lee, C. Ren, Y. Liu, D.A. Grejner-brzezinska, C.K. Toth, in comparison to the nonlinear structure of the EKF and Cart-mounted geolocation system for unexploded ordnance with UKF. Additionally, in our proposed method, the global adaptive ZUPT assistance, IEEE Trans. Instrum. Meas. 61 (4) (2012) covariance matrix propagated in the EKF or UKF is divided 974–979. [5] K. Parsa, T.A. Lasky, B. Ravani, Design and implementation of a into smaller-sized covariance matrices being propagated in mechatronic, all-accelerometer inertial measurement unit, IEEE/ the individual subsystems. The smaller-sized matrices in ASME Trans. Mech. 12 (6) (2007) 640–650. the proposed method require less calculation than the [6] H. Kuusniemi, J. Liu, L. Pei, Y. Chen, L. Chen, R. Chen, Reliability considerations of multi-sensor multi-network pedestrian navigation, EKF or UKF. IET Radar, Sonar Navig. 6 (3) (2012) 157. [7] X. Liu, X. Xu, Y. Liu, L. Wang, A fast and high-accuracy transfer alignment method between M/S INS for ship based on iterative 5. Conclusion calculation, Measurement 51 (2014) 297–309. [8] B. Wang, Z. Deng, C. Liu, Y. Xia, M. Fu, Estimation of information This work proposes the use of a cascaded Kalman filter sharing error by dynamic deformation between inertial navigation systems, IEEE Trans. Ind. Electron. 61 (4) (2014) 2015–2023. for integration of a low-cost GPS/MEMS-IMU system. Its [9] S. Lailiang, Z. Chunxi, C. Daihong, An elastic deformation direct application is the trajectory determination in sports measurement method for helicopter based on double-IMUs/DGPS such as snowboarding, skiing, or mountain biking. The pro- TRAMS, Measurement 46 (5) (2013) 1704–1714. posed algorithm uses a separate and novel orientation [10] K.D. Sebesta, N. Boizot, A real-time adaptive high-gain EKF, applied to a quadcopter inertial navigation system, IEEE Trans. Ind. Electron. Kalman filter to correct orientation and estimate gyro- 61 (1) (2014) 495–503. scope’s bias error employing accelerometer and magne- [11] F. Jiancheng, Y. Sheng, Study on innovation adaptive EKF for in-flight tometer. This orientation filter is cascaded with a alignment of airborne POS, IEEE Trans. Instrum. Meas. 60 (4) (2011) 1378–1388. position/velocity filter for navigation and estimation of [12] T. Zhang, X. Xu, A new method of seamless land navigation for GPS/ accelerometer’s bias error. Our experimental results for a INS integrated system, Measurement 45 (4) (2012) 691–701. number of downhill snowboarding runs show that the pro- [13] J. Huang, H.S. Tan, A low-order DGPS-based vehicle positioning system under urban environment, IEEE/ASME Trans. Mech. 11 (5) posed algorithm is 70% and 80% faster than the two com- (2006) 567–575. mon nonlinear Kalman filter-based sensor fusion [14] H. Ren, P. Kazanzides, Investigation of attitude tracking using an methods, i.e. the EKF and UKF, respectively without sacri- integrated inertial and magnetic navigation system for hand-held surgical instruments, IEEE/ASME Trans. Mech. 17 (2) (2012) 210– ficing the tracking accuracy. This computational efficiency 217. happens as a result of reduced matrix operations in its cas- [15] S.P. Won, F. Golnaraghi, W.W. Melek, A fastening tool tracking caded structure is desirable for onboard processing in system using an IMU and a position sensor with Kalman filters and a fuzzy expert system, IEEE Trans. Ind. Electron. 56 (5) (2009) 1782– wearable systems. 1792. Compared to the commercially available video-based or [16] J.S. Wang, Y.L. Hsu, J.N. Liu, An inertial-measurement-unit-based pen camera-based systems, the proposed trajectory tracking with a trajectory reconstruction algorithm and its applications, IEEE system based on GPS/MEMS-IMU fusion has lower cost Trans. Ind. Electron. 57 (10) (2010) 3508–3521. [17] S. Panzieri, F. Pascucci, G. Ulivi, An outdoor navigation system using and can provide a greater range of useful quantitative vari- gps and inertial platform, IEEE/ASME Trans. Mech. 7 (2) (2002) 134– ables for sports applications. Additionally, the system is 142. not limited to a confined space, making it suitable for out- [18] G. Panahandeh, M. Jansson, Vision-aided inertial navigation based on ground plane feature detection, 19(4) (2014) 1206–1215. door sport activities that happen over large distances such [19] F. Aghili, A. Salerno, Driftless 3-D attitude determination and as skiing and snowboarding. Compared to the available positioning of mobile robots by integration of IMU with two RTK GPS-based tracking devices, the proposed fused system GPSs, IEEE/ASME Trans. Mech. 18 (1) (2013) 21–31. [20] T. Liu, Y. Inoue, K. Shibata, K. Shiojima, M.M. Han, Triaxial joint can maintain tracking of the trajectory during short-term moment estimation using a wearable three-dimensional gait GPS outages. The experimental results show that the pro- analysis system, Measurement 47 (2014) 125–129. posed method can successfully bridge the frequent GPS [21] M. Brodie, A. Walmsley, W. Page, Fusion motion capture: a prototype system using inertial measurement units and GPS for the outage periods of 5 s in skiing and snowboarding with biomechanical analysis of ski racing, Sport. Technol. 1 (1) (2008) the horizontal position accuracy of less than 2 m. 17–28. Additionally, with a complete GPS blockage of up to 30 s, [22] M. Jin, J. Zhao, J. Jin, G. Yu, W. Li, The adaptive Kalman filter based on fuzzy logic for inertial motion capture system, Measurement 49 the position tracking accuracy remains less than about (March) (2014) 196–204. 6 m and 5 m for East and North directions, respectively. [23] D. Dusha, L. Mejias, Error analysis and attitude observability of a monocular GPS/visual odometry integrated navigation filter, Int. J. Rob. Res. 31 (6) (2012) 714–737. Acknowledgement [24] E. Foxlin, Pedestrian tracking with shoe-mounted inertial sensors, IEEE J. Comput. Graph. Appl. 25 (6) (2005) 38–46. This work was supported in part by the Natural [25] R. Van Der Merwe, E. Wan, Sigma-point Kalman filtering for integrated GPS and inertial navigation, IEEE Trans. Aerosp. Sciences and Engineering Research Council of Canada Electron. Syst. 42 (2) (2006) 750–756. (NSERC). [26] Z. Lendek, R. Babuška, B. De Schutter, Distributed Kalman filtering for cascaded systems, Eng. Appl. Artif. Intell. 21 (3) (2008) 457–469. [27] J.K. Lee, E.J. Park, S. Robinovitch, Estimation of attitude and external References acceleration using inertial sensor measurement during various dynamic conditions, IEEE Trans. Instrum. Meas. 61 (8) (2012) [1] A. Waegli, J. Skaloud, Optimization of two GPS/MEMS-IMU 2262–2273. integration strategies with application to sports, GPS Solut. 13 (4) [28] S. Zihajehzadeh, D. Loh, M. Lee, R. Hoskinson, E.J. Park, S. Member, A (2009) 315–326. cascaded two – step Kalman filter for estimation of human body [2] F. Sadi, R. Klukas, New jump trajectory determination method using segment orientation using MEMS – IMU, in: 36th Annual Inter. Conf. low-cost MEMS sensor fusion and augmented observations for GPS/ of the IEEE Eng. in Medicine and Biology, Aug. 2014. INS integration, GPS Solut. 17 (2) (2012) 139–152.

62 210 S. Zihajehzadeh et al. / Measurement 73 (2015) 200–210

[29] O. Svabensky, J. Weigel, R. Machotka, On GPS heighting in local [31] A. Waegli, Trajectory Determination and Analysis in Sports by networks, Acta Geodyn. Geomater. 3 (143) (2006) 39–43. Satellite and Inertial Navigation, Ph.D. Dissertation, Swiss Federal [30] S. Zihajehzadeh, T.J. Lee, J.K. Lee, R. Hoskinson, E.J. Park, Integration Institute of Technology, Lausanne, 2009. of MEMS inertial and pressure sensors for vertical trajectory [32] X. Chen, C. Shen, W. Zhang, M. Tomizuka, Y. Xu, K. Chiu, Novel hybrid determination, IEEE Trans. Instrum. Meas. 64 (3) (2015) of strong tracking Kalman ﬁlter and wavelet neural network for GPS/ 804–814. INS during GPS outages, Measurement 46 (10) (2013) 3847–3854.

Appendix D.

UWB-aided inertial motion capture for lower body 3-D dynamic activity and trajectory tracking

(Reproduced with permission from IEEE)

64 IEEE TRANSACTIONS ON INSTRUMENTATION AND MEASUREMENT, VOL. 64, NO. 12, DECEMBER 2015 3577 UWB-Aided Inertial Motion Capture for Lower Body 3-D Dynamic Activity and Trajectory Tracking Shaghayegh Zihajehzadeh, Student Member, IEEE, Paul K. Yoon, Student Member, IEEE, Bong-Soo Kang, Member, IEEE, and Edward J. Park, Senior Member, IEEE

Abstract— This paper introduces a novel method for simul- activities of the upper body [6], [7] or lower body [8]. taneous 3-D trajectory tracking and lower body motion cap- Although wearable miniature IMU can give suitable orien- ture (MoCap) under various dynamic activities such as walking tation information required for typical MoCap applications, and jumping. The proposed method uses wearable inertial sensors fused with an ultrawideband localization system using a cas- they suffer from an accumulated drift over time if used for caded Kalman filter-based sensor fusion algorithm, which consists localization purposes [9]. The most common method to reduce of a separate orientation filter cascaded with a position/velocity this drift in self-contained wearable inertial sensors for human filter. In addition, to further improve the joint angle tracking body tracking is considering the ground contact constraint, i.e., accuracy, anatomical constraints are applied to the knee joint. the velocity and vertical position of the foot contacting the Currently, available self-contained inertial tracking methods are not only drift-prone over time but also their accuracy is degraded ground being zero in the global reference frame [10]. under fast movements with unstable ground contact states due to To further reduce the drift in self-contained inertial tracking the lack of external references. However, our experimental results, systems, some researchers have recently employed anatomical which benchmark the system against a reference camera-based constraints of the human body [11] and the stride infor- motion tracking system, show that the proposed fusion method mation [9]. Furthermore, the use of force sensors has been can accurately capture the dynamic activities of a subject without drift. The results show that the proposed system can maintain proposed in [12] for more accurate determination of the ground similar accuracies between fast and slow motions in lower body contacts of the feet. Although these existing methods in [12] MoCap and 3-D trajectory tracking. The obtained accuracies of can significantly reduce the drift rate in localization, drift the system for 3-D body localization and knee joint angle tracking errors still accumulate over time. Another problem associated for fast motions were less than 5 cm and 2.1°, respectively. with ground contact-based inertial tracking systems is their Index Terms— Inertial measurement unit (IMU), inertial loss of accuracy under highly dynamic activities. For instance, motion tracking, Kalman filter (KF), microelectromechanical higher inaccuracies have been reported for running and jump- systems (MEMS), ultrawideband (UWB) localization, wearable ing in [13] and hopping in [14] compared with walking. technology. This is due to the large body accelerations generated in such I. INTRODUCTION activities causing errors in the orientation estimation, which IMULTANEOUS trajectory tracking and motion cap- in turn affects the position estimation [14]. In addition, these Sture (MoCap) of human body segments is important in ground contact-based methods may suffer from unbounded many applications such as sports science [1], [2], human–robot drift in highly complex motions such as acrobatic routines interactions in industrial environments [3], and rehabilita- where ground contacts are less frequent and, when available, tion [4] where continuous monitoring of human subjects’ shorter in duration. activities is useful. Recently, with the advent of the micro- On the other hand, vision-based [15]–[18] and radio fre- electromechanical systems (MEMS) technology, miniature quency (RF)-based indoor localization systems [19], [20] inertial measurement units (IMUs) have emerged for wearable can provide accurate and drift free location data over time. MoCap applications [5] for ambulatory monitoring of physical However, the vision-based positioning systems, although low in cost, are not reliable in dynamically changing environ- Manuscript received February 26, 2015; revised June 22, 2015; accepted ments [19]. They are affected by lighting conditions and June 23, 2015. Date of publication August 10, 2015; date of current version cannot be easily adopted to track multiple subjects [19]. November 6, 2015. This work was supported by the Natural Sciences and Engineering Research Council of Canada. The Associate Editor coordinating RF-based localization systems such as the ones based on the review process was Dr. Jesus Urena. radio-frequency identification (RFID) [21], WiFi [22], and S. Zihajehzadeh, P. K. Yoon, and E. J. Park are with the School of Mecha- Bluetooth [19] suffer from multipath effects of the reflected tronic Systems Engineering, Simon Fraser University, Surrey, BC V3T 0A3, Canada (e-mail: [email protected]). signals from the walls. Amongst various RF-based systems, B.-S. Kang is with the Department of Mechanical Engineering, Hannam ultrawideband (UWB) technology has the ability to distin- University, Daejeon 306-791, Korea. guish between the original and reflected signals using narrow Color versions of one or more of the figures in this paper are available online at http://ieeexplore.ieee.org. pulses (less than 1 ns). In addition to excellent precision Digital Object Identifier 10.1109/TIM.2015.2459532 and reasonable cost, the UWB system can provide relatively 0018-9456 © 2015 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information. 65 3578 IEEE TRANSACTIONS ON INSTRUMENTATION AND MEASUREMENT, VOL. 64, NO. 12, DECEMBER 2015 large coverage range (about 400 m2) and its tracked tags, position/velocity trajectory and lower body kinematics under which are easily wearable due to their small size and weight, dynamic conditions, the performance of an MEMS-IMU inte- have ultralow power consumption [19], [23]. Consequently, grated with an UWB localization system is investigated in this UWB localization systems have been popular in tracking paper. A loosely coupled cascaded two-step Kalman filter (KF) people [24], [25] and robots [26]. However, materials like is used to fuse IMU measurements for orientation estimation. metal and liquid can cause interferences in the UWB radio The orientation KF is then cascaded with a position/velocity signals [20]. Thus, to increase the trajectory tracking accuracy KF for UWB-aided 3-D inertial tracking. Unlike the existing for human motion tracking applications, some researchers contact-based methods, the novelty of the proposed system is have fused UWB systems with inertial sensors using tightly that it has high 3-D trajectory tracking and MoCap accuracy coupled [27] or loosely coupled [3] methods. In the tightly under dynamic conditions and is capable of maintaining it over coupled approach, the UWB time of transmission is used in time. Furthermore, the algorithm has the potential to capture the fusion filter, whereas in the loosely coupled approach, the complex human motions with less frequent or no ground transmission time is preprocessed to calculate position and contacts, as well as vertical motions that are more difficult the final position data are used in the fusion filter. In this to track accurately even with the UWB/IMU fusion. fusion, the accurate and drift free UWB measurements enable The remainder of this paper is organized as follows. the correction of drift errors in inertial tracking and the high In Section II, the theory behind the orientation and sampling rate of the IMU enables the overall system to give a 3-D trajectory estimation algorithm is explained. The lower continuous trajectory that bridges any short-term interferences body skeleton model and anatomical constraints are also on the radio signals. Similar to the GPS/IMU integration [28], explained in this section. The experimental setup and experi- a tightly coupled UWB/IMU fusion might be advantageous mental protocol are explained in Section III. The experimental under UWB outages for outliers detection [27] by having results consisting of 3-D trajectory determination of the root the ability to carry out UWB measurement updates, even if joint (waist) and knee joint angle tracking during walking as there are not sufficient time difference of arrival (TDOA) mea- an example of slow motion and also jumping as an example surements for triangulation [27]. However, since UWB-based of highly dynamic/fast motion are presented in Section IV. localization has no line-of-sight (NLOS) requirements [19], Finally, Section V concludes this paper. UWB outages are less probable in a dedicated MoCap room in which unnecessary obstacles are avoided. In addition, when II. THEORETICAL METHOD a combination of TDOA and angle of arrival are used to In this section, the proposed trajectory tracking and lower calculate the final position, the number of UWB sensors body MoCap algorithms are described. The purpose of the required for localization can be reduced [20]. Furthermore, trajectory tracking algorithm is to track 3-D position and the loosely coupled approach offers simpler implementation. velocity of the waist, which is considered as the root joint in Another important issue in UWB positioning systems, how- our biomechanical model, while the purpose of the lower body ever, is that the UWB-derived vertical position tends to be MoCap algorithm is to extract orientation of the lower body less accurate than the horizontal position [27]. One of the segments for full representation and biomechanical analysis of major reasons behind this is the geometry of UWB sensors lower body dynamic activities. and the fact that they are installed almost on the same horizontal plane [29]. With the limited number of UWB sensors, A. Trajectory Tracking Algorithm typically, this geometry helps to cover more horizontal area In inertial tracking systems, the navigation parameters, i.e., but suffers from the vertical accuracy. The existing UWB/IMU position and velocity, are estimated by solving strapdown fusion algorithms have only considered the tracking problem inertial navigation equations through integration of external in the horizontal plane [3], [30], [31]. Nevertheless, there acceleration [13]. Thus, it is important for the inertial track- is an increased need to accurately track trajectory in the ing systems to accurately estimate the external acceleration vertical direction in physical activities that involve jumping during dynamic conditions by subtracting the gravitational and hopping as well as more complex acrobatic motions. acceleration from the accelerometer signal. To realize this, an To the best of the authors’ knowledge, there is only one accurate estimation of orientation is needed. For the trajectory existing work in [27] that uses UWB/IMU fusion (tightly tracking purposes, in this paper, a loosely coupled cascaded coupled) for 3-D trajectory tracking, which was applied to KF (CKF) consisting of a separate orientation KF followed by walking, and, despite small height variations in walking, large a position/velocity KF is proposed. errors in the vertical direction was reported. In addition, the Our reason for choosing the CKF structure is based primar- previous works on UWB/IMU fusion have only focused on ily on its computational efficiency. By removing the calcula- the tracking accuracies under less dynamic or slow movements tions related to orientation states in the position/velocity KF, such as walking [3], [27]. However, one potential benefit that the cascade formulation has a significantly reduced computa- the IMU offers to the system is the improvement on the tional overhead than that of a global KF [32]. Although the tracking accuracy under highly dynamic and fast motions, cascaded filter is suboptimal, its performance is comparable when the low update rate and noisy measurements of the UWB with the global KF’s performance [32], [33]. In addition to fails to track such motions. the improved computational cost, the cascaded implementa- For the purpose of full trajectory tracking and MoCap, tion allows for increased flexibility and ease of implementa- which include the 3-D tracking of the root joint (i.e., waist) tion and tuning. The structure of the individual filters, i.e., 66 ZIHAJEHZADEH et al.: UWB-AIDED INERTIAL MoCap FOR LOWER BODY 3-D DYNAMIC ACTIVITY AND TRAJECTORY TRACKING 3579 the orientation KF and position/velocity KF, is presented below. The assumption here is that the triaxial accelerometers and gyroscopes are calibrated for bias, scale factor, and nonorthogonality using [34] and the triaxial magnetometers are calibrated for the effects of the deterministic interferences using [35] prior to being used in the proposed algorithm. 1) Orientation Filter: The proposed orientation filter is based on [2] and [36]. In this novel orientation filter, Euler angles are estimated using two cascaded liner KFs: tilt KF and yaw KF. The orientation of the sensor frame (i.e., the coordinate frame attached to the IMU) with respect to a navigation frame (i.e., the coordinate frame of the UWB localization system in which the X-axis and Y -axis are in the horizontal plane of motion and Z is orthogonal to the XY plane, as shown in Fig. 2) can be represented by a rotation matrix, which maps a vector’s rotation from the sensor frame to the navigation frame n = n s Fig. 1. Structure of the proposed trajectory tracking algorithm including x s Rx (1) orientation filter and position/velocity filter. where x is an arbitrary 3 × 1 vector and the superscripts n and s represent the navigation and sensor frames, respectively. The orientation filter in this paper uses the inertial and nR is the 3 × 3 rotation matrix from the sensor frame to the s magnetic data from the seven IMUs attached to the seven lower navigation frame. Using the conventional ZYX Euler angles, body segments including the pelvis, the thighs, the shanks, it is expressed as [36] ⎡ ⎤ and the feet; and outputs the 3-D orientation of these body CαCβ CαSβsγ − SαCγ CαSβCγ + SαSγ segments in the navigation frame for 3-D MoCap as explained n = ⎣ α β α β γ + α γ α β γ − α γ ⎦ s R S C S S S C C S S C C S in Section II-B. Furthermore, the orientation of the pelvis is −Sβ CβSγ CβCγ used in the proposed trajectory tracking algorithm presented (2) in the following section to track the position and velocity of the waist. where C and S are abbreviations for cos and sin, respectively; 2) Position/Velocity Filter: The objective of this particular α (yaw), β (pitch), and γ (roll) are the rotation angles about filter is to estimate the 3-D position and velocity of the the Z-, Y -, and X-axes, respectively. Note that the last row n root joint (the waist) using the rotation matrix s R from of the rotation matrix is the unit gravity vector expressed the above orientation filter, accelerometer output vector, and in the sensor frame, independent from the yaw angle. UWB position measurements (see Fig. 1). The following By estimating this row, the roll and pitch (i.e., the tilt) angles model equations are used in design of this proposed position/ can be determined [36] velocity KF: n n − R , − − R , γ = tan 1 s 3 2 ,β = tan 1 s 3 1 (3) x(k) = A(k−1)x(k−1) + B(k−1)u(k−1) + w(k−1) (5) nR , nR , / sin γ s 3 3 s 3 2 z(k) = C(k)x(k) + v(k) (6) n where s Ri, j represents the ith row and jth column in the n where x(k) =[rn(k)vn(k)]T is the 6×1 state vector consisting rotation matrix s R. In addition, by estimating the first row in the rotation matrix and using the tilt angles, the yaw angle (α) of 3-D position and velocity vectors in the navigation frame; can be readily determined by A(k − 1) and B(k − 1) are the state transition and input matrices for the position/velocity subsystem; u(k − 1) is the n n − −cγ R , + sγ R , α = tan 1 s 1 2 s 1 3 . (4) input vector, being the gravity compensated acceleration in the n / β s R1,1 c navigation frame; w(k−1) is the 6 × 1 vector of process noise; ( ) ( ) As shown in Fig. 1, at the first step in the orientation filter, z k is the UWB position measurement vector; C k is the ( ) i.e., the tilt KF, the gyroscope and accelerometer measurements observation matrix; and v k is the noise of the UWB position measurements. These variables are calculated as follows: are used along with an acceleration model to estimate the last row of the rotation matrix to calculate the tilt angles. At the I × tI × A(k − 1) = 3 3 3 3 (7) second step, i.e., the yaw KF, the gyroscope and magnetometer 03×3 I3×3 measurements are used along with the estimated tilt angles 1 t2I from the first step to accurately estimate the first row of the B(k − 1) = 2 3×3 (8) tI × rotation matrix in determining the yaw angle. This yaw KF has 3 3 u(k − ) = nR(k) y (k) + gn the advantage of detecting magnetic disturbances to bridge 1 s A (9) 1 2 the temporary disturbances (less than about 20-s long) that 2 t I × w(k − 1) = 3 3 nA (10) frequently happen in an indoor environment [2]. tI3×3 67 3580 IEEE TRANSACTIONS ON INSTRUMENTATION AND MEASUREMENT, VOL. 64, NO. 12, DECEMBER 2015

( ) =[UWB( )] z k r k (11) C(k) = I3×3 03×3 (12) ( ) × where t is the sampling period; yA k is the 3 1 output n vector of accelerometer; g is the gravity vector; nA is the accelerometer measurement noise vector; and rUWB(k) is the 3 × 1 output position vector from the UWB system. It is n ( ) important to note that s R k in (9) is calculated using the estimated 3-D orientation form the orientation filter and is considered as a deterministic input [33] to the position/velocity filter. The process and measurement noise covariance matrices in the KF, Q(k − 1) and M(k), are calculated as follows: Q(k − 1) = E[w(k − 1)w(k − 1)T ] 1 2 1 2 T 2 t I × 2 t I × = 3 3 Σ A 3 3 (13) tI3×3 tI3×3 ( ) = σ 2 ( ) M k UWB k I3×3 (14) where Σ A is the covariance matrix of the accelerometer’s [ T ] measurement noise which is defined as E nAnA . By assuming σ 2 that the accelerometer noise variances are equal to A in the 2 2 three axes, Σ A is set to σ I3. σ (k) is the noise variance Fig. 2. Coordinate system of the navigation frame and seven lower body A UWB segment frames, including the right shank, the pelvis, the left thigh, and the of the UWB position measurement at each time step. This left foot, assigned during initialization. variance is the output of the UWB system that defines the accuracy of the position data, which can be used to detect and the pelvis, the thighs, the shanks, and the feet using an IMU remove the outliers. Thus, when UWB position measurement attached to each segment. The following three steps, con- is not accurate (such as the inaccuracies that happen as a result sisting of initialization, orientation tracking with anatomical of obstacles and NLOS conditions), the value of σ 2 (k) is UWB constraints, and motion reconstruction, are employed by the higher, which forces the trajectory tracking KF to rely more on algorithm. the inertial data than the UWB measurements. Fig. 1 shows 1) Initialization (Sensor to Segment Calibration): The coor- the overall structure of the proposed CKF for the trajectory dinate mappings from the IMU sensor frame to its correspond- tracking of the root joint. 3) Trajectory Smoothing: In applications where real-time ing body segment frame are carried out during the initialization step. Thus, the objective of this step is to find the rotation data processing is not required, the navigation performance matrix from each IMU sensor frame to its corresponding can be further improved by smoothing, which combines the body segment frame, bR. For this purpose, the subject starts forward and backward processed data [37]. In this paper, the s initialization by standing in such a way that the Y -axis of Rauch–Tung–Striebel (RTS) smoothing algorithm, a widely the navigation frame points to the anterior direction and the used algorithm in navigation applications, is utilized to smooth X-axis points to the right, as shown in Fig. 2. Based on out position and velocity trajectories [37]. The RTS smoother Fig. 2, the coordinate frame of each body segment is defined consists of one forward data processing and one backward data processing parts. The former is the above position/velocity in a way that it is aligned with the navigation frame during initialization. Since the IMU is attached to each segment, it KF algorithm in which the smoother stores the estimate and its outputs its orientation as a rotation matrix from its sensor covariance. Then, it recursively updates the smoothed estimate frame to the navigation frame, i.e., nR and its covariance in a backward sweep using the following s , and the body segment bR = nR equations [38]: frames are aligned with the navigation frame, s s ,when the initialization step is completed. + − −1 Ks(k) = P (k)A(k)[P (k + 1)] (15) 2) Segment Orientation and Anatomical Constraints: In the + − T Ps(k) = P (k)+Ks(k)[Ps (k + 1) − P (k + 1)]Ks (k) (16) human biomechanical model, some joints can have significant + − movements in only one degree of freedom (DoF). For instance, xs(k) = x (k)+Ks(k)[xs (k + 1) − x (k + 1)] (17) + − while the knee joint has multiple DoF (e.g., flexion/extension, where P and P are the a posteriori and the apriori abduction/adduction, and axial rotation), its major move- covariance estimates, respectively; Ks is the smoother gain; + − ment types are considered to be flexion and extension. Only x and x are the a posteriori and the aprioristate estimates, injured knees can have significant abduction/adduction move- respectively; and xs is the smoothed state vector. All of the ments [10] and also the amount of the limited axial rotation is above mentioned variables are for the position/velocity KF. dependent on the flexion/extension position of the knee [39]. In addition, the abduction/adduction angle and the inter- B. Lower Body MoCap nal/external rotation of the knee are limited to less than 10° The purpose of the proposed lower body MoCap algorithm and are strongly affected by soft-tissue artifacts [40]. Thus, the is to capture the seven lower body segments’ motion including knee joint can be conveniently modeled as a hinge joint with 68 ZIHAJEHZADEH et al.: UWB-AIDED INERTIAL MoCap FOR LOWER BODY 3-D DYNAMIC ACTIVITY AND TRAJECTORY TRACKING 3581

As shown in Fig. 3, to reconstruct full lower body motion, the position of the root joint (the waist in this case) at each time step is used as the base position of the two kinematic chains. Then, using the orientation of each segment at each time step and the segments’ dimensions in forward kinematic equations, the pelvis, the thighs, and the shanks are attached to form the left and right kinematic chains. For example, the following equations show how the left kinematic chain is formed: n ( ) = n ( )[− / − ]T + n ( ) hp p k pvR k lpv,x 20 lpv,z rtp k (20) n ( ) = n ( )[ − ]T + n ( ) kn p k thR k 00 lth hp p k (21) n ( ) = n ( )[ − ]T + n ( ) ak p k shR k 00 lsh kn p k (22) n ( ) = n ( )[ ]T + n ( ) ft p k ftR k 0 lft 0 akp k (23) n where rtp is the root position calculated by the posi- n tion/velocity filter; hp p is the position of the hip; lpv,x and lpv,z are the lengths of the pelvis in x-direction and z-direction, n n Fig. 3. Structure of the proposed lower body MoCap algorithm. respectively; knp is the position of the knee; akp is the position of the ankle; lthand lsh are the lengths of the thigh and shank, n ( ) respectively; ftp k is the position of the foot tip; and lft is the only 1 DoF for typical human MoCap applications [11], [40]. length of the foot. As shown in Fig. 3, the following steps are performed to calculate the orientation of seven lower body segments. III. EXPERIMENTAL SETUP AND PROTOCOL 1) Raw IMU data from each body segments in the sensor A. Experimental Setup s ( ) b frame y k are rotated by s R to get the raw IMU data in the body segment frame by(k) To evaluate the performance of the proposed algorithm, raw inertial/magnetic data from Xsens MVN suit was collected. b ( ) = b s ( ) y k s R y k (18) Since the focus of this paper is on lower body motion tracking, as shown in Fig. 4(a), only seven MTx units were employed where y is the 3 × 1 output vector of accelerometer, in the MVN suit, including one unit on the waist and six units gyroscope, or magnetometer. for the right and left thighs, shanks, and feet. Each MTx unit 2) For the IMU attached to the pelvis, the thigh, and the consists of a triaxial gyroscope, a triaxial accelerometer, and foot, the IMU data in the body segment frame (by) a triaxial magnetometer. The raw inertial/magnetic data are are fed into the orientation algorithm (in Section II-A1) collected at the rate of 100 Hz. As for the UWB-based local- to get the rotation matrices from the pelvis, the thigh, ization system, Ubisense Series 7000 was used in this paper. and the foot body frames to the navigation frame, i.e., This UWB system consists of four anchor nodes (sensors) n R(k), n R(k),andn R(k), respectively. pv th ft and one mobile node (tag) and collects 3-D position data at 3) To constrain the knee joint motion for each leg, the pitch the rate of 16 Hz. The tag is attached to the subject’s waist, and yaw angles of the shank (the rotations about Y -and as shown in Fig. 4(a), and the four UWB sensors are located Z-axes, respectively) are set equal to the ones calculated at each corner of a 1.9 × 2.3-m rectangular-shaped test field for the thigh and only the roll angle (the rotation about [Figs. 4(b) and 5]. X-axis) of the shank is calculated in the orientation The manufacturer-specified localization accuracy of the algorithm. Thus, the following equation is used to get UWB system is within 15 cm in 3-D space. A Qualisys optical the rotation matrix from the shank body frame to the MoCap system, with an accuracy of 0.25 mm, is used as a ref- navigation frame n R(k): sh erence system for the purpose of verification. The positions of n ( ) = (α ) (β ) (γ ) shR k R th R th R sh (19) its reflective markers on the subject’s body and eight cameras around the test field are shown in Figs. 4(a) and 5, respec- (γ ) where R sh represents the rotation of the shank about tively. A GoPro Hero3+ camera was also used to capture the (β ) X-axis; R th is the rotation of the thigh about Y -axis; subject’s body motion for further comparison. and R(αth) is the rotation of the thigh about Z-axis. σ 2 The accelerometer noise variance A was obtained from sta- 3) Skeleton Model and Motion Reconstruction: For full rep- tic measurements and is set to 10−4 m2/s4. The noise variance resentation of lower body motion, the lower body is modeled σ 2 ( ) of the UWB position measurement UWB k was a variable as two separate kinematic chains starting from the pelvis to the obtained from the UWB system for each measurement. left and right foot, respectively. The lower limb segments are assumed to be rigid bodies (as it is a common assumption in human motion analysis [13]) and their dimensions are B. Experimental Protocol measured for each subject to have accurate and subject-specific The proposed algorithm should be able to determine motion reconstruction. 3-D trajectories during typical human activities involving both 69 3582 IEEE TRANSACTIONS ON INSTRUMENTATION AND MEASUREMENT, VOL. 64, NO. 12, DECEMBER 2015

Fig. 4. (a) Position of MTx units, UWB tag, and reﬂective markers on lower body. (b) Experimental setup.

Fig. 5. Position of UWB sensors and reference cameras. slow and fast dynamic motions such as walking and jumping, respectively. Our lab-based experiments were designed to Fig. 6. Norm of the accelerometer and gyroscope measurements from the simulate these two types of motions. Less dynamic motion waist IMU during a typical double-leg horizontal jump trial. was simulated by asking the subject to walk around the rectangular test field, starting from Point D in Fig. 5 in the counter-clockwise direction. The fast dynamic motions were physically fit, 27 years old male with a height of 180 cm and simulated by the subject performing three types of jumps. The a weight of 73 kg. The lower body segments’ dimensions of first jump was a single-leg zigzag jump using the left foot the subject are: 34.5 and 8 cm for the pelvis width and height, between Points A and E and using the right foot between respectively; and 36, 42, and 28 cm for the thigh, shank, and Points E and C (Fig. 5). The second jump was a single-leg foot lengths. lateral jump between Points F and C using both the right The above experimental protocol was approved by the and left foot alternately. The third jump was a double-leg Research Ethics Board of Simon Fraser University. horizontal jump between Points A and D. The two last experiments were specifically designed to introduce considerable IV. EXPERIMENTAL RESULTS AND DISCUSSION height variations to evaluate the performance of the proposed This section is divided into three parts. The perfor- algorithm in the vertical direction. Each experimental trial mance of the proposed algorithm in horizontal and ver- started with a few seconds of stationary initialization followed tical trajectory tracking of the root joint is investigated by about 2 min of continuous motion. The trials were repeated in Sections IV-A and IV-B, respectively. In Section IV-C, the eight times for walking and four times for each of the three accuracy of the algorithm for lower body MoCap is discussed. jumping activities; hence, a total of 20 tests were performed. Sample raw angular velocity and acceleration measurements from the waist IMU during a typical double-leg horizontal A. Horizontal Tracking Accuracy jump trial are shown in Fig. 6. In Fig. 6, each peak in the A horizontal position trajectory from a sample trial for acceleration or angular velocity profiles correspond to one walking and single-leg zigzag jumps estimated by the complete jump and a total of 23 jumps were performed by proposed algorithm is presented in Fig. 7(a) and (b), respec- the subject during the trial shown in Fig. 6. The subject was a tively, compared against the one obtained by the reference 70 ZIHAJEHZADEH et al.: UWB-AIDED INERTIAL MoCap FOR LOWER BODY 3-D DYNAMIC ACTIVITY AND TRAJECTORY TRACKING 3583

Fig. 7. Horizontal trajectory for (a) typical walking experiment and (b) typical single-leg zigzag jump. TABLE I MEAN AND STANDARD DEVIATION OF POSITION AND JOINT ANGLE TRACKING ERROR

camera system. As it can be observed in Fig. 7, the UWB system, used on its own, gives out noisy signals. On the other hand, when these measurements are fused with the IMU data using the proposed fusion algorithm, a smooth trajectory without drift is obtained for both cases. Also, Fig. 7(a) shows the presence of the UWB outliers and the fact that the proposed algorithm is robust against them, which are caused by inevitable errors in UWB localization including NLOS in some segment links. Table I presents the root-mean-square error (RMSE) and one standard deviation in trajectory tracking using the proposed algorithm. These RMSE and standard deviation values presented in Table I under walking are averaged out over the eight walking trials and the ones under jumping are averaged out over the 12 jumping trials. It can be clearly noted in Table I that the fused trajectory in x-direction and y-direction has higher accuracies compared with the UWB system for both walking and jumping cases. In addition, although the accuracy of the UWB system signiﬁcantly degrades (by over 27% in all directions) from walking to jumping, the fused system can Fig. 8. Horizontal position trajectory during the whole experiment for a closely maintain similar accuracies. By fusing IMU with UWB typical single-leg zigzag jump trial. using the proposed algorithm, the accuracies in x-direction and y-direction improved by 57% and 52% for the slow motion Fig. 8 shows the x and y horizontal position trajecto- activities and by 64% and 65% for the fast motion activities, ries from a single-leg zigzag jump experiment that includes respectively. several consecutive jumps. Fig. 8 shows that the proposed 71 3584 IEEE TRANSACTIONS ON INSTRUMENTATION AND MEASUREMENT, VOL. 64, NO. 12, DECEMBER 2015

Fig. 9. Vertical position and velocity trajectories for two consecutive (a) single-leg lateral jumps and (b) double-leg horizontal jumps.

Fig. 11. Knee ﬂexion/extension angle and its error during two consecutive single-leg lateral jumps.

tracking due to the lack of an assistive absolute localization system.

B. Vertical Tracking Accuracy Vertical position and velocity trajectories for two consecutive single-leg lateral jumps and double-leg horizontal jumps Fig. 10. Hip and ankle joint angle values during (a) one gait cycle are shown in Fig. 9(a) and (b), respectively. As it is evident in in a walking trial, (b) two consecutive single-leg lateral jumps, and Fig. 9(a) and (b), the UWB measurements become even noisier (c) two consecutive double-leg horizontal jumps. in the vertical direction in comparison with the horizontal algorithm maintains its accuracy without drift during the direction (Fig. 7). However, the proposed fusion algorithm entire duration of the experiment. However, as also reported can effectively use the noisy UWB measurements and estimate by [12], drifting is inevitable in the self-contained inertial accurate vertical position and velocity trajectories. 72 ZIHAJEHZADEH et al.: UWB-AIDED INERTIAL MoCap FOR LOWER BODY 3-D DYNAMIC ACTIVITY AND TRAJECTORY TRACKING 3585

Fig. 12. Double-leg horizontal jump motion. Top: captured by GoPro camera. Bottom: reconstructed motion using the proposed algorithm. A: start of the jump, B: toe-off, C: heel-strike, D: end of the jump.

Based on Table I, UWB’s localization accuracy in the hip during consecutive single-leg lateral jumps and double-leg vertical direction for walking and jumping is about 31% horizontal jumps, respectively. Fig. 11 shows the estimated and 29% worse than the average accuracy in the horizontal flexion/extension angle as well as the estimation error for the direction. The reported spatial distribution of the horizontal knee joint versus the one from the reference camera system and vertical dilution of precision values for a number of during two consecutive single-leg lateral jumps. As it can different UWB receivers in [41] also verifies reduced accuracy be seen, the estimated angles with the anatomical constraints in the vertical direction for a limited number of UWB sensors will provide improved accuracies, in comparison to the ones (less than 6). This reduced accuracy in the vertical direction, without the constraints. A major reason behind this is the which is a geometry-related issue [29], is also observed in soft tissue artifacts over the bony structure that does not keep other triangulation-based localization systems such as GPS the location of the IMU constant, resulting in measurement receivers [42]. With the limited number of UWB sensors, errors [43]. Fig. 11 shows that applying the kinematic con- typically, this geometry helps to cover more horizontal area straints and limiting the rotation of the knee to only 1 DoF will but suffers from the vertical accuracy. However, as shown effectively reduce this error for our MoCap purposes. Fig. 12 in Table I, the fusion algorithm can effectively reduce the compares the reconstructed double-leg horizontal jump with UWB error in the vertical direction by about 60% and 68% the one captured by GoPro camera. Fig. 12 suggests that the for walking and jumping, respectively. The obtained vertical proposed algorithm can track the lower body kinematics well accuracy for walking is similar to the 4 cm reported accuracy qualitatively. in [14] using self-contained inertial sensors alone. However, as Table I presents the joint angle measurement errors and reported by [14] as well, with the sole use of inertial sensors, their associated standard deviations for walking and jumping the accuracy is significantly degraded when highly dynamic activities. Table I shows that the mean joint angle tracking activities (running and jumping) are involved. On the other error ranges from 0.9° to 3.0° and 1.1° to 3.5° for walking hand, based on the reported fused accuracies in Table I, our and jumping, respectively. The achieved accuracy for hip and proposed system is capable of maintaining similar accuracies ankle tracking is very similar to the ones obtained in [44] for such activities. for walking. However, the knee angle tracking accuracy for jumping is an improvement over the reported 5° accuracy C. Lower Body Motion Tracking under highly dynamic conditions in [13]. This demonstrates The orientation tracking accuracy of the orientation KF that the orientation filter used in our work is more robust under dynamic sport activities has already been evaluated against external acceleration caused by such conditions. in [2] and [36]. The previous results showed that the orientation accuracy, even during highly dynamic conditions, V. C ONCLUSION is less than about 2° for the roll and pitch angles and This paper proposes a novel method for 3-D trajectory about 3° for the yaw angle. Based on the estimated orientation tracking of human subjects combined with lower body seg- of the lower body segments, the joint angles of the hip, the ment MoCap across various dynamic motion conditions. The knee, and the ankle are calculated during the experimental method uses a cascaded two-step KF orientation estimation trial. The flexion/extension angle of the ankle, which is the algorithm to track orientation and anatomical constraints main direction of ankle motion [14], is shown in Fig. 10(a) applied to specific joints of the lower body are proposed during one gait cycle. Fig. 10(b) and (c) illustrates the abduc- to further improve the orientation accuracy. The orientation tion/adduction angle and the flexion/extension angle of the KF is cascaded with a position/velocity KF, which fuses 73 3586 IEEE TRANSACTIONS ON INSTRUMENTATION AND MEASUREMENT, VOL. 64, NO. 12, DECEMBER 2015

UWB with IMU measurements for 3-D trajectory tracking [7] M. El-Gohary and J. McNames, “Shoulder and elbow joint angle purposes. The fused UWB/IMU system includes seven body tracking with inertial sensors,” IEEE Trans. Biomed. Eng., vol. 59, no. 9, pp. 2635–2641, Sep. 2012. worn IMU sensors and one UWB tag. The performance of the [8] H. Fourati, N. Manamanni, L. Afilal, and Y. Handrich, “Complementary system has been tested experimentally using indoor lab-based observer for body segments motion capturing by inertial and magnetic walking and jumping trials. sensors,” IEEE/ASME Trans. Mechatronics, vol. 19, no. 1, pp. 149–157, The experimental results show that the fusion algorithm Feb. 2014. [9] X. Meng, Z.-Q. Zhang, J.-K. Wu, W.-C. Wong, and H. Yu, “Self- can significantly improve the 3-D localization accuracy of contained pedestrian tracking during normal walking using an iner- the UWB system. In fact, under fast dynamic motion, when tial/magnetic sensor module,” IEEE Trans. Biomed. Eng., vol. 61, no. 3, the UWB system tends to be less accurate due to its low pp. 892–899, Mar. 2014. [10] I. Skog, P. Handel, J.-Q. Nilsson, and J. Rantakokko, “Zero-velocity sampling rate, the fused system can significantly improve the detection—An algorithm evaluation,” IEEE Trans. Biomed. Eng., vol. 57, localization accuracy of the UWB system by at least 65%. no. 11, pp. 2657–2666, Nov. 2010. The fusion algorithm shows the accuracies of about 4.2, 3.6, [11] X. L. Meng, Z. Q. Zhang, S. Y. Sun, J. K. Wu, and W. C. Wong, “Biome- and 4.9 cm in x, y,andz directions, respectively. Furthermore, chanical model-based displacement estimation in micro-sensor motion capture,” Meas. Sci. Technol., vol. 23, no. 5, pp. 055101-1–055101-11, the algorithm can also accurately track joint angles during fast May 2012. ◦ dynamic activities, with the accuracy of about 2.1 in the case [12] Q. Yuan and I.-M. Chen, “3-D localization of human based on an of the knee joint. inertial capture system,” IEEE Trans. Robot., vol. 29, no. 3, pp. 806–812, Jun. 2013. In contrast to the existing self-contained inertial motion [13] Q. Yuan and I.-M. Chen, “Human velocity and dynamic behavior tracking methods based on the ground contact detection, tracking method for inertial capture system,” Sens. Actuators A, Phys., which loses their accuracies under fast motion conditions, vol. 183, pp. 123–131, Aug. 2012. our experimental results show that the proposed system can [14] X. Meng, Z.-Q. Zhang, J.-K. Wu, and W.-C. Wong, “Hierarchical information fusion for global displacement estimation in microsensor motion maintain similar accuracies across various dynamic activities capture,” IEEE Trans. Biomed. Eng., vol. 60, no. 7, pp. 2052–2063, such as jumping. In addition, in contrary to the existing Jul. 2013. systems that are drift-prone over time, the proposed system [15] A. Colombo, D. Fontanelli, D. Macii, and L. Palopoli, “Flexible indoor localization and tracking based on a wearable platform and sensor can maintain its accuracy without drift over long period of data fusion,” IEEE Trans. Instrum. Meas., vol. 63, no. 4, pp. 864–876, time in a UWB-covered area. The proposed system can be Apr. 2014. used in motion tracking applications such as indoor/outdoor [16] Y. Tian, W. R. Hamel, and J. Tan, “Accurate human navigation using sports, daily exercise activities, human–robot interactions in wearable monocular visual and inertial sensors,” IEEE Trans. Instrum. Meas., vol. 63, no. 1, pp. 203–213, Jan. 2014. industrial environment and also biomechanical motion analysis [17] X. Zhang and G. Fan, “Dual gait generative models for human motion in a clinical environment as long as the tracking area can be estimation from a single camera,” IEEE Trans. Syst., Man, Cybern. B, covered by the UWB system, which is significantly more cost Cybern., vol. 40, no. 4, pp. 1034–1049, Aug. 2010. effective than the typical optical MoCap systems. For future [18] J. Gu, X. Ding, S. Wang, and Y. Wu, “Action and gait recognition from recovered 3-D human joints,” IEEE Trans. Syst., Man, Cybern. B, work, the authors will further test and verify the proposed Cybern., vol. 40, no. 4, pp. 1021–1033, Aug. 2010. system in a large MoCap studio where more complex human [19] Y. Gu, A. Lo, and I. Niemegeers, “A survey of indoor positioning activities that do not necessarily have frequent ground contact systems for wireless personal networks,” IEEE Commun. Surveys Tuts., can be captured. vol. 11, no. 1, pp. 13–32, Mar. 2009. [20] G. Deak, K. Curran, and J. Condell, “A survey of active and passive indoor localisation systems,” Comput. Commun., vol. 35, no. 16, ACKNOWLEDGMENT pp. 1939–1954, Sep. 2012. The authors would like to thank EA Canada’s Capture Lab [21] A. R. J. Ruiz, F. S. Granja, J. C. P. Honorato, and J. I. G. Rosas, “Accurate pedestrian indoor navigation by tightly coupling foot-mounted in Burnaby, British Columbia, Canada, for lending the IMU and RFID measurements,” IEEE Trans. Instrum. Meas., vol. 61, Xsens MVN suit. no. 1, pp. 178–189, Jan. 2012. [22] H. Shin, Y. Chon, and H. Cha, “Unsupervised construction of an indoor REFERENCES floor plan using a smartphone,” IEEE Trans. Syst., Man, Cybern. C, Appl. Rev., vol. 42, no. 6, pp. 889–898, Nov. 2012. [1] S. Zihajehzadeh, T. J. Lee, J. K. Lee, R. Hoskinson, and E. J. Park, [23] D. Morche et al., “Double-quadrature UWB receiver for wide- “Integration of MEMS inertial and pressure sensors for vertical trajectory range localization applications with sub-cm ranging precision,” IEEE determination,” IEEE Trans. Instrum. Meas., vol. 64, no. 3, pp. 804–814, J. Solid-State Circuits, vol. 48, no. 10, pp. 2351–2362, Oct. 2013. Mar. 2015. [24] B. Gulmezoglu, M. B. Guldogan, and S. Gezici, “Multiperson tracking [2] S. Zihajehzadeh, D. Loh, M. Lee, R. Hoskinson, and E. J. Park, with a network of ultrawideband radar sensors based on Gaussian “A cascaded two-step Kalman filter for estimation of human body mixture PHD filters,” IEEE Sensors J., vol. 15, no. 4, pp. 2227–2237, segment orientation using MEMS-IMU,” in Proc. Conf. IEEE Eng. Med. Apr. 2015. Biol. Soc., Chicago, IL, USA, Aug. 2014, pp. 6270–6273. [3] J. A. Corrales, F. A. Candelas, and F. Torres, “Sensor data integration for [25] Y. Kilic, H. Wymeersch, A. Meijerink, M. J. Bentum, and W. G. Scanlon, indoor human tracking,” Robot. Auto. Syst., vol. 58, no. 8, pp. 931–939, “An experimental study of UWB device-free person detection and Aug. 2010. ranging,” in Proc. IEEE Int. Conf. Ultra-Wideband (ICUWB), Sydney, [4]J.Favre,B.M.Jolles,R.Aissaoui, and K. Aminian, “Ambulatory NSW, Australia, Sep. 2013, pp. 43–48. measurement of 3D knee joint angle,” J. Biomech., vol. 41, no. 5, [26] D. Kocur et al., “Short-range UWB radar: Surveillance robot equipment pp. 1029–1035, Jan. 2008. of the future,” in Proc. IEEE Int. Conf. Syst., Man, Cybern., San Diego, [5] C.-H. Wu and Y.-C. Tseng, “Deploying sensors for gravity measurement CA, USA, Oct. 2014, pp. 3767–3772. in a body-area inertial sensor network,” IEEE Sensors J., vol. 13, no. 5, [27] J. D. Hol, F. Dijkstra, H. Luinge, and T. B. Schon, “Tightly coupled pp. 1522–1533, May 2013. UWB/IMU pose estimation,” in Proc. IEEE Int. Conf. Ultra-Wideband, [6] Z.-Q. Zhang, L.-Y. Ji, Z.-P. Huang, and J.-K. Wu, “Adaptive information Vancouver, BC, Canada, Sep. 2009, pp. 688–692. fusion for human upper limb movement estimation,” IEEE Trans. [28] A. Waegli and J. Skaloud, “Optimization of two GPS/MEMS-IMU Syst., Man, Cybern. A, Syst., Humans, vol. 42, no. 5, pp. 1100–1108, integration strategies with application to sports,” GPS Solutions, vol. 13, Sep. 2012. no. 4, pp. 315–326, Sep. 2009. 74 ZIHAJEHZADEH et al.: UWB-AIDED INERTIAL MoCap FOR LOWER BODY 3-D DYNAMIC ACTIVITY AND TRAJECTORY TRACKING 3587

[29] M. Tanigawa, J. D. Hol, F. Dijkstra, H. J. Luinge, and P. J. Slycke, Paul K. Yoon (S’15) received the B.A.Sc. degree in “Augmentation of low-cost GPS/MEMS INS with UWB positioning mechatronic systems engineering from Simon Fraser system for seamless outdoor/indoor positioning,” in Proc. 21st Int. Tech. University, Surrey, BC, Canada, in 2013, where he Meeting Satellite Division Inst. Navigat. ION GNSS, Savannah, GA, is currently pursuing the M.A.Sc. degree with the USA, Sep. 2008, pp. 1–8. School of Mechatronic Systems Engineering. [30] S. Pittet, V. Renaudin, B. Merminod, and M. Kasser, “UWB and MEMS His current research interests include inertial based indoor navigation,” J. Navigat., vol. 61, no. 3, pp. 369–384, sensor-based human motion capture, wireless indoor Jun. 2008. positioning, optimal and robust state estimation, and [31] A. Benini, A. Mancini, and S. Longhi, “An IMU/UWB/vision-based sensor fusion. extended Kalman filter for mini-UAV localization in indoor environment using 802.15.4a wireless sensor network,” J. Intell. Robot. Syst., vol. 70, nos. 1–4, pp. 461–476, Apr. 2013. [32] D. B. Kingston and R. W. Beard, “Real-time attitude and position estimation for small UAVs using low-cost sensors,” in Proc. AIAA 3rd ‘Unmanned Unlimited’ Tech. Conf., Workshop Exhibit, Chicago, IL, USA, Sep. 2004, pp. 1–9. [33] Z. Lendek, R. Babuška, and B. De Schutter, “Distributed Kalman filtering for cascaded systems,” Eng. Appl. Artif. Intell., vol. 21, no. 3, pp. 457–469, Apr. 2008. [34] W. T. Fong, S. K. Ong, and A. Y. C. Nee, “Methods for in-field user calibration of an inertial measurement unit without external equipment,” Meas. Sci. Technol., vol. 19, no. 8, p. 085202, Jul. 2008. [35] S. A. H. Tabatabaei, A. Gluhak, and R. Tafazolli, “A fast calibra- Bong-Soo Kang (M’09) received the B.Sc., M.Sc., tion method for triaxial magnetometers,” IEEE Trans. Instrum. Meas., and Ph.D. degrees in mechanical engineering from vol. 62, no. 11, pp. 2929–2937, Nov. 2013. the Korea Advanced Institute of Science and Tech- [36] S. Zihajehzadeh, D. Loh, T. J. Lee, R. Hoskinson, and E. J. Park, nology, Daejeon, Korea, in 1991, 1993, and 1999, “A cascaded Kalman filter-based GPS/MEMS-IMU integration for sports respectively. applications,” Measurement, vol. 73, pp. 200–210, Sep. 2015. He was a Post-Doctoral Fellow with the Depart- [37] H. Liu, S. Nassar, and N. El-Sheimy, “Two-filter smoothing for accurate ment of Mechanical and Industrial Engineering, INS/GPS land-vehicle navigation in urban centers,” IEEE Trans. Veh. University of Toronto, Toronto, ON, Canada, from Technol., vol. 59, no. 9, pp. 4256–4267, Nov. 2010. 2000 to 2002. He is currently a Professor of Mechan- [38] D. Simon, Optimal State Estimation. New York, NY, USA: Wiley, 2006. ical Engineering with Hannam University, Daejeon. [39] G. Mcginty, J. J. Irrgang, and D. Pezzullo, “Biomechanical con- His current research interests include medical robot- siderations for rehabilitation of the knee,” Clin. Biomech., vol. 15, ics, biomimetic mechanism, and intelligent control. pp. 160–166, Mar. 2000. [40] T. Seel, J. Raisch, and T. Schauer, “IMU-based joint angle measurement for gait analysis,” Sensors, vol. 14, no. 4, pp. 6891–6909, Apr. 2014. [41] L. Zwirello, T. Schipper, M. Harter, and T. Zwick, “UWB localization system for indoor applications: Concept, realization and analysis,” J. Elect. Comput. Eng., vol. 2012, Feb. 2012, Art. ID 849638. [42] R. Machotka, O. Svabensky, and J. Weigel, “On GPS heighting in local networks,” Acta Geodyn. Geomater., vol. 3, no. 143, pp. 39–43, Jun. 2006. [43] X. Chen, “Human motion analysis with wearable inertial sensors,” Ph.D. dissertation, Univ. Tennessee, Knoxville, TN, USA, 2013. [44] J. Zhang, A. C. Novak, B. Brouwer, and Q. Li, “Concurrent validation of Xsens MVN measurement of lower limb joint angular kinematics,” Edward J. Park (M’03–SM’13) received the Physiol. Meas., vol. 34, no. 8, pp. N63–N69, Jul. 2013. B.A.Sc. degree from the University of British Columbia, Vancouver, BC, Canada, in 1996, and the M.A.Sc. and Ph.D. degrees from the University of Toronto, Toronto, ON, Canada, in 1999 and 2003, respectively, all in mechanical engineering. Shaghayegh Zihajehzadeh (S’15) received the B.Sc. degree in electrical He was an Assistant Professor with the Depart- engineering/control systems from the Isfahan University of Technology, ment of Mechanical Engineering, University of Vic- Isfahan, Iran, in 2008, and the M.Sc. degree in electrical engineer- toria, Victoria, BC, Canada, from 2003 to 2008. ing/mechatronics from the Amirkabir University of Technology, Tehran, Iran, He is currently a Professor with the School of in 2011. She is currently pursuing the Ph.D. degree with the School of Mechatronic Systems Engineering, Simon Fraser Mechatronic Systems Engineering, Simon Fraser University, Surrey, BC, University, Surrey, BC, Canada, and the Director of the Biomechatronic Canada. Systems Laboratory. He is also a member of the faculty of Health Sciences. Her research focuses on the development of an accurate and ubiquitous He has authored or co-authored around 100 journal and conference papers in motion capture system using wearable technology. Her current research his research areas. His current research interests include biomechatronics and interests include sensor fusion, inertial motion capture, and indoor/outdoor biomendical technologies for life sciences, rehabilitation and medicine, and localization. mechatronics applied to next generation vehicular, robotic, and space systems.

Appendix E.

A magnetometer-free indoor human localization based on loosely coupled IMU/UWB fusion

(Reproduced with permission from IEEE)

A Magnetometer-Free Indoor Human Localization Based on Loosely Coupled IMU/UWB Fusion

Shaghayegh Zihajehzadeh, Student Member, IEEE, Paul K. Yoon, Student Member, IEEE and Edward

J. Park1, Senior Member, IEEE

 Abstract— The magnetic distortions in indoor environment The available IMU/UWB fusion algorithms are divided affects the accuracy of yaw angle estimation using into tightly coupled [6] and loosely coupled [7] based on magnetometer. Thus, the accuracy of indoor localization based whether the raw time of transmission data or the triangulated on inertial-magnetic sensors will be affected as well. To address position data is used in IMU/UWB fusion. Similar to the this issue, this paper proposes a magnetometer-free solution for GPS/IMU integration, a tightly coupled UWB/IMU fusion indoor human localization and yaw angle estimation. The might be advantageous under UWB outages for outliers proposed algorithm fuses a wearable inertial sensor consisting of MEMS-based accelerometer and gyroscope with a portable detection [6]. However, since UWB based localization has ultra-wideband (UWB) localization system in a cascaded two- no line-of-sight requirements [5], UWB outages are less step filter consisting of a tilt Kalman filter and a localization probable in a confined room environment. Furthermore, the Kalman filter. By benchmarking against an optical motion loosely coupled approach offers simpler implementation and capture system, the experimental results show that the less computation. In the existing loosely coupled UWB/IMU proposed algorithm can accurately track position and velocity fusion algorithms, magnetic data from magnetometer are as well as the yaw angle without using magnetometer. used to help the estimation of yaw (heading) angle and thus I. INTRODUCTION the horizontal position and velocity [7],[8]. However, in indoor environments, the Earth’s magnetic field can be Indoor localization and motion capture (MoCap) have easily perturbed by the presence of ferromagnetic objects numerous applications in medicine, rehabilitation [1] and [1][3][9], necessitating the need for magnetometer-free sports science [2]. Due to the small size and light weight, a localization. Although a nonlinear tightly coupled Kalman MEMS-based wearable inertial measurement unit (IMU) filter-based fusion algorithm for magnetometer-free consisting of accelerometers, gyroscopes and magnetometers UWB/IMU integration has previously been proposed in [6], has been widely used for localization purposes [3]. the complexity and computational costs of the nonlinear However, although accurate for short period of time, the approach may not be desirable for low cost, battery powered navigation solution based on MEMS-IMU will drift over ambulatory analysis in human localization [6]. time due to the IMU’s inherent errors [3]. While the use of This paper introduces a novel two-step cascaded Kalman zero velocity update (ZUPT) will improve the localization filter for a loosely coupled magnetometer-free IMU/UWB accuracy for a shoe-mounted IMU, the drift will still appear integration. The study investigates the effect of not using over time and get worse for dynamic activities with less magnetometer for indoor human localization and yaw angle apparent ground contact [4]. Thus, to improve the indoor estimation. The proposed algorithm uses two linear Kalman localization performance, absolute localization technologies filters. At the first step, i.e. the tilt Kalman filter which is based on vision and/or radio frequency (RF) [5] have been based on our algorithm in [10], tilt angles are estimated. At fused with IMU. However, vision based and RF based the second step, i.e. the localization Kalman filter, which is localization suffer from the non-line-of-sight (NLOS) the main contribution of this paper, the UWB localization problem and multipath effects respectively [5]. Among the system is fused with the IMU’s accelerometer and gyroscope RF based localization technologies, ultra-wideband (UWB) to estimate position, velocity and the yaw angle. The system can distinguish between the original and reflected experimental results show that the proposed method can signal by using narrow pulses (less than 1 ns) [5]. Due to the accurately estimate yaw angle, and the position and velocity excellent precision, reasonable cost and low power solutions match the ones from an optical MoCap system. consumption, UWB-based localization has become very popular [5]. A fused UWB/IMU localization system takes II. METHOD advantage of the high sampling rate of the IMU and the In this work, a cascaded two-step Kalman filter is used to precision of the UWB system. estimate the full 3-D orientation as well as the 3-D position and velocity. Euler angle representation is used to show the orientation of the sensor frame 𝑠 (the coordinate system This work is supported by the Natural Sciences and Engineering fixed to the IMU sensor) with respect to the navigation Research Council of Canada (NSERC) and the experimental protocol (No, 2013s0750) is approved by the Office of Research Ethics of Simon Fraser frame 𝑛 (the coordinate system of UWB localization). The University. cascaded Kalman filter tends to be computationally more S. Zihajehzadeh, P. K. Yoon and E. J. Park are with the School of efficient than the centralized filters [2] and consists of a tilt Mechatronic Systems Engineering, Simon Fraser University, 250-13450 102nd Avenue, Surrey, BC, Canada, V3T 0A3 (email: [email protected], angle Kalman filter and a localization Kalman filter. The [email protected], [email protected]) structure of these filters is explained below.

system in addition to the tilt angles information from the tilt 𝛾(푘) Kalman filter. The following system model is used for the Gyroscope 𝛽(푘) Tilt Localization 𝛼(푘) localization filter: Kalman Kalman 푝(푘) Accelerometer filter filter 𝒙(푘 + 1) = 𝑨(푘) 𝒙(푘) + 𝑩(푘) 𝒖(푘) + 풏(푘) (3) 푣(푘) 𝒚(푘) = 𝑪(푘) 𝒙(푘) + 𝒘(푘) (4)

where 𝒙 is the state vectors; 𝑨 is the state transition matrix; UWB 𝑩 is the input matrix; 𝒖 is the input vector; 풏 is the process

Fig. 1. Overview of the proposed algorithm structure noise vector; 𝒚 is the measurement vector; 𝑪 is the measurement matrix; and 𝒘 is the measurement noise A. Tilt Kalman Filter vector. The tilt Kalman filter is based on our previous algorithm The state vector 𝒙(푘) = [풓(푘) 𝒗(푘) 𝒛(푘)]𝑇 is the in [10], which allows accurate determination of roll and 9 × 1 vector where 풓 and 𝒗 are the 3-D position and velocity pitch angles. As shown in Fig. 1, in this filter, the tri-axial vectors, respectively; and 𝒛 is the first row of the rotation gyroscope and tri-axial accelerometer are used in a Kalman 푛 matrix 𝑠𝑹 and is equal to: filter to estimate the normalized gravity vector in the sensor frame (i.e. the third row of the rotation matrix from the 𝒛(푘) = [𝑧1 𝑧2 𝑧3] = 푛 (5) sensor frame to navigation frame 𝑠𝑹): = [푐𝛼푐𝛽 푐𝛼𝑠𝛽𝑠𝛾 − 𝑠𝛼푐𝛾 푐𝛼𝑠𝛽푐𝛾 + 𝑠𝛼𝑠𝛾]

푐𝛼푐𝛽 푐𝛼𝑠𝛽𝑠𝛾 − 𝑠𝛼푐𝛾 푐𝛼𝑠𝛽푐𝛾 + 𝑠𝛼𝑠𝛾 The measurement vector 𝒚, consists of the 3-D position 푛𝑹 = [𝑠𝛼푐𝛽 𝑠𝛼𝑠𝛽𝑠𝛾 + 푐𝛼푐𝛾 𝑠𝛼𝑠𝛽푐𝛾 − 푐𝛼𝑠𝛾] (1) 𝑠 and velocity measurements from the UWB system, i.e. −𝑠𝛽 푐𝛽𝑠𝛾 푐𝛽푐𝛾 𝑇 𝒚(푘) = [풓𝑈푊𝐵(푘) 𝒗𝑈푊𝐵(푘)] . Thus, 𝑪 can be written as: where 𝛼 (yaw), 𝛽 (pitch) and 𝛾 (roll) are the rotation angles ( ) [푰 ퟎ ] about the 𝑍-, 𝑌-, and 𝑋- axes respectively. 𝑪 푘 = 6×6 6×3 (6)

Using the estimated last row of the rotation matrix, the Next, the following inertial navigation equations are used desired roll and pitch angles can be calculated [10]. to calculate 𝑨 and 𝑩 matrices in the state space model (3):

B. Localization Kalman Filter 풓(푘 + 1) = 풓(푘) + ∆𝑡 𝒗(푘) (7) The localization Kalman filter is designed to estimate 𝒗(푘 + 1) = position and velocity vectors as well as the first row of the 푛 𝑇 (8) 푛 𝒗(푘) + ∆𝑡 ( 𝑠𝑹(푘)𝒚𝐴(푘) − [0 0 푔] ) rotation matrix 𝑠𝑹. By estimating the first row of the rotation matrix and using the tilt angles information from the 𝒛(푘 + 1) = 횽(푘) 𝒛(푘) (9) tilt Kalman filter, the yaw angle (𝛼) can be estimated using: 횽(푘) = 푰3×3 − ∆𝑡 𝒚̃퐺(푘) (10)

푛 푛 𝑇 −푐𝛾 𝑠𝑹1,2 + 𝑠𝛾 𝑠𝑹1,3 where 𝒚 (푘) = [𝑎𝑥 𝑎𝑦 𝑎𝑧] is the bias compensated −1 (2) 𝐴 𝛼 = tan ( 푛 ) 𝑠𝑹1,1⁄푐𝛽 output vector of the accelerometer; 푔 is the norm of the gravity vector; 𝒚̃ is the 3 × 3 skew symmetric matrix of tri- 푛 th th 퐺 where 𝑠𝑹𝑖,𝑗 represents the i row and j column in rotation axial gyroscope measurements and 푰3×3 is the 3 × 3 identity matrix 푛𝑹. 푛 𝑠 matrix. To find 𝑨, 𝑩 and 𝒖, 𝑠𝑹(푘) in Eq. (8) is re-written in As shown in Fig. 1, the localization Kalman filter uses the terms of the three elements of the 𝒛 vector and the known tilt tri-axial gyroscope, tri-axial accelerometer and the UWB angles from the tilt Kalman filter as following: 𝑧1 𝑧2 𝑧3 푛 𝑠𝑹(푘) = [푘3(−푘1𝑧2 + 푘2𝑧3) 푘2푘4(−푘1𝑧2 + 푘2𝑧3) + 푘4𝑧1/푘3 푘1푘4(−푘1𝑧2 + 푘2𝑧3) − 푘2𝑧1/푘3] (11) −푘4 푘2푘3 푘1푘3

In Eq. (11), 푘 = 푐𝛾, 푘 = 𝑠𝛾, 푘 = 푐𝛽 and 푘 = 𝑠𝛽. By ퟎ 1 2 3 4 5×4 (15) substituting Eq. (11) into Eq. (8) and expanding it, 𝑨, 𝑩 and 𝑩(푘) = [−∆𝑡 푘4 ∆𝑡 푘2푘3 ∆𝑡 푘1푘3 −∆𝑡 ] 𝒖 can be calculated as: ퟎ 3×4

푰3×3 ∆𝑡 푰3×3 ퟎ3×3 where 푘5 to 푘7 are as following:

𝑨(푘) = [ퟎ3×3 푰3×3 𝑨23(푘)] (12) 푘5 = 𝑎𝑦 푘1⁄푘3 − 𝑎𝑧 푘2⁄푘3 ퟎ3×3 ퟎ3×3 횽(푘) 푘6 = −푘1푘3𝑎𝑥 − 푘1푘2푘4𝑎𝑦 − 푘1푘2푘4𝑎𝑧 (16) 𝑎𝑥 𝑎𝑦 𝑎𝑧 2 2 푘7 = 푘2푘3𝑎𝑥 + (푘2) 푘4𝑎𝑦 + (푘2) 푘4𝑎𝑧 𝑨23(푘) = ∆𝑡 [푘5 푘6 푘7] (13) 0 0 0 The process and measurement noise covariance matrices in the tilt Kalman filter, 𝑸(푘) and 푴(푘), are calculated 𝒖(푘) = [𝒚 (푘)𝑇 푔]𝑇 (14) 𝐴 using the following equations [2]:

3142 78

𝑸(푘) = 퐸[풏(푘) 풏(푘)𝑇] = (17) 1 1 𝑇 UWB 2 2 Sensor ∆𝑡 푰3×3 ퟎ3×3 휮 ퟎ ∆𝑡 푰3×3 ퟎ3×3 [2 ] [ 𝐴 3×3] [2 ] ∆𝑡 푰3×3 ퟎ3×3 ퟎ3×3 휮퐺 ∆𝑡 푰3×3 ퟎ3×3 ퟎ3×3 −∆𝑡 𝒛̃(푘) ퟎ3×3 −∆𝑡 𝒛̃(푘)

2 Reference 𝑇 휎𝑟,𝑈푊𝐵 푰3×3 ퟎ3×3 푴(푘) = 퐸[𝒘(푘) 𝒘(푘) ] == [ 2 ] (18) Cameras ퟎ3×3 휎푣,𝑈푊𝐵 푰3×3 Xsens MTx

UWB Tag where 휮퐺 and 휮𝐴 are the covariance matrix of the gyroscope’s and accelerometer’s measurement noise Optical marker respectively. By assuming that their noise variances for the

2 2 Fig. 2. Experimental setup: showing one corner of the test area gyroscope and accelerometer are equal to 휎퐺 and 휎𝐴 , 휮퐺 and 2 2 2 2 휮𝐴 are set to 휎퐺 푰3×3 and 휎𝐴 푰3×3; 휎𝑟,𝑈푊𝐵 and 휎푣,𝑈푊𝐵 are the IV. EXPERIMENTAL RESULTS AND DISCUSSION UWB position and velocity noise variance respectively. In this section, performance of the proposed algorithm in C. Smoothing yaw angle, position and velocity estimation is evaluated. In this paper, the Rauch-Tung-Striebel (RTS) smoothing A. Yaw angle tracking algorithm [2], a widely used smoothing algorithm in Fig. 3 shows the performance of our proposed algorithm navigation applications, is utilized in the localization for yaw angle estimation with and without RTS smoother. Kalman filter. RTS smoother recursively updates the The reference yaw angle is calculated by employing smoothed estimate and its covariance in a backward sweep magnetic data in addition to accelerometer and gyroscope using the following equations: measurements using our previously validated method in + − −1 푲𝑠(푘) = 𝑷 (푘) 𝑨(푘) [𝑷 (푘 + 1)] (19) [10]. To ensure the accuracy of the reference curve, as 𝑷 (푘) = shown in Fig. 3 (bottom), the experiment took place in a 𝑠 (20) 𝑷+(푘) + 푲 (푘) [𝑷 (푘 + 1) − 𝑷−(푘 + 1)] 푲 (푘)𝑇 uniform magnetic field, away from electromagnetic 𝑠 𝑠 𝑠 disturbances. As it can be seen the blue curve in Fig. 3 (top), ( ) +( ) ( ) ( ) −( ) (21) 𝒙𝑠 푘 = 𝒙 푘 + 푲𝑠 푘 [𝒙𝑠 푘 + 1 − 𝒙 푘 + 1 ] after about 20 s from the start of the motion the yaw angle where 𝑷+ and 𝑷− are the “a posteriori” and the “a priori” which is initialized to zero will converge to the reference + − value. Additionally, by the use of smoother, the algorithm covariance estimates; 푲𝑠 is the smoother gain; 𝒙 and 𝒙 are the “a posteriori” and the “a priori” state estimates; and can correct for the initial estimation errors before convergence (the red curve in Fig. 3 (top)). Furthermore, the 𝒙𝑠 is the smoothed state vector. green curve in Fig. 3 (top) show the estimated yaw angle III. Experimental Setup during a 20 s of simulated magnetic disturbance when raw In order to evaluate the performance of the proposed magnetic data are fused with accelerometer and gyroscope. algorithm, raw inertial data from Xsens MTx at the rate of This magnetic disturbance is simulated based on the 100 Hz are used. As shown in Fig. 2, the sensor is worn by a magnetic field data in proximity of an iron disk captured in human subject on his waist. The MTx unit consists of tri- our previous work [10]. This green curve verifies that the axial gyroscopes, tri-axial accelerometers and tri-axial accuracy of the magnetometer aided estimation suffers in the magnetometers. The magnetometer data are only used for presence of the magnetic disturbances, while the verification purposes. For the UWB-based localization magnetometer-free solution does not get affected. system, the commercially available Ubisense Series 7000 is used. This system consists of four anchor nodes (sensors) and one mobile node (tag) which collects 3-D position data at the rate of about 10 Hz. The tag is attached to the subject’s waist and the four UWB sensors are located at each corner of our 1.9  2.3 𝑚 rectangular-shaped test field. A Qualisys optical MoCap system, with the localization accuracy of 0.25 mm, is used as a reference motion tracking to provide ground truth position and velocity trajectories. Since the inertial MoCap systems are comparatively less accurate under dynamic conditions [4], jumping activity is chosen to simulate highly dynamic motions in order to evaluate the performance of the proposed algorithm in the worst-case scenario. Thus, to collect experimental data, the subject is asked to jump from one corner of the rectangular test field to another and vice versa continuously for about

90 s. The experiment is repeated for nine times to carry out a statistical error analysis. Fig. 3. Top: estimated yaw angle with and without smoother compared to the reference yaw angle, Bottom: normalized magnetic field norm

3143 79

shows that the algorithm can accurately track velocity during the experimental trials. Since the choice of not using magnetometer will not affect the position and velocity states in the vertical direction, the evaluation of these states was omitted from this study. Further analysis in the vertical direction has been done in our previous work [2].

V. CONCLUSION This paper introduces a novel two-step cascaded Kalman

filter for a loosely coupled magnetometer-free IMU/UWB Fig. 4. State covariance estimation for the three elements of 퐳 vector integration for indoor human tracking. The algorithm The time it takes for the estimated yaw angle to converge consists of two linear Kalman filters: the tilt Kalman filter is due to the fact that the filter requires some motion to build and the localization Kalman filter and is able to accurately the correlation between the states and correct for the initial estimate the tilt angles, position, velocity and the yaw unknown yaw angle. The same phenomena has been (heading) angle. Compared to the magnetometer-aided observed in [11] for GPS aided inertial navigation in outdoor tracking, using the proposed magnetometer-free algorithm, environment. The state covariance estimation for the 𝒛 any indoor magnetic disturbances will no longer affect the vector in Fig. 4 also verifies that the covariance decreases horizontal position and velocity estimation. However, the rapidly after the start of the motion. experimental results show that about 20 s of motion is required for the yaw angle to converge which is the trade-off B. Position and velocity tracking for not using the magnetometer. Benchmarking against the The estimated horizontal position for a part of optical MoCap system, experimental results show that the experimental trial is shown in Fig. 5. As it can be seen in this horizontal position and velocity can be accurately tracked figure, the fused solution can accurately track position and using the proposed method. In future, further remove the inevitable drift in inertial tracking and the noise experimentation will be performed to quantify the tracking in UWB localization. The error analysis over the nine improvement in the presence of real magnetic disturbances. experimental trials shows that localization accuracy of the proposed method is less than 4 푐𝑚 in both 𝑥 any 𝑦 REFERENCES directions. Additionally, the horizontal velocity in Fig. 6 [1] S. House, S. Connell, I. Milligan, D. Austin, T. L. Hayes and P. Chiang, “Indoor Localization Using Pedestrian Dead Reckoning Updated with RFID-Based Fiducials,” in Conf. Proc. IEEE Eng. Med. Biol. Soc., Boston, MA, USA, Aug. 2011, pp. 7598–7601. [2] S. Zihajehzadeh, T. J. Lee, J. K. Lee, R. Hoskinson, and E. J. Park, “Integration of MEMS Inertial and Pressure Sensors for Vertical Trajectory Determination,” IEEE Trans. Instrum. Meas., vol. 64, no. 3, pp. 804-814, Mar. 2015. [3] S. Zihajehzadeh, D. Loh, T. J. Lee, R. Hoskinson, and E. J. Park, “A cascaded Kalman filter-based GPS/MEMS-IMU integration for sports applications,” Measurement, vol. 73, pp. 200-210, Sept. 2015. [4] X. Meng, Z. Q. Zhang, J. K. Wu, and W. C. Wong, “Hierarchical information fusion for global displacement estimation in microsensor motion capture.,” IEEE Trans. Biomed. Eng., vol. 60, no. 7, pp. 2052– 63, Jul. 2013. [5] Y. Gu, A. Lo, and I. Niemegeers, “A survey of indoor positioning systems for wireless personal networks,” IEEE Commun. Surv. Tutorials, vol. 11, no. 1, pp. 13–32, Mar. 2009.

Fig. 5. Position trajectories in horizontal directions for a part of one [6] J. D. Hol, F. Dijkstra, H. Luinge, and T. B. Schon, “Tightly Coupled experimental trial UWB / IMU Pose Estimation.” Int. Conf. on Ultra-Wideband,

Vancouver, Canada, Sep. 2009, pp. 688 - 692. [7] J. A. Corrales, F. A. Candelas, and F. Torres, “Sensor data integration for indoor human tracking,” Rob. Auton. Syst., vol. 58, no. 8, pp. 931– 939, Aug. 2010. [8] S. Pittet, V. Renaudin, B. Merminod, and M. Kasser, “UWB and MEMS Based Indoor Navigation,” J. Navig., vol. 61, no. 03, pp. 369- 384, Jun. 2008. [9] T. H. Riehle, S. M. Anderson, P. A. Lichter, N. A. Giudice, S. I. Sheikh, R. J. Knuesel, D. T. Kollmann, D. S. Hedin, and A. P. Hardware, “Indoor Magnetic Navigation for the Blind,” in Conf. Proc. IEEE Eng. Med. Biol. Soc., San Diego, CA, USA, Aug. 2012, pp. 1972–1975. [10] S. Zihajehzadeh, D. Loh, M. Lee, R. Hoskinson, and E. J. Park, “A Cascaded Two - Step Kalman Filter for Estimation of Human Body Segment Orientation Using MEMS - IMU,” in Conf. Proc. IEEE Eng. Med. Biol. Soc., Chicago, Illinois, Aug. 2014, pp. 6270-6273. [11] A. Waegli, J. Skaloud, P. Tomé, and J. Bonnaz, “Assessment of the Integration Strategy between GPS and Body-Worn MEMS Sensors

with Application to Sports,” in Conf. Proc. ION GNSS, Texas, USA, Fig. 6. Velocity trajectories in horizontal directions Sep. 2007. 3144 80

Appendix F.

A novel biomechanical model-aided IMU/UWB fusion for magnetometer-free lower body motion capture

(Reproduced with permission from IEEE)

81 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS: SYSTEMS 1 A Novel Biomechanical Model-Aided IMU/UWB Fusion for Magnetometer-Free Lower Body Motion Capture Shaghayegh Zihajehzadeh, Student Member, IEEE, and Edward J. Park, Senior Member, IEEE

Abstract—The available human body inertial motion cap- surface is accurately tracked during activities, have been com- ture (MoCap) systems are aided by magnetometers to remove monly used for human body MoCap [10]. However, these the drift error in yaw angle estimation, which in turn limits their systems are costly, confined to limited space and suffer from application in the presence of long-lasting magnetic disturbances in indoor environments. This paper introduces a magnetometer- occlusion [11]. Recently, with the advent of microelectrome- free algorithm for lower-body MoCap including 3-D localization chanical systems technology, miniature inertial measurement and posture tracking by fusing inertial sensors with an ultraw- units (IMUs) consisting of a tri-axial accelerometer and gyro- ideband (UWB) localization system and a biomechanical model scope have emerged as wearable MoCap devices for both of the human lower body. Using our novel Kalman filter-based localization [12]Ð[14] and posture tracking [15], [16]. fusion algorithm, the UWB localization data aided by the biomechanical model can eliminate the drift in inertial yaw angle Wearable IMUs have been extensively used for 3-D localiza- estimation of the lower-body segments. This magnetometer-free tion purposes using inertial navigation techniques [17], [18]. yaw angle estimation makes the algorithm insensitive to the In many of the proposed inertial navigation methods, magne- magnetic disturbances. The algorithm is benchmarked against tometers have been used as aiding devices to provide a stable the optical MoCap system for various indoor activities includ- heading (yaw) angle over time [12], [19]. However, the local- ing walking, jogging, jumping, and stairs ascending/descending. The results show that the proposed algorithm outperforms the ization solution based on a magnetometer-aided IMU will available magnetometer-aided algorithms in yaw angle tracking still drift due to the integration of inconstant accelerome- under magnetic disturbances. In a uniform magnetic field, the ter errors over time [20]. Although the use of zero velocity algorithm shows similar accuracies in localization and joint angle update (ZUPT) can eliminate the drift error for a shoe- tracking when compared with the magnetometer-aided methods. mounted IMU, the accuracy of such algorithms degrades over The localization accuracy of the proposed method is better than 4.5 cm in a 3-D space and its accuracy for knee angle track- time, specifically for fast movements with unstable ground ing is about 3.5◦ and 4.5◦ in low and high dynamic motions, contact [13], [21]. Thus, to improve the indoor localization respectively. performance while taking advantage of high sampling rate of > Index Terms—Inertial measurement unit (IMU), iner- the IMU ( 100 Hz), absolute localization technologies [22] tial motion tracking, Kalman filter, magnetometer-free, with relatively low sampling rates (<10 Hz) have been ultrawideband (UWB) localization, wearable technology. fused with the IMU. Among these technologies, ultrawideband (UWB) localization has become very popular due to its excellent precision, reasonable cost, and low-power consumption [22], [23]. However, UWB localization suffers I. INTRODUCTION from outliers caused by the metallic materials, nonline-of- OTION capture (MoCap) systems are used by differ- sight (NLOS) conditions, and reflection that degrades the M ent disciplines to capture and reconstruct the movement localization accuracy [24]. The available IMU/UWB fusion and posture of the human body [1], [2]. With the advances algorithms are divided into two groups, tightly coupled [25] in 3-D visualization technology, MoCap has found numer- and loosely coupled [18], [26]Ð[28], based on whether the raw ous applications in gaming and film making [3], augmented time of transmission data or the triangulated position data is reality [4], and sport science [5]Ð[8] along with its applica- used in the IMU/UWB fusion. While the former can be advan- tion in rehabilitation and medical sciences [9]. Optical sys- tageous under UWB outages for outliers detection [25], the tems, in which the position of reflective markers on a body latter offers simpler implementation and less computation, making them desirable in wearable MoCap. However, in the Manuscript received May 18, 2015; accepted August 31, 2015. This work existing loosely coupled UWB/IMU fusion algorithms, magne- was supported by the Natural Sciences and Engineering Research Council of tometers are used to help the estimation of horizontal position Canada. This paper was recommended by M.-Y. Lee. (Corresponding author: and velocity [17], [18]. Thus, the presence of any magnetic Edward J. Park.) The authors are with the School of Mechatronic Systems Engineering, disturbances in an indoor environment will affect the accuracy Simon Fraser University, Surrey, BC V3T 0A3, Canada (e-mail: of their localization solution. [email protected]). In addition to localization, the estimation of 3-D orien- Color versions of one or more of the figures in this paper are available online at http://ieeexplore.ieee.org. tation for each body segment is required for 3-D posture Digital Object Identifier 10.1109/TSMC.2016.2521823 tracking in a typical MoCap application. Although a 3-D 2168-2216 c 2016 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information. 82 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

2 IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS: SYSTEMS orientation estimate can be obtained by integrating the angular velocities from the tri-axial gyroscope, this integration causes unbounded orientation drift due to the gyroscope’s bias instability [29]. To solve this problem, some researchers have fused a tri-axial magnetometer with the IMUs using fusion algorithms [29], [30]. In the fused system, the orientation drift can be removed by employing an accelerometer and magnetometer as the vertical (the gravity) and horizontal (the Earth’s magnetic field) references, respectively [29]. However, in indoor environments, the Earth’s magnetic field can be easily disturbed by the presence of ferromagnetic objects [31]. Although a constant magnetic disturbance can be effectively identified and removed by a proper magnetometer calibration [32], indoor magnetic disturbances can be from varying sources and change over time. To deal with these magnetic disturbances, model-based sensor fusion [33], threshold- based switching [34], and vector selector [29] approaches were proposed in the literature. Although these approaches are effective for short periods of time (e.g., 5Ð10 s), they are drift- prone under varying disturbances and during longer periods of Fig. 1. Lower body skeleton model and sensors’ locations. time [35]. TABLE I As a result, the development of magnetometer-free algo- PARAMETERS FOR THE LOWER-BODY SKELETON MODEL rithms for human body inertial MoCap has been the focus of many recent studies. Some papers have proposed algorithms for fusing a tri-axial accelerometer and gyroscope with the biomechanical constraints of the human body to estimate the 3-D knee joint angle [9], the knee flexion/extension angle [36], [37], and the elbow angle [38]. However, when it comes to the absolute orientation estimation about the vertical axis (i.e., the yaw angle), these algorithms either fail to estimate it [9], [36], [37]orsufferfromdriftovertime[9]. Although the joint angle provides sufficient information for biomedical applications in which the joint functionality is of main concern, this information is not sufficient for 3-D filter for the purposes of this paper, the algorithm can be gen- motion reconstruction in gaming and sport science applica- eralized to work with any other absolute localization systems tions where absolute 3-D orientation of each body segment is that have similar accuracies when compared with the UWB required. Thus, magnetometers are still essential aiding com- technology (e.g., better than about 15 cm). ponents in the available 3-D full body [18], upper-body [39], The remainder of this paper is organized as follows. In and lower-body inertial MoCap algorithms [21]. Section II, the theory behind the magnetometer-free 3-D lower- The focus of this paper is on a novel magnetometer-free body inertial MoCap is explained. The experimental setup algorithm for 3-D lower-body inertial MoCap, including 3-D and protocol are explained in Section III. The experimental localization, posture tracking, and motion reconstruction. The results showing the 3-D localization and yaw angle tracking proposed method employs UWB localization system in addi- of the lower-body segments during walking, jogging, jump- tion to the IMUs attached to the lower-body segments. Also, ing, and stairs ascending/descending activities are presented it makes use of the fact that with the known inclination of in Section IV. Finally, Section V concludes this paper. the thigh and shank segments, the horizontal components of the hip-to-ankle vector depend on the orientation of the leg segments around the vertical axis. The algorithm uses this II. THEORETICAL METHOD fact to remove the drift in the yaw angle of the thigh and In this section, the proposed algorithm for a magnetometer- shank without using magnetometers. The proposed algorithm free 3-D inertial MoCap aided by UWB localization system has two novel aspects: 1) the UWB localization data is not only is described. As shown in Fig. 1, data from seven IMU sen- used for position tracking, but also aids in the estimation of sors that are attached to the seven lower-body segments and yaw when fused with the IMU in our novel magnetometer-free three UWB tags are used in the proposed method. In addition, loosely coupled localization filter and 2) by aiding the inertial the lower-body skeleton model including its segments, joints, data of the lower-body segments with position data (of the geometric parameters, and the coordinate frames is also shown feet and waist) from the localization filter, a magnetometer- in Fig. 1. The parameters for the lower-body skeleton model free 3-D lower-body posture tracking is developed. Although are explained in Table I. The navigation frame (n)istheref- the UWB localization system is chosen in the proposed fusion erence coordinate system. The body segment frame (b)isthe 83 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

ZIHAJEHZADEH AND PARK: NOVEL BIOMECHANICAL MODEL-AIDED IMU/UWB FUSION 3

Fig. 2. Structure of the proposed MoCap algorithm for the left half of the lower body. coordinate system attached to the uppermost point of each occasion that the magnetic data are being used and that this body segment. The b-frame is a general representation of the static standing calibration happens at a certain location for body segments when not explicitly specified by a segment a very short time. Thus, the magnetic disturbances during the name. The sensor frame (s) is the coordinate system attached calibration are constant and, as mentioned in Section I, are to each IMU. Fig. 2 shows the overall structure of the proposed of the type that can be removed by a proper magnetometer algorithm for the left half of the lower body. Because of the calibration [32]. similarity of the right and left half of the lower body, for the Before capturing the motion, the anatomical model calibra- sake of brevity, the algorithm is shown only for the left side. tion is carried out for every subject. This involves adjusting AsshowninFig.2, the proposed algorithm consists of four the model’s parameters including the length of the rigid seg- parts, including A: tilt Kalman filter, B: localization Kalman ments via manual measurements that are obtained based on filter, C: yaw Kalman filter, and D: motion reconstruction. anatomical landmarks on the body segments, similar to the A typical MoCap algorithm should be able to track the 3-D approach used in [12]. position of the root joint (the waist joint in Fig. 1) and the In the following, ρ and α subscripts denote the quantities 3-D orientation (including the two tilt angles and yaw angle) of for the localization Kalman filter and the yaw Kalman filter, the body segments. In this proposed algorithm, the tilt Kalman respectively. filter estimates the tilt angles (e.g., roll and pitch) for the seven body segments. The localization Kalman filter estimates the A. Tilt Kalman Filter position of the right and left mid foot and the waist as well as The tilt Kalman filter is based on our previous algorithm the yaw angle of the feet and pelvis. Next, the yaw Kalman in [40], which allows accurate determination of roll and pitch filter estimates the yaw angle of the thighs and shanks. Once angles. In this filter, the tri-axial gyroscope and tri-axial all the 3-D position and orientation of the seven lower-body accelerometer data in the body frame of each segment are segments are obtained, the motion reconstruction provides the used in a Kalman filter (Fig. 2) to estimate the normalized necessary correction for visual 3-D motion representation. gravity vector in the body frame (i.e., the third row of the The magnetometer-free MoCap algorithm in Fig. 2 assumes n rotation matrix from the b-frame to n-frame, bR) that the sensor-to-segment calibration has been performed in ⎡ ⎤ priori and that the inertial data from the tri-axial accelerometer cαcβ cαsβsγ − sαcγ cαsβcγ + sαsγ n = ⎣ α β α β γ + α γ α β γ − α γ ⎦ and gyroscope are provided in the b-frame for each segment bR s c s s s c c s s c c s (1) rather than the s-frame of each sensor. This requires finding −sβ cβsγ cβcγ a rotation matrix from the s-frame to b-frame (bR) for each s where α (yaw), β (pitch), and γ (roll) are the rotation angles segment during an initialization phase. At the initialization about the z-, y-, and x-axes, respectively. phase, the human subject stands in such a way that the y-axis Using the estimated last row of the rotation matrix, the of the navigation frame points to the anterior and the x-axis desired roll and pitch angles can be calculated [40]. This tilt points to the right, as shown in Fig. 1. During this phase, Kalman filter outputs the roll and pitch angles for the seven which lasts only for a couple of seconds, the 3-D inertial and segments including the feet (rf and lf ), shanks (rs and ls), magnetic data are entered into our previous algorithm [7]to thighs (rt and lt), and pelvis (pv). These outputs for the left find the rotation matrix from the sensor frame to the navigation n half of the lower body are shown in Fig. 2. frame (s R) for each IMU. Based on Fig. 1, the coordinate frame of each body segment (only the one for the left thigh is shown in this figure) is aligned with the navigation frame B. UWB/IMU Localization Kalman Filter b = n during the initialization, resulting in s R s R. Note that the The novel idea behind our localization Kalman filter is to few seconds of initial sensor-to-segment calibration is the only provide drift-free position and velocity as well as the yaw 84 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

4 IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS: SYSTEMS

angle without using magnetometers. This filter fuses the IMUs elements of the zpv vector and the known tilt angles from the with a UWB localization system for the following three points tilt Kalman⎡ filter as follows: ⎤ rm lm on the lower body: the right and left mid foot ( and ) and z1,pv the waist (w). It is designed to estimate the position and veloc- n = ⎣ β − γ + γ ⎦ pv R11 c pv c pvz2,pv s pvz3,pv ity vectors for these three points, as well as the first row of −sβpv the rotation matrix nR for their three corresponding segments ⎡ ⎤ b z2,pv (rf , lf , and pv, respectively). For the sake of brevity, the algo- ⎢ β ⎥ n ⎢ s pvz1,pv ⎥ rithm is only explained for the points w on pv. By estimating R12 = sγpvsβpv −cγpvz2,pv + sγpvz3,pv + pv ⎣ cβ ⎦ the first row of the rotation matrix and using the tilt angles pv sγ cβ information from the tilt Kalman filter, the yaw angle can be ⎡ pv pv ⎤ z , estimated using 3 pv ⎢ z1,pv ⎥ n = cγ sβ −cγ z , + sγ z , − sγ − n γ + n γ pv R13 ⎣ pv pv pv 2 pv pv 3 pv pv ⎦ − pv R1,2c pv pv R1,3s pv cβpv α = tan 1 (2) pv n / β cγpvcβpv pv R1,1 c pv n ( ) = n n n . n n pv R k pv R11 pv R12 pv R13 (11) where pv Ri,j represents the ith row and jth column in pv R. As shown in Fig. 2, the localization Kalman filter uses By substituting (11)into(8) and using (7)Ð(10), Aρ, Bρ, the tri-axial gyroscope, tri-axial accelerometer, and tri-axial and uρ can be obtained, that is ⎡ ⎤ position and velocity measurements from the UWB sys- I3×3 t I3×3 03×3 tem in addition to the tilt angles information from the tilt ⎣ ⎦ Aρ(k) = 03×3 I3×3 A23(k) (12) Kalman filter. The following system model is employed by 03×3 03×3 pv(k) the localization filter: ⎡ ⎤ ax ay az ⎣ ⎦ xρ(k + 1) = Aρ(k)xρ(k) + Bρ(k)uρ(k) + qρ(k) (3) A23(k) = t k1 k2 k3 (13) yρ(k) = Cρ(k)xρ(k) + mρ(k) (4) 000 T uρ(k) = y (k)T g (14) where xρ is the state vector, Aρ is the state transition matrix, ⎡ A,pv ⎤ Bρ is the input matrix, uρ is the input vector, qρ is the pro- 05×4 ⎣ ⎦ cess noise vector, yρ is the measurement vector, Cρ is the Bρ(k) = −tsβpv tsγpvcβpv tcγpvcβpv −t measurement matrix, and mρ is the measurement noise vector. 03×4 ( ) = ( ) ( ) ( ) T × The state vector xρ k [ pw k vw k zpv k ] is a 9 1 (15) vector where pw and vw are the 3-D position and velocity where k1Ðk3 are as follows: vectors of the waist, respectively; and zpv is the first row of n = γ / β − γ / β the rotation matrix pv R and is equal to k1 ayc pv c pv azs pv c pv k2 =−cγpvcβpvax − cγpvsγpvsβpvay − cγpvsγpvsβpvaz zpv(k) = z1,pv z2,pv z3,pv 2 2 k3 = sγpvcβpvax + (sγpv) sβpvay + (sγpv) sβpvaz. (16) z1,pv = cαpvcβpv z2,pv = cαpvsβpvsγpv − sαpvcγpv The process and measurement noise covariance matrices in the localization Kalman filter, Qρ(k) and Mρ(k), are calculated z3,pv = cαpvsβpvcγpv + sαpvsγpv. (5) using the following equations [6]: The measurement vector yρ, consists of the 3-D position and T Qρ(k) = E qρ(k)qρ(k) velocity measurements from the UWB system, i.e., yρ(k) = ⎡ ⎤ T 1 [ p (k) vUWB(k) ] . Thus, Cρ can be written as 2 UWB ⎢ t I3×3 03×3 ⎥ = ⎣ 2 ⎦ Cρ(k) = I6×6 06×3 . (6) tI3×3 03×3 0 × −tz˜ (k) Next, the following inertial navigation equations are used to 3 3 pv ⎡ ⎤T 1 calculate Aρ and Bρ matrices in the state space model of (3): t2I × 0 × 0 × ⎢ 3 3 3 3 ⎥ A 3 3 ⎣ 2 ⎦ (17) p (k + 1) = p (k) + tv (k) (7) tI × 0 × w w w 03×3 G 3 3 3 3 n T 03×3 −tz˜pv(k) v (k + 1) = v (k) + t R(k)y , (k) − 00g (8) w w pv A pv T p,UWB 03×3 zpv(k + 1) = pv(k)zpv(k) (9) Mρ(k) = E mρ(k)mρ(k) = (18) 0 × , ( ) = − ˜ ( ) 3 3 v UWB pv k I3×3 tyG,pv k (10) where G and A are the covariance matrix of the gyro- ( ) = T where yA,pv k [ ax ay az ] is the bias compensated scope’s and accelerometer’s measurement noise, respectively. output vector of the accelerometer, g is the norm of the grav- By assuming that their noise variances for the gyroscope and σ 2 σ 2 ity vector, and yG,pv is the tri-axial gyroscope measurements. accelerometer are equal to G and A, G and A are set ∼ σ 2 σ 2 The operator denotes a skew-symmetric matrix function of to GI3×3 and AI3×3. p,UWB and v,UWB are the UWB a vector and I3×3 is the 3 × 3 identity matrix. To find Aρ, position and velocity noise covariance matrices at each step. n ( ) Bρ, and uρ, pv R k in (8) is rewritten in terms of the three These matrices are obtained from the UWB system and define 85 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

ZIHAJEHZADEH AND PARK: NOVEL BIOMECHANICAL MODEL-AIDED IMU/UWB FUSION 5 its localization accuracy at each step. Thus, when UWB posi- The process model is based on the thigh gyroscope mea- tion measurement is not accurate (such as the inaccuracies that surement and is the first-order approximation of the strapdown happen as a result of reflections and NLOS conditions), these integration [40]. Thus covariance matrices become higher, which forces the localiza- Aα(k) = I × − ty˜ , (k) (24) tion Kalman filter to rely more on the inertial data rather than 3 3 G lt ˜ the UWB measurements [27]. where yG,lt is the skew symmetric matrix of the thigh gyro- 1) Zero Velocity Update: The ground contact of the feet can scope measurement. yα(k) is a 2 × 1 vector consisting of the improve the localization through the use of ZUPT [19]. Thus, horizontal components of the position vector from the hip to ( ) for tracking the right and left mid foot, the stance detection ankle ( plh−la k ). Since the internal/external rotation of the based on angular rate energy [41] and ZUPT during the stance knee is assumed to be negligible, with the known inclina- period is employed in the localization filter. tion of the thigh and shank (obtained from the tilt Kalman Finally, using the known tilt angles from the tilt Kalman filter), their orientation around the vertical axis (i.e., the yaw ( ) filter and the estimated yaw angle from (2), the rotation angle) will not affect the vertical component of plh−la k . Thus, matrix for each of the three mentioned body segment can be this vertical component does not provide any information for calculated using (1). the yaw angle estimation. By using the information from the 2) Smoothing: In this paper, the RauchÐTungÐ localization filter, yα(k) is calculated as follows: Striebel (RTS) smoothing algorithm [6], a widely used ( ) = ( ) + n ( ) − − T plh k pw k pv R k [ dpv 0 hpv ] (25) smoothing algorithm in navigation applications, is utilized p (k) = p (k) − nR(k) 0 d 0 T (26) in the localization Kalman filter. RTS smoother recur- la lm lf ft sively updates the smoothed estimate and its covariance in ( ) = ( ) = ( ) − ( ) . yα k plh−la k x,y pla k x,y plh k x,y (27) a backward sweep using the following equations: To calculate Cα(k) in (23), yα(k) should be written in terms ( ) = +( ) ( ) −( + ) −1 ( ) Ks k L k Aρ k [L k 1 ] (19) of the state vector xα k (see the Appendix) as + − ⎡ ⎤ ( ) = ( ) + ( ) ( + ) − ( + ) ( )T T ⎡ ⎤ Ls k L k Ks k Ls k 1 L k 1 Ks k (20) hsh + − sβlscγls z ,lt ρ ( ) = ( ) + ( ) ( + ) − ( + ) ⎢ β ⎥ 1 x s k xρ k Ks k [xs k 1 xρ k 1 ] (21) ( )| = ⎢ c ls ⎥ ⎣ ⎦ rlh−la k x ⎣ ⎦ z2,lt + − −cγltsγlshsh where L and L are the “a posteriori” and the “apriori” z3,lt hth + sγlshshsγlt covariance estimates in the Kalman filter; Ks is the smoother + − = C11αxα(k) (28) gain; xρ and xρ are the a posteriori and the aprioristate esti- ⎡ ⎤ − γ − γ T ⎡ ⎤ mates; and xρs is the smoothed state vector of the localization s lthth s lshsh ⎢ ⎥ z1,lt filter. ( )| = ⎢ k3,th ⎥ ⎣ ⎦ rlh−la k y ⎣ 2 ⎦ z2,lt In the end, as shown in Fig. 2 for the left side of the lower −(cγlt) sβlthth − cγltcγlssβlshsh z3,lt body, the localization filter outputs the 3-D position of the feet cγltsγltsβlthth + sγltcγlssβlshsh ( plm, prm) and the waist ( pw) and the 3-D orientation of the = C21αxα(k). (29) feet ( n R, n R) and pelvis ( nR) to the yaw Kalman filter. lf rf pv Then, Cα(k) can be written as C11α C. Yaw Kalman Filter Cα(k) = . (30) C21α The yaw Kalman filter uses the position of the hip and the ankles, the tilt angles of the thighs and shanks, and the rate Finally, the process and measurement noise covariance ( ) ( ) of turn measurements from the IMUs attached to the thighs matrices in the yaw Kalman filter, Qα k , and Mα k ,are n ( ) calculated using the following equations [40]: to estimate the first row of the thigh rotation matrices ltR k n T 2 and R(k), which are needed to calculate the yaw angle of the Q (k) = E q (k)q (k) =−(t) x˜α(k) x˜α(k) (31) rt α α α G thighs and the shanks. Herein, the algorithm is explained for σ 2 T α, 0 the left side of the lower body. Mα(k) = E mα(k)mα(k) = x (32) 0 σ 2 The yaw Kalman filter assumes the same yaw angle for α,y α σ 2 σ 2 the thigh ( lt) and shank. This implies that the yaw angle where α,x and α,y are the measurement variances for the estimation filter assumes the internal/external rotation of the two horizontal components of yα, which depends on the knee to be negligibly small, which is the case for uninjured localization accuracy of the hip and ankle. knees [37]. This filter uses the following system model: D. Motion Reconstruction xα(k + 1) = Aα(k)xα(k) + qα(k) (22) The motion reconstruction algorithm uses the posi- y (k) = Cα(k)xα(k) + mα(k) (23) α tion and orientation data from the above three filters T where xα(k) = [ z1,lt z2,lt z3,lt ] is a 3 × 1 state vector con- (in Sections II-AÐC) to reconstruct the full 3-D motion of the n ( ) sisting of the first row of the rotation matrix ltR k ; and Aα, qα, lower body. As mentioned earlier, the feet localization tends yα, Cα, and mα are the state transition matrix, process noise to be more accurate than the localization of the waist due to vector, measurement vector, measurement matrix, and mea- the use of ZUPT. Thus, as shown in Fig. 3, the motion recon- surement noise vector, respectively, all for the yaw Kalman struction algorithm uses the position of the feet and the 3-D filter. orientation of the lower body segments to find the position of 86 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

6 IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS: SYSTEMS

Fig. 3. Motion reconstruction algorithm. the waist and build the lower body model (in Fig. 1)usingthe following steps. 1) The corrected position of the waist is calculated as ⎡ ⎤ ⎡ ⎤ 0 0 ( ) = ( ) − n ( )⎣ ⎦ − n ( )⎣ ⎦ pw,l k plm k lf R k dft lsR k 0 −h ⎡ ⎤ 0 ⎡ ⎤sh 0 −dpv − n ( )⎣ ⎦ − n ( )⎣ ⎦ ltR k 0 pv R k 0 (33) −h −h th ⎡ ⎤ ⎡pv ⎤ Fig. 4. Xsens MVN suit, UWB tag, and reflective markers on the lower 0 0 body, and the surrounding experimental setup. ( ) = ( ) − n ( )⎣ ⎦ − n ( )⎣ ⎦ pw,r k prm k rf R k dft rsR k 0 −h ⎡ ⎤ 0 ⎡ ⎤ sh 0 dpv as a reference system for the purpose of verification and the − n ( )⎣ ⎦ − n ( )⎣ ⎦ rtR k 0 pv R k 0 (34) reflective markers are placed on the anatomical landmarks + −hth −hpv (Fig. 4). A GoPro Hero3 camera was also used to capture ( ) + ( ) pw,l k pw,r k the subject’s body motion for further comparison. p , (k) = (35) w c 2 It is important to mention that the reflective markers of the optical MoCap system are placed on the anatomical landmarks where pw,l and pw,r are the calculated waist position of the lower body as suggested in [37]. This is in contrast using the left and right legs, respectively, and pw,c is the corrected waist position. to most of the studies that the markers are used in clusters 2) The right and left legs are reconstructed as two kinematic and placed on the IMU [38], [39], [42], [43] to eliminate any ( ) soft tissue and skin movement effects between the optical and chains, using pw,c k as the root position. inertial systems. However, placing the optical markers on the III. EXPERIMENTAL SETUP AND PROTOCOL anatomical landmarks (which is a common practice in camera- based MoCap) allows benchmarking of the inertial MoCap A. Experimental Setup against the optical MoCap rather than comparing the accuracy To evaluate the performance of the proposed algorithm, raw of the inertial and optical systems [37]. inertial data from an Xsens MVN suit (Fig. 4) was collected. The parameters of the proposed Kalman filters were set as Since the focus of this paper is on the lower body, only seven σ 2 σ 2 follows. A and G were obtained from static measurements −4 2 4 −4 2 2 MTx units were embedded in the MVN suit, including one and set to 10 m /s and 10 rad /s , respectively. p,UWB unit on the waist and six units for the right and left thighs, and v,UWB were obtained from the UWB system for each shanks, and feet. Each MTx unit consists of a tri-axial gyro- σ 2 σ 2 measurement. α,x and α,y were obtained from the corre- scope, a tri-axial accelerometer, and a tri-axial magnetometer. sponding diagonal elements of the covariance matrix in the The magnetic data is only used for the initialization phase and localization Kalman filter. the inertial data is collected at the rate of 100 Hz throughout the experiment. As for the UWB-based localization system, Ubisense Series 7000 was used in this paper. This UWB sys- B. Experimental Protocol tem consists of four anchor nodes (“sensors”) and mobile Our laboratory-based experiments were designed to simulate nodes (“tags”) and collects 3-D position data at the rate of the types of activities that frequently happen in MoCap appli- about 10 Hz. As shown in Fig. 4, one slim tag is attached cations for gaming and entertainment. Thus, the experimental to the subject’s waist and two compact tags are attached to trials included walking and jogging around the 1.9×2.3 m rect- the feet. The four UWB sensors are located at each corner of angular test field, jumping on diagonal and around the field, a1.9 × 2.3 m rectangular-shaped test field. The manufacturer- stairs ascending/descending, and stairs ascending/jumping-off specified localization accuracy of the UWB system is within (Fig. 12). Each trial was dedicated to one type of motion 15 cm in 3-D space. A Qualisys optical MoCap system, with with the duration of about 100 s. The latter two experi- an accuracy of 0.25 mm, consisting of eight cameras is used ments were specifically designed to introduce considerable 87 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

ZIHAJEHZADEH AND PARK: NOVEL BIOMECHANICAL MODEL-AIDED IMU/UWB FUSION 7

Fig. 5. (a) Horizontal trajectory of the waist for a typical jumping trial. (b) Vertical trajectory of the left foot for a typical stairs ascending/descending trial. height variations. To provide a statistical analysis, the trials were repeated four times for each type of motion and 20 trials were performed in total. The subject of the experimental trials was a physically ﬁt, 28-year-old male with a height of 180 cm and a weight of 73 kg. The above experimental protocol was approved by the Research Ethics Board of Simon Fraser University.

IV. EXPERIMENTAL RESULTS AND DISCUSSION A. Performance of the UWB/IMU Localization Kalman Filter The performance of the proposed UWB/IMU fusion Kalman filter in 3-D localization and yaw angle estimation is investigated in this section. Fig. 5(a) shows the trajectory of the waist for a typical jumping trial that involves jumping with both legs around the test field. Since the jumps happen quickly and the sampling rate of the UWB is relatively low, the UWB Fig. 6. Yaw angle of the waist for a typical jogging trial and the norm of position measurements are more concentrated in the rest peri- magnetic field during the trial. ods between the jumps than during the jumps. However, as shown in Fig. 5(a), the proposed algorithm can accurately value (180◦ in this case). However, after about 20 s from bridge the UWB gaps during the jumps. Fig. 5(b) shows the the start of the motion, the yaw angle will converge to the estimated vertical trajectory of the left foot during a stair magnetometer-aided solution. The time taken for the esti- ascending/descending trial. As it can be seen in this figure, the mated yaw angle to converge is due to the fact that the filter proposed filter can overcome the fluctuations in UWB mea- requires some motion to build the correlation between the surements in the vertical direction and provides accurate and states and correct for the initial unknown yaw angle. The same drift-free trajectory estimation in the vertical direction. phenomena has been observed in [44] for GPS-aided inertial Fig. 6 (top) shows the result of yaw angle estimation for navigation in an outdoor environment. the pelvis during a typical jogging trial. The reference yaw Fig. 7 shows the 3-D localization root mean square angle is calculated by employing the magnetic data in addi- error (RMSE) for the feet and the waist calculated over the tion to the accelerometer and gyroscope measurements using 20 experimental trials. This figure shows that the algorithm has our previously validated method in [7]. To ensure the accu- a slightly better accuracy for tracking of the feet when com- racy of the reference curve, as shown in Fig. 6 (bottom), the pared with the waist, which is the improvement that ZUPT can experiment took place in a uniform magnetic field, away from bring for foot tracking. In general, the tracking accuracy is bet- magnetic disturbances. As shown by the solid red curve in ter than about 4.5 cm in a 3-D space. This obtained accuracy Fig. 6 (top), since the magnetic data is not used in the pro- is similar to the reported accuracy of about 4 cm in [18]for posed magnetometer-free algorithm, the initial yaw angle is waist tracking using a magnetometer-aided UWB/IMU fusion not known and the algorithm starts with an arbitrary initial Kalman filter. 88 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

8 IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS: SYSTEMS

Fig. 7. Three-dimensional localization accuracy for the feet and the waist.

Fig. 9. Yaw angle estimation under simulated magnetic disturbances from 30 to 80 s.

magnetic disturbances, with the following two specific cases of the latter. The black dotted line (designated as gyr+mag) is the yaw angle solution when the magnetic disturbances are not identified and the magnetometer-aided algorithm fuses both inertial and magnetic data to calculate the yaw angle (case A). Fig. 8. Yaw angle of the left thigh for a typical stairs ascending/descending The blue dotted line (designated as gyr) is the case where the trial. disturbances are identified—i.e., the disturbed magnetic data is removed by the filter and the gyroscope data is used during B. Performance of the Yaw Kalman Filter the disturbance (case B). The red solid curve is the estimated yaw from the proposed magnetometer-free yaw Kalman filter. The result of yaw angle estimation for the left thigh from The error in yaw angle estimation during the period of distur- the yaw Kalman filter for a typical stairs ascending/descending bance is calculated in Fig. 9 (middle). The reference for the trial is shown in Fig. 8. Similar to Fig. 6, the reference curve error calculation is the magnetometer-aided estimation using in Fig. 8 is obtained from our magnetic-aided algorithm [7] the original nondisturbed magnetic field. While the disturbed and the magnetic field is uniform during the trial. As shown in magnetic field does not affect the performance of our proposed Fig. 8, similar to the yaw angle estimation in the localization magnetometer-free algorithm, cases A and B are affected sig- Kalman filter, the unknown arbitrary initial yaw angle fluctu- nificantly as shown in Fig. 9 (middle). The disturbances result ates for about 20 s and then converges to the reference curve. in yaw drift in case B and cause large erroneous spikes in The superiority of the proposed magnetometer-free algo- case A for the magnetometer-aided solution. rithm over the traditional magnetic-aided algorithms manifests itself when yaw tracking is required in the presence of magnetic disturbances. A common method in the literature to C. Lower Body MoCap Accuracy overcome the magnetic disturbances is to rely on the gyro- By placing the reflective markers on the anatomical land- scope for the duration of the disturbances [7], [29], [30]. To marks, the individual Euler angles (roll, pitch, and yaw) for investigate the effect of magnetic disturbances, as shown in the body segments are not accessible from the camera-based Fig. 9 (bottom), a 50 s long magnetic disturbance is added to MoCap. Therefore, it necessitates a different benchmark- the uniform magnetic field. Based on [31], the norm of the ing scheme instead of the direct comparison of each Euler magnetic field varies by about 50% of its nominal value in angle between the systems for each body segment as sug- a typical indoor environment and thus the disturbances are gestedin[16], [21], and [45]. For benchmarking purposes, simulated accordingly. Fig. 9 (top) benchmarks the perfor- the lower body skeleton model in Fig. 2 was formed for the mance of the proposed magnetometer-free algorithm against camera-based MoCap. Similar to the IMU-based MoCap, the our magnetometer-aided algorithm [7] under the influence of camera-based MoCap requires a standing posture calibration to 89 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

ZIHAJEHZADEH AND PARK: NOVEL BIOMECHANICAL MODEL-AIDED IMU/UWB FUSION 9

Fig. 10. (a) Ankle dorsiflexion/plantar flexion. (b) Knee flexion/extension during one gait cycle of a walking trial. (c) Hip abduction/adduction. (d) Hip flexion/extension during 10 s of a jogging trial.

TABLE II map the markers’ position measurements to the skeleton model MEAN AND STANDARD DEVIATION OF LOWER in order to find the required rotation matrices for this mapping. BODY JOINT ANGLE TRACKING Then, the two reconstructed models from the camera-based and IMU-based system are benchmarked against each other in two different ways. First, the lower body joint angles, which are based on the rotation of the lower body segments about the horizontal axis (according to Fig. 1, x-axis for the flexion/extension angles and y-axis for the abduction/adduction angle), are compared between the two models. These angles include the hip abduction/adduction and flexion/extension, and knee flexion/ extension and ankle dorsiflexion/plantar flexion. Since the orientation of the leg around the vertical axis (the estimated yaw angle by the yaw Kalman filter) will not affect these joint angles, they are the angles of interest that avoid the use of the magnetometer [37], [46]. However, the tracking accuracy for those typical lower body joint angles is not sufficient to evaluate the system’s performance for 3-D MoCap in gaming and sport science application. For these applications, to perform 3-D motion reconstruction, the instantaneous 3-D orientation of the segments is of importance. Thus, a second comparison method is devised in this paper that also takes the estimations from the yaw Kalman filter into account. In this comparison, the angle in a 3-D space between the thigh segments of the IMU-based and camera-based models (the thigh error) and the one for the shank segments (the shank error) during the Fig. 11. Estimation error for the thigh and shank 3-D orientation. experimental trials is calculated. Fig. 10(a) and (b) shows the ankle dorsiflexion/plantar flex- summarized in Table II. As shown in this table, the lower ion and the knee flexion/extension for one gait cycle during body joint angle tracking varies from 1.4◦ to 3.5◦ for rel- a walking trial. Fig. 10(c) and (d) shows the hip abduction/ atively slow motions (RMSE values are averaged over the adduction and flexion/extension angle during 10 s of a jog- four walking and the four stairs ascending/descending trials) ging trial. This figure shows that the proposed algorithm can and from 2.1◦ to 4.5◦ for relatively fast motions (RMSE val- accurately track the lower body joint angles during slow and ues are averaged over the four jogging and the four jumping fast dynamic motions. The joint angle tracking accuracies are trials). This is because in higher dynamic activities, the inertial 90 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

10 IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS: SYSTEMS

Fig. 12. (a) Sequences of the recorded motion for a typical stairs ascending/jumping-off trial. (b) Reconstructed motion for the same trial.

sensors and the markers on the human leg move relative to However, in reality, a healthy knee can exhibit up to each other as a result of muscle and skin motions, resulting in about 8◦ [37] of internal/external rotation which affects the higher deviations between the inertial and optical joint angle 3-D orientation of the shank. tracking. In comparison, the reported accuracies in Table II Fig. 12 compares the reconstructed stairs ascending/ are within the accuracy range reported in [43] for lower body jumping-off with the one captured by the GoPro camera. This joint angle tracking using the same inertial sensors but aided figure shows that the proposed algorithm can track the lower by magnetometers. body kinematic well qualitatively as well. The results of the second comparison are shown in Fig. 11. This figure shows the calculated RMSE values for the thigh and shank errors for various activities. As observed in Table II, V. C ONCLUSION Fig. 11 also confirms the slightly higher errors for jumping This paper proposes a novel method of capturing the motion and jogging when compared with walking and stairs ascend- of human lower body including 3-D localization and pos- ing/descending. Additionally, Fig. 11 shows that the error in ture tracking. The algorithm fuses the inertial data from seven the thigh orientation is consistently less than the one for the lower body segments and the UWB localization data on the shank for various activities. This is because the algorithm feet and waist with the biomechanical model of the lower assumes the internal/external rotation of the knee to be negli- body to develop a magnetometer-free lower body MoCap gible, which is in line with many studies in [12] and [37]. solution. The available inertial MoCap methods are aided by 91 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

ZIHAJEHZADEH AND PARK: NOVEL BIOMECHANICAL MODEL-AIDED IMU/UWB FUSION 11

z , magnetometers for drift correction and will get affected by cα (k) = 1 lt (A4) lt cβ long-lasting magnetic disturbances in the indoor environment. lt However, the proposed magnetometer-free method is com- and pletely robust against any type of magnetic disturbances. The performance of the proposed system has been tested p − (k) = p (k) − p (k) lh la la lh experimentally using indoor subject trials including walking, = n ( ) − T + n ( ) − T . ltR k 00 hth lsR k 00 hsh jogging, jumping, and stairs ascending/descending. (A5) The experimental results show that the algorithm can provide the accuracy of better than 4.5 cm in a 3-D space. In Substituting (A-3) and (A-4) into (A-1) and (A-2) and terms of joint angle tracking, the accuracy of the algorithm expanding (A-5), (28) and (29) can be found as the first and is about 3.5◦ and 4.5◦ for knee angle tracking in low and second components of plh−la, respectively. high dynamic motions, respectively. A comparison against the reported results in the literature reveals that the proposed method can reach similar accuracies in localization and pos- ACKNOWLEDGMENT ture tracking as the ones obtained by magnetometer-aided The authors would like to thank EACanada’s Capture methods. However, based on our experimental results, the Laboratory, Burnaby, BC, Canada, for lending the Xsens proposed algorithm outperforms the available magnetometer- MVN suit. aided methods in the presence of magnetic disturbances after an initial convergence time of about 20 s. REFERENCES The proposed system is targeted toward indoor MoCap applications in gaming, film making, sports, augmented reality, [1] X. Zhang and G. Fan, “Dual gait generative models for human motion estimation from a single camera,” IEEE Trans. Syst., Man, Cybern. B, humanÐrobot interactions, etc., where the magnetic distur- Cybern., vol. 40, no. 4, pp. 1034Ð1049, Aug. 2010. bances from the infrastructure and electrical equipment are [2] J. Gu, X. Ding, S. Wang, and Y. Wu, “Action and gait recognition unavoidable. The limitation is that the tracking area has to from recovered 3-D human joints,” IEEE Trans. Syst., Man, Cybern. be covered by the UWB system, which is still significantly B, Cybern., vol. 40, no. 4, pp. 1021Ð1033, Aug. 2010. [3] P. Chen, J. Li, M. Luo, and N. Zhu, “Real-time human motion capture more cost-effective than the typical optical MoCap systems. driven by a wireless sensor network,” Int. J. Comput. Games Technol., However, the algorithm can be generalized to work with vol. 2015, Jan. 2015, Art. ID 695874. any other absolute localization system (such as a vision sys- [4] I. Damian, M. Obaid, F. Kistler, and E. Andre, “Augmented reality using a 3D motion capturing suit,” in Proc. 4th Augmented Human Int. tem) that has similar accuracies to the UWB technology. In Conf. (AH), Stuttgart, Germany, Mar. 2013, pp. 233Ð234. future, we will extend the algorithm to the upper body for [5] A. Shingade and A. Ghotkar, “Animation of 3D human model using a magnetometer-free full-body MoCap solution. markerless motion capture applied to sports,” Int. J. Comput. Graph. Animat., vol. 4, no. 1, pp. 27Ð39, Jan. 2014. [6] S. Zihajehzadeh, T. J. Lee, J. K. Lee, R. Hoskinson, and E. J. Park, APPENDIX “Integration of MEMS inertial and pressure sensors for vertical trajectory determination,” IEEE Trans. Instrum. Meas., vol. 64, no. 3, pp. 804Ð814, DERIVATION OF (28) AND (29) Mar. 2015. Using (1), the rotation matrix for the left thigh, n R(k), can [7] S. Zihajehzadeh, D. Loh, M. Lee, R. Hoskinson, and E. J. Park, “A cas- lt caded two-step Kalman filter for estimation of human body segment be written in terms of the states of the yaw Kalman filter orientation using MEMS-IMU,” in Proc. IEEE Eng. Med. Biol. Soc. zi,lt, i = 1, 2, 3, and the thigh yaw angle αlt Conf., Chicago, IL, USA, Aug. 2014, pp. 6270Ð6273. [8] S. Zihajehzadeh, D. Loh, T. J. Lee, R. Hoskinson, and E. J. Park, n ( ) = “A cascaded Kalman filter-based GPS/MEMS-IMU integration for sports ltR k ⎡ ⎤ applications,” Measurement, vol. 73, pp. 200Ð210, Sep. 2015. z1,lt z2,lt z3,lt ⎣ ⎦ [9] J. Favre, B. M. Jolles, R. Aissaoui, and K. Aminian, “Ambulatory sαltcβlt sαltsγltsβlt + cαltcγlt sαltcγltsβlt − cαltsγlt . measurement of 3D knee joint angle,” J. Biomech., vol. 41, no. 5, pp. 1029Ð1035, Jan. 2008. −sβlt sγltcβlt cγltcβlt [10] E. Roux, S. Bouilland, A.-P. Godillon-Maquinghen, and D. Bouttens, (A1) “Evaluation of the global optimisation method within the upper limb kinematics analysis,” J. Biomech., vol. 35, no. 9, pp. 1279Ð1283, Similar to (A-1) and assuming the yaw angle of the shank Sep. 2002. being equal to the one of the thigh, the rotation matrix for the [11] G. Welch and E. Foxlin, “Motion tracking: No silver bullet, but n ( ) α a respectable arsenal,” IEEE Comput. Graph. Appl., vol. 22, no. 6, left shank, lsR k , can be written in terms of lt pp. 24Ð38, Nov./Dec. 2002. n R(k) = [12] X. L. Meng, Z. Q. Zhang, S. Y. Sun, J. K. Wu, and W. C. Wong, ls ⎡ ⎤ “Biomechanical model-based displacement estimation in micro- cαltcβls cαltsβlssγls − sαltcγls cαltk4,shcγls + sαltsγls sensor motion capture,” Meas. Sci. Technol., vol. 23, no. 5, ⎣sα cβ sα sγ sβ + cα cγ sα cγ sβ − cα sγ ⎦. pp. 055101-1Ð055101-11, May 2012. lt ls lt ls ls lt ls lt ls ls lt ls [13] X. Meng, Z.-Q. Zhang, J.-K. Wu, and W.-C. Wong, “Hierarchical infor- −sβls sγlscβls cγlscβls mation fusion for global displacement estimation in microsensor motion (A2) capture,” IEEE Trans. Biomed. Eng., vol. 60, no. 7, pp. 2052Ð2063, Jul. 2013. [14] Q. Yuan and I. M. Chen, “3-D localization of human based on an iner- Using z1,lt = cαltcβlt, z2,lt = cαltsβltsγlt − sαltcγlt, and = α β γ + α γ α α tial capture system,” IEEE Trans. Robot., vol. 29, no. 3, pp. 806Ð812, z3,lt c lts ltc lt s lts lt, s lt and c lt can be written in Jun. 2013. terms of zi,lt, i = 1, 2, 3 as follows: [15] C.-H. Wu and Y.-C. Tseng, “Deploying sensors for gravity measurement in a body-area inertial sensor network,” IEEE Sensors J., vol. 13, no. 5, sαlt(k) =−cγltz2,lt + sγltz3,lt (A3) pp. 1522Ð1533, May 2013. 92 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

12 IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS: SYSTEMS

[16] Z.-Q. Zhang, L.-Y. Ji, Z.-P. Huang, and J.-K. Wu, “Adaptive informa- [38] H. J. Luinge, P. H. Veltink, and C. T. M. Baten, “Ambulatory mea- tion fusion for human upper limb movement estimation,” IEEE Trans. surement of arm orientation,” J. Biomech., vol. 40, no. 1, pp. 78Ð85, Syst., Man, Cybern. A, Syst., Humans, vol. 42, no. 5, pp. 1100Ð1108, Jan. 2007. Sep. 2012. [39] Y. Tian, X. Meng, D. Tao, D. Liu, and C. Feng, “Upper limb motion [17] S. Pittet, V. Renaudin, B. Merminod, and M. Kasser, “UWB and tracking with the integration of IMU and kinect,” Neurocomputing, MEMS based indoor navigation,” J. Navig., vol. 61, no. 3, pp. 369Ð384, vol. 159, pp. 207Ð218, Jul. 2015. Jun. 2008. [40] J. K. Lee, E. J. Park, and S. N. Robinovitch, “Estimation of attitude [18] J. A. Corrales, F. A. Candelas, and F. Torres, “Sensor data integration for and external acceleration using inertial sensor measurement during var- indoor human tracking,” Robot. Auton. Syst., vol. 58, no. 8, pp. 931Ð939, ious dynamic conditions,” IEEE Trans. Instrum. Meas., vol. 61, no. 8, Aug. 2010. pp. 2262Ð2273, Aug. 2012. [19] E. Foxlin, “Pedestrian tracking with shoe-mounted inertial sensors,” [41] I. Skog, P. Handel, J. O. Nilsson, and J. Rantakokko, “Zero-velocity IEEE Comput. Graph. Appl., vol. 25, no. 6, pp. 38Ð46, Nov./Dec. 2005. detection—An algorithm evaluation,” IEEE Trans. Biomed. Eng., vol. 57, [20] Y. Zhang, J. Tan, Z. Zeng, W. Liang, and Y. Xia, “Monocular camera no. 11, pp. 2657Ð2666, Nov. 2010. and IMU integration for indoor position estimation,” in Proc. IEEE Eng. [42] Q. Li and J. T. Zhang, “Post-trial anatomical frame alignment procedure Med. Biol. Soc. Conf., Chicago, IL, USA, Aug. 2014, pp. 1198Ð1201. for comparison of 3D joint angle measurement from magnetic/inertial [21] Q. Yuan and I.-M. Chen, “Human velocity and dynamic behavior measurement units and camera-based systems,” Physiol. Meas., vol. 35, tracking method for inertial capture system,” Sens. Actuators A Phys., no. 11, pp. 2255Ð2268, Nov. 2014. vol. 183, pp. 123Ð131, Aug. 2012. [43] J. T. Zhang, A. C. Novak, B. Brouwer, and Q. Li, “Concurrent validation [22] Y. Gu, A. Lo, and I. Niemegeers, “A survey of indoor positioning sys- of Xsens MVN measurement of lower limb joint angular kinematics,” tems for wireless personal networks,” IEEE Commun. Surveys Tuts., Physiol. Meas., vol. 34, no. 8, Aug. 2013, Art. ID N63-9. vol. 11, no. 1, pp. 13Ð32, Mar. 2009. [44] A. Waegli, J. Skaloud, P. Tomé, and J.-M. Bonnaz, “Assessment of [23] J. Fortes, D. Nov, P. Komensk, and R. Zetik, “Short-range UWB radar: the integration strategy between GPS and body-worn MEMS sensors Surveillance robot equipment of the future,” in Proc. IEEE Int. Conf. with application to sports,” in Proc. ION GNSS Conf., Texas, TX, USA, Syst. Man Cybern., San Diego, CA, USA, Oct. 2014, pp. 3767Ð3772. Sep. 2007, pp. 2075Ð2084. [24] F. Zampella, A. R. Jiménez, and F. Seco, “Robust indoor positioning [45] Z.-Q. Zhang and J.-K. Wu, “A novel hierarchical information fusion fusing PDR and RF technologies: The RFID and UWB case,” in Proc. method for three-dimensional upper limb motion estimation,” IEEE Int. Conf. Indoor Position. Indoor Navig. (IPIN), Montbéliard, France, Trans. Instrum. Meas., vol. 60, no. 11, pp. 3709Ð3719, Nov. 2011. Oct. 2013, pp. 1Ð10. [46] R. Takeda, S. Tadano, A. Natorigawa, M. Todoh, and S. Yoshinari, [25] J. D. Hol, F. Dijkstra, H. Luinge, and T. B. Schon, “Tightly cou- “Gait posture estimation using wearable acceleration and gyro sensors,” pled UWB/IMU pose estimation,” in Proc. Int. Conf. Ultra-Wideband, J. Biomech., vol. 42, no. 15, pp. 2486Ð2494, Nov. 2009. Vancouver, BC, Canada, Sep. 2009, pp. 688Ð692. [26] F. Zampella, A. De Angelis, I. Skog, D. Zachariah, and A. Jimenez, “A constraint approach for UWB and PDR fusion,” in Proc. Int. Conf. Indoor Position. Indoor Navig. (IPIN), Sydney, NSW, Australia, Shaghayegh Zihajehzadeh (S’15) received the B.Sc. degree in electrical Nov. 2012, pp. 1Ð9. engineering and control systems from the Isfahan University of Technology, [27] S. Zihajehzadeh, P. K. Yoon, B.-S. Kang, and E. J. Park, “UWB- Isfahan, Iran, in 2008, and the M.Sc. degree in electrical engineering/ aided inertial motion capture for lower body 3-D dynamic activity mechatronics from the Amirkabir University of Technology, Tehran, Iran, and trajectory tracking,” IEEE Trans. Instrum. Meas., vol. 64, no. 12, in 2011. She is currently pursuing the Ph.D. degree with the School of pp. 3577Ð3587, Dec. 2015. Mechatronic Systems Engineering, Simon Fraser University, Burnaby, BC, [28] S. Zihajehzadeh, P. K. Yoon, and E. J. Park, “A magnetometer-free Canada, focusing on the development of an accurate and ubiquitous motion indoor human localization based on loosely coupled IMU/UWB fusion,” capture system using wearable technology. in Proc. IEEE Eng. Med. Biol. Soc. Conf., Milan, Italy, Aug. 2015, Her current research interests include sensor fusion, inertial motion capture, pp. 3141Ð3144. and indoor/outdoor localization. [29] J. K. Lee and E. J. Park, “Minimum-order Kalman filter with vector selector for accurate estimation of human body orientation,” IEEE Trans. Robot., vol. 25, no. 5, pp. 1196Ð1201, Oct. 2009. [30] J. K. Lee and E. J. Park, “A fast quaternion-based orientation optimizer via virtual rotation for human motion tracking,” IEEE Trans. Biomed. Eng., vol. 56, no. 5, pp. 1574Ð1582, May 2009. [31] T. H. Riehle et al., “Indoor magnetic navigation for the blind,” in Proc. IEEE Eng. Med. Biol. Soc. Conf., San Diego, CA, USA, Aug./Sep. 2012, pp. 1972Ð1975. Edward J. Park (M’03–SM’13) received the [32] S. A. H. Tabatabaei, A. Gluhak, and R. Tafazolli, “A fast calibra- B.A.Sc. degree from the University of British tion method for triaxial magnetometers,” IEEE Trans. Instrum. Meas., Columbia, Vancouver, BC, Canada, and the vol. 62, no. 11, pp. 2929Ð2937, Nov. 2013. M.A.Sc. and the Ph.D. degrees from the University [33] D. Roetenberg, H. J. Luinge, C. T. M. Baten, and P. H. Veltink, of Toronto, Toronto, ON, Canada, in 1996, “Compensation of magnetic disturbances improves inertial and magnetic 1999, and 2003, respectively, all in mechanical sensing of human body segment orientation,” IEEE Trans. Neural Syst. engineering. Rehabil. Eng., vol. 13, no. 3, pp. 395Ð405, Sep. 2005. He was an Assistant Professor with the [34] A. M. Sabatini, “Quaternion-based extended Kalman filter for determin- Department of Mechanical Engineering, University ing orientation by inertial and magnetic sensing,” IEEE Trans. Biomed. of Victoria, Victoria, BC, Canada, from 2003 to Eng., vol. 53, no. 7, pp. 1346Ð1356, Jul. 2006. 2008. He is currently a Professor with the School [35] M. El-Gohari and J. McNames, “Human joint angle estimation with of Mechatronic Systems Engineering, Simon Fraser University, Surrey, inertial sensors and validation with a robot arm,” IEEE Trans. Biomed. BC, Canada, where he is also the Director of the Biomechatronic Systems Eng., vol. 62, no. 7, pp. 1759Ð1767, Jul. 2015. Laboratory, and a member of the Faculty of Health Sciences. His current [36] G. Cooper et al., “Inertial sensor-based knee flexion/extension angle research interest include biomechatronics and biomedical technologies for estimation,” J. Biomech., vol. 42, no. 16, pp. 2678Ð2685, Sep. 2009. life sciences, rehabilitation and medicine, and mechatronics applied to next [37] T. Seel, J. Raisch, and T. Schauer, “IMU-based joint angle measurement generation vehicular, robotic, and space systems. He has authored/co-authored for gait analysis,” Sensors, vol. 14, no. 4, pp. 6891Ð6909, Apr. 2014. over 100 journal and conference papers in the above areas.

Appendix G.

Experimental evaluation of regression model-based walking speed estimation using lower body-mounted IMU

(Reproduced with permission from IEEE)

Experimental Evaluation of Regression Model-Based Walking Speed Estimation Using Lower Body-Mounted IMU

Shaghayegh Zihajehzadeh, Student Member, and Edward J. Park1, Senior Member, IEEE

Abstract—This study provides a concurrent comparison of as sensor fusion). For instance, the external (gravity regression model-based walking speed estimation accuracy compensated) acceleration is a processed variable obtained using lower body mounted inertial sensors. The comparison is from acceleration and angular velocity using a Kalman filter based on different sets of variables, features, mounting fusion algorithm [7]. Furthermore, the features can be locations and regression methods. An experimental evaluation generally divided into time-domain and frequency-domain was performed on 15 healthy subjects during free walking features. Due to the periodicity of walking, frequency- trials. Our results show better accuracy of Gaussian process domain features of raw inertial variables from a waist- regression compared to least square regression using Lasso. mounted IMU have been extensively used for walking speed Among the variables, external acceleration tends to provide estimation [5]. On the other hand, conventional time-domain improved accuracy. By using both time-domain and frequency- domain features, waist and ankle-mounted sensors result in features (such as minimum, maximum, sum of absolute similar accuracies: 4.5% for the waist and 4.9% for the ankle. values, etc.) have also been used in some other studies for When using only frequency-domain features, estimation the same raw variables in walking speed estimation [1]. accuracy based on a waist-mounted sensor suffers more For each IMU mounting location (waist, foot, etc.), the compared to the one from ankle. choice of variables and features will affect the ultimate walking speed estimation accuracy based on a regression I. INTRODUCTION model. Hypothetically, due to the periodic leg swing during Variation in walking speed is an important indicator of walking, a leg-mounted IMU would be ideal in capturing the changes in health status [1]. For instance, decline in walking periodicity of walking. Thus, frequency-domain features speed is an early marker of the progression of mild cognitive calculated for the inertial variables from a leg-mounted IMU impairment [2]. Additionally, walking speed is an important would be better representatives of walking speed compared indicator of energy expenditure [3], which in turn can be to the ones from a waist-mounted IMU. used to prevent obesity. Thus, longitudinal monitoring of This paper provides a comparative experimental study to walking speed is important for early detection of the disease shed light on the dependency of walking speed estimation to allow for timely intervention enhancing public health. accuracy on various variables, features, mounting locations The advent of affordable wearable inertial sensors (e.g., and regression methods. For the variables, external inertial measurement units or IMU) facilitated considerable acceleration is compared to raw acceleration from a tri-axial research to estimate walking speed in an ambulatory fashion. accelerometer. For the features, the effect of using time- For example, in [4], shank and foot-mounted inertial data are domain and frequency-domain features on the ankle and used to estimate walking speed in each stride cycle by waist inertial data is investigated. For the regression models, integrating acceleration data. Apart from the integration the walking speed estimation accuracy based on a GPR method, which is drift-prone over time, regression models model is compared to a least square regression model using have emerged as a preferred methodology for walking speed Lasso (LSR-Lasso). estimation based on a set of inertial features. A support vector regression (SVR) model based on acceleration data II. THEORETICAL METHOD and a Gaussian process regression (GPR) model based on A. Problem Definition acceleration and angular velocity data from a waist-mounted Considering that walking is represented by a set of inertial IMU are proposed in [1] and [5], respectively, to estimate features, this section is focused on driving a mapping from average walking speed. In [6], a linear regression model is walking-related features (predictors) to walking speed proposed to estimate instantaneous walking speed based on (response value) using a regression model. In a regression acceleration data from a wrist-mounted IMU. problem, a training set (풮) consisting of 푁-number of 퐷- Accuracy of the regression algorithms depends on the set dimensional predictors 𝒙 and noisy observations of the of variables and the extracted features. In inertial walking 𝑖 response value 𝑦 is given (풮 = {(𝒙 , 𝑦 )}푁 ). The goal of a speed estimation, the variables can be divided into raw 𝑖 𝑖 𝑖 𝑖=1 variables (the signals collected directly from the inertial regression model is to find the best-fit function 𝑓(𝒙𝑖) which sensors) and the processed variables (the signals obtained by predicts the response values. The GPR and LSR-Lasso manipulation of raw variables using filtering techniques such models are the two candidate regression methods used in this study.

This work is supported by the Natural Sciences and Engineering B. Gaussian Process Regression Research Council of Canada (NSERC). The objective of GPR is to model the dependency [3]: S. Zihajehzadeh, and E. J. Park are with the School of Mechatronic nd Systems Engineering, Simon Fraser University, 250-13450 102 Avenue, 𝑦𝑖 = 𝑓(𝒙𝑖) + 휀𝑖 (1) Surrey, BC, Canada, V3T 0A3 (email: [email protected], [email protected]). 978-1-4577-0220-4/16/$31.00 ©2016 IEEE 243 95

2 where 휀𝑖 = 푁(0, 휎푛 ) is the independent and identically, normally distributed noise terms. GPR has two main advantages compared to conventional Waist-IMU regression methods, including [3]: 1) It is a non-parametric regression method and the model structure is determined from data. 2) It uses a probabilistic approach which can model the prediction uncertainty. 10 m A Gaussian process is completely identified by its mean 𝜇(𝒙𝑖) and covariance function 휮(𝒙𝑖, 𝒙𝑗). The covariance function used in here is a parameterized squared exponential covariance function: Xsens 1 Ankle-IMU 휮(𝒙 , 𝒙 ) = 휎 2exp (− (𝒙 − 𝒙 )푾−1(𝒙 − 𝒙 )푇) (2) MTw 𝑖 𝑗 𝑓 2 𝑖 𝑗 𝑖 𝑗 where 휎 is the signal variance and 푾 = 𝑑𝑖푎𝑔(𝑙 , … , 𝑙 ) is 𝑓 1 퐷 Fig. 1. Experimental setup, left: a subject wearing MTw units, right: MTw the diagonal matrix of length-scale parameters. This unit and schematic of the test field. covariance function implements automatic relevance 65±10 𝑘𝑔. Informed written consent was obtained from the determination (ARD) since the length-scale values participants and the experimental protocol (No, 2013s0750) determine the effect of each predictor on the regression. was approved by the Office of Research Ethics of Simon Assuming 푁 training samples are available, for a new Fraser University. ∗ input, 𝒙 , the covariance matrix in (2) can be partitioned B. Hardware and Experimental Protocol into two blocks [3]: Raw inertial data is collected from tri-axial accelerometer 휮푁 휮푁∗ and tri-axial gyroscope at the rate of 100 Hz. The sensors are 휮푁+1 = ( 푇 ) (3) 휮푁∗ 𝛼 Xsens MTw units worn by human subjects on the waist and where 휮푁 is the 푁 × 푁 covariance matrix of the training ankle (Fig. 1). Each subject is asked to walk for a distance of ∗ samples; 휮푁∗ includes elements of 휮(𝒙𝑖, 𝒙 ) and 𝛼 30 𝑚 for a total of 12 times in 3 different self-selected represents 휮(𝒙∗, 𝒙∗). A posteriori mean estimation and walking speeds: slow, normal and fast. The subjects are related variance can be given respectively by: asked to keep their walking speed constant during each 30 𝑚 ∗ −1 trial and each trial is repeated for 4 times per speed. To get 𝜇( 𝒙 ) = 휮푁∗ 휮푁 𝒚 (4) 2 ∗ 푇 −1 the reference average walking speed, the floor is divided into 휎 ( 𝒙 ) = 𝛼 − 휮 ∗ 휮 휮 ∗ (5) 푁 푁 푁 three segments of 10 𝑚 long (accurately measured by laser 푇 where 𝒚 = [𝑦1, … , 𝑦푁] . distance measuring tool with sub-centimeter accuracy) as C. Least Square Regression Using Lasso shown in Fig. 1, and the time it takes for the subject to pass The Lasso (least absolute shrinkage and selection each segment is measured using a stopwatch with the operator) is shrinkage and selection method for regularized accuracy of 0.01 𝑠. linear regression. LSR-Lasso, minimizes the usual sum of C. Preprocessing squared errors, with a bound on the sum of the absolute The sensor data from the IMU are low-pass filtered using values of the coefficients to deliver a sparse solution, i.e. a a Butterworth filter with a cut-off frequency of 20 퐻𝑧. Then, set of estimated regression coefficients in which only a small the raw data are processed to get the following variables: number are non-zero [8]. 1) Magnitude of the 3D acceleration (acc): The norm Given a linear regression, the LSR-Lasso solves the (square root of the sum of squares) of acceleration ℓ1-penalized regression so as to minimize [8]: components: 푁 퐷 1 𝑠 2 2 2 푇 2 푎𝑐𝑐 = | 풂| = √ 𝑠푎 + 𝑠푎 + 𝑠푎 (7) ∑(𝑦𝑖 − 𝛽0 − 𝒙𝑖 휷) + 𝜆 ∑|𝛽𝑗| (6) 𝑥 𝑦 𝑧 2 𝑠 𝑖=1 𝑗=1 where 푎𝑖 , 𝑖 = 𝑥, 𝑦, 𝑧 is the acceleration measured by each for unknown parameters 𝛽0 and 휷 = [𝛽1, … , 𝛽퐷]. The axis of the accelerometer in the sensor frame (𝑠-frame: a second term in (6) is the penalty function balancing the fit of coordinate frame attached to the sensor). the model with its complexity with the non-negative 2) Magnitude of external acceleration (ext-acc): This parameter 𝜆 governing this trade-off [8]. The value of 𝜆 is variable is the norm of gravity compensated chosen based on 10-fold cross-validation in this study. acceleration. To get this variable, an attitude solution III. EXPERIMENTAL METHOD (roll and pitch angles) is obtained using our previous A. Participants algorithm in [7],[9],[10] to level the acceleration data and remove the vertical gravitational component: 15 young healthy adults (9 male, 6 female) participated in 푛 푛 𝑠 푛 this study. The participants had the average age of 27±4 풂𝑒𝑥𝑡 = 𝑠푹𝑡𝑖푙𝑡 풂 − 품 (8) 푛 years, average height of 1.69±0.08 𝑚 and average weight of 𝑒𝑥𝑡 − 푎𝑐𝑐 = | 풂𝑒𝑥𝑡| (9) 244 96

푛 where 풂𝑒𝑥𝑡 is the tri-axial vector of external acceleration in waist and ankle-mounted IMU. This brings out an interesting the navigation frame (𝑛-frame: a local-level frame with its fact that even though an acceleration variable aligned with 𝑥- and 𝑦-axis in the horizontal plane and its 𝑧-axis aligned forward movement might seem more correlated with the 푛 with the gravity vector); 𝑠푹𝑡𝑖푙𝑡 is the leveling rotation horizontal walking speed [3], the vertical component of the matrix; and 품 is the gravity vector. acceleration (although perpendicular to the direction of 3) Magnitude of the leveled horizontal plane acceleration forward movement) can provide more useful information for (hor-acc): This variable is the norm of external the regression model due to the up and down motion of body acceleration in the horizontal plane: segments during walking. Thus, among these variables, the norm of external acceleration is selected for the rest of the 푛 ℎ𝑜𝑟 − 푎𝑐𝑐 = | 풂𝑒𝑥𝑡−𝑥𝑦| (10) analysis in parts B and C. Compared to the results in the 푛 where 풂𝑒𝑥𝑡−𝑥𝑦 is the vector of the first two components of literature the reported accuracies in Table I, is much better 푛 than the reported accuracies of around 9~26 𝑐𝑚/𝑠 in [4] 풂𝑒𝑥𝑡. when integration-based method is used to estimate walking D. Feature Extraction speed using a leg-mounted IMU. Each of the above mentioned three variable streams is B. Frequency-domain Features vs. Time-domain Features divided into 5-s epochs. Within each epoch, the following Fig. 2 shows the accuracy degradation of walking speed time-domain and frequency-domain features are calculated. estimation when the frequency-domain and time-domain Time-domain features: The time-domain features used in this features were used separately. The comparison is done study include the statistical features consisting of mean, against the case shown in Table I, where frequency-domain standard deviation, median, mod, mean of absolute values, and time-domain features were used together. Also, in this plus other features including number of mean crossing, comparison, the anthropomorphic features were used along 푁 signal magnitude area (∑푛=1 |𝑥[𝑛]|), and energy with each of the time-domain and frequency-domain feature 푁 2 (∑푛=1 𝑥 [𝑛]). set. Based on Fig. 2, when only the time-domain features Frequency-domain features: A 512-point fast Fourier were used, the accuracy degradation is less than about 15% transform (FFT) is used within each epoch to obtain for both waist and ankle. However, when using only the frequency information. The magnitude of FFT coefficients frequency-domain features, the accuracy degradation is are used as frequency-domain features. significantly higher for the waist compared to the ankle. This emphasizes the importance of using the time-domain Anthropomorphic features: Height and weight of the features when the waist inertial data is being used to subjects are the two anthropomorphic features used in this estimate walking speed. study. To shed further light on this observation, Fig. 3 shows the E. Training versus Testing magnitude of the Fourier transform for the external The walking speed estimation model for subject 𝑛 is acceleration signal from a subject in the three walking speed trained using all data from the remaining subjects and regimes: slow, normal and fast, and the two locations: waist predictions are made on subject 𝑛’s data. The final reported and ankle. As it can be seen, the Fourier transform error is the error averaged across all participants. magnitude for the ankle has a very distinct pattern of peaks in terms of both amplitude and frequency, i.e. the pattern of IV. EXPERIMENTAL RESULTS AND DISCUSSION A. Performance of Gaussian Process Regression Table I shows the accuracy of the GPR model in walking speed estimation for three different variables when frequency-domain, time-domain and anthropomorphic features are used. In this table, the reported mean absolute error (MAE) and the root mean square error (RMSE) are used as measures of accuracy and the standard deviation (SD) shows the precision. Based on this table, there is no significant difference between the accuracy obtained from the waist and the one from the ankle. On the other hand, among the variables, norm of the external acceleration (ext- acc), which is free from gravitational bias, has the best accuracy, whereas the norm of the horizontal Fig. 2. Degradation of walking speed estimation accuracy using the GPR model with reduced features. acceleration (hor-acc) has the least accuracy for both

TABLE I. EVALUATION OF THE WALKING SPEED ESTIMATION ERROR BASED ON GPR MODEL FOR VARIOUS VARIABLES AND IMU MOUNTING LOCATIONS. Waist Ankle Variable MAE (%) SD (%) RMSE (cm/s) SD (cm/s) MAE (%) SD (%) RMSE (cm/s) SD (cm/s) hor-acc 5.5 5.2 6.5 5.2 5.4 4.7 6.6 4.9 ext-acc 4.5 4.3 5.3 4.0 4.9 4.0 6.3 4.2 acc 4.8 4.7 6.0 4.8 5.6 4.9 7.0 5.1

245 97

illustrates the walking speed estimation errors for the two methods (including all experimental trials), show better accuracy of the GPR model compared to the LSR-Lasso model. The RMSE obtained by using the LSR-Lasso model is 12.6 𝑐𝑚/𝑠 in comparison to the one of 6.3 𝑐𝑚/𝑠 when using the GPR model.

V. CONCLUSION This paper investigates the effect of various variables, features and mounting locations of IMU on accuracy of walking speed estimation using regression-based models. Among the three variables, including raw acceleration signal, external (gravity compensated) acceleration and the external acceleration in the horizontal plane, external acceleration shows the best results. Based on subject trials, a GPR model shows superior performance compared to a Fig. 3. Fourier transform of external acceleration norm for three different LSR-Lasso model. In terms of features, a combination of walking regimes of one subject. time-domain and frequency-domain features results in the three distinct peaks moves toward higher frequencies and best walking speed estimation accuracy of larger amplitudes as the walking speed increases. However, 5.3 𝑐𝑚/𝑠 for the waist-IMU. When using only frequency- the peaks for the waist are distributed in a wider range of domain features, the estimation accuracy for the waist will frequencies with relatively less distinct pattern with respect suffer significantly compared to the case of using the to various walking regimes resulting in a relatively smaller combined time-domain and frequency-domain features. In discriminative capacity. This justifies the observation in future, the authors are planning to extend this study to Fig. 2 where the use of only frequency-domain features for include walking speed estimation models using wrist-worn the waist results in more accuracy degradation compared to inertial data. that of the ankle. REFERENCES C. GPR vs. LSR-Lasso Regression [1] M. Schimpl, C. Lederer, and M. Daumer, “Development and Fig. 4 shows the performance of the GPR model versus Validation of a New method to Measure Walking speed in Free- ® LSR-Lasso model for the ankle-mounted IMU when all of Living Environments Using the Actibelt Platform.,” PLoS One, vol. 6, no. 8, p. e23080, Aug. 2011. the frequency-domain, time-domain and anthropomorphic [2] A. Akl, B. Taati, and A. Mihailidis, “Autonomous Unobtrusive features were used. This figure along with Fig. 5, which Detection of Mild Cognitive Impairment in Older Adults,” IEEE Trans. Biomed. Eng., vol. 62, no. 5, pp. 1383–1394, May 2015. [3] H. Vathsangam, A. Emken, E. T. Schroeder, D. Spruijt-Metz, and G. S. Sukhatme, “Determining energy expenditure from treadmill walking using hip-worn inertial sensors: an experimental study,” IEEE Trans. Biomed. Eng., vol. 58, no. 10, pp. 2804–15, 2011. [4] A. Laudanski, S. Yang, and Q. Li, “A concurrent comparison of inertia sensor-based walking speed estimation methods,” in Conf. Proc. IEEE Eng. Med. Biol. Soc., Boston, Massachusetts, Aug. 2011, pp. 3484–3487. [5] H. Vathsangam, B. A. Emken, D. Spruijt-Metz, and G. S. Sukhatme, “Toward free-living walking speed estimation using Gaussian Process-based Regression with on-body accelerometers and gyroscopes,” in Conf. Proc. Pervasive Comput. Technol. Healthc. , Munich, Germany, Mar. 2010, pp. 1–8. [6] B. Fasel, F. Dadashi and K. Aminian, “Instantaneous walking speed estimation for daily life activity monitoring based on wrist acceleration,” in Conf. Proc. Amb. Mon. of Phy. Act. and Mov, Fig. 4. Performance of the GPR and LSR-Lasso models on test data from Limerick, Ireland, Jun. 2015. the ankle-mounted IMU [7] S. Zihajehzadeh, D. Loh, M. Lee, R. Hoskinson, and E. J. Park, “A Cascaded Two - Step Kalman Filter for Estimation of Human Body Segment Orientation Using MEMS - IMU,” in Conf. Proc. IEEE Eng. Med. Biol. Soc., Chicago, Illinois, Aug. 2014, pp. 6270-6273. [8] R. Tibshirani, “Regression Shrinkage and Selection via the LASSO,” J. Royal Statistical Soc. B, vol. 58, no. 1, pp. 267-288, 1996. [9] S. Zihajehzadeh, T. J. Lee, J. K. Lee, R. Hoskinson and E. J. Park, “Integration of MEMS Inertial and Pressure Sensors for Vertical Trajectory Determination,” EEE Trans. Instrum. Meas., vol. 64, no. 3, pp. 804–814, Mar. 2015. [10] J. K. Lee, E. J. Park, and S. Robinovitch, “Estimation of attitude and external acceleration using inertial sensor measurement during various Fig. 5. Box plot of walking speed estimation errors on test data for the GPR dynamic conditions,” IEEE Trans. Instrum. Meas., vol. 61, no. 8, pp. 2262–2273, Aug. 2012. and LSR-Lasso models 246 98

Appendix H.

Regression model-based walking speed estimation using wrist-worn inertial sensor

99 RESEARCH ARTICLE Regression Model-Based Walking Speed Estimation Using Wrist-Worn Inertial Sensor

Shaghayegh Zihajehzadeh, Edward J. Park*

School of Mechatronic Systems Engineering, Simon Fraser University, 250±13450 102nd Avenue, Surrey, BC, V3T 0A3, Canada

* [email protected]

a11111 Abstract

Walking speed is widely used to study human health status. Wearable inertial measurement units (IMU) are promising tools for the ambulatory measurement of walking speed. Among wearable inertial sensors, the ones worn on the wrist, such as a watch or band, have relatively higher potential to be easily incorporated into daily lifestyle. Using the arm OPEN ACCESS swing motion in walking, this paper proposes a regression model-based method for longitu- Citation: Zihajehzadeh S, Park EJ (2016) dinal walking speed estimation using a wrist-worn IMU. A novel kinematic variable is pro- Regression Model-Based Walking Speed posed, which finds the wrist acceleration in the principal axis (i.e. the direction of the arm Estimation Using Wrist-Worn Inertial Sensor. PLoS swing). This variable (called pca-acc) is obtained by applying sensor fusion on IMU data to ONE 11(10): e0165211. doi:10.1371/journal. pone.0165211 find the orientation followed by the use of principal component analysis. An experimental evaluation was performed on 15 healthy young subjects during free walking trials. The Editor: Houbing Song, West Virginia University, UNITED STATES experimental results show that the use of the proposed pca-acc variable can significantly improve the walking speed estimation accuracy when compared to the use of raw accelera- Received: May 17, 2016 tion information (p<0.01). When Gaussian process regression is used, the resulting walking Accepted: October 7, 2016 speed estimation accuracy and precision is about 5.9% and 4.7%, respectively. Published: October 20, 2016

Copyright: © 2016 Zihajehzadeh, Park. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and Introduction reproduction in any medium, provided the original author and source are credited. Walking speed is widely used to study human health status. Based on previous studies, walk-

Data Availability Statement: The experimental ing speed can be used as a marker of mild cognitive impairment (MCI) [1–3]. For example, protocol of this study (No. 2013s0750) was the trajectories of weekly walking speed and the coefficient of variation of the walking speed approved by the Office of Research Ethics of are shown to be among the most important parameters for the early detection of MCI in older Simon Fraser University. This ethical restriction adults [1]. In addition to MCI, walking speed can also be used as a marker of multiple sclerosis prohibits the authors from making the minimal (MS) [4], Parkinson’s disease [5, 6], risk of falls [7], kidney disease [8], and adverse outcomes data set publicly available. However, the in aging [9]. Hence, it can be considered as a powerful predictor of hospitalization, disability, anonymized data are available to all interested researchers upon request. Interested readers may and survival [10, 11]. In a clinical setting, different protocols including the 4-meter [12], contact the Office of Research Ethics of Simon 10-meter [8], and 6-minute walking tests [13] and the timed up and go (TUG) test [9, 14] Fraser University ([email protected]) and Dr. Edward J. have been used as standard tools to evaluate walking speed and gait parameters. However, the Park ([email protected]) to request data. short walking tests (e.g. the 10-meter walking test) are subject to bias due to their brevity [15] Funding: This work was fully funded by the Natural and the longer tests are less accepted due to the space and time constraints in clinical exams Sciences and Engineering Research Council of [16]. Additionally, the walking speed results of clinical tests cannot be fully applied to the

PLOS ONE | DOI:10.1371/journal.pone.0165211 October 20, 2016 1 / 16 100 Regression Model-Based Walking Speed Estimation Using Wrist-Worn Inertial Sensor

Canada (http://www.nserc-crsng.gc.ca/): SPG/ free-living environment [17]. Furthermore, building precise pathological models of some dis- 430592-2012, and Vanier Canada Graduate ease for development of monitoring and treatment guidelines, requires access to longitudinal Scholarship. The funder provided support in the measurements [18,19]. This emphasizes the need for a reliable system/method for longitudi- form of salaries for the first author [SZ] and research materials, but did not have any additional nal (i.e. over long periods of time) and continuous walking speed measurement in real-world role in the study design, data collection and situations. analysis, decision to publish, or preparation of the Aiming at longitudinal walking speed measurement outside the clinical setting, some manuscript. The specific roles of these authors are researchers have used passive infrared (PIR) motion sensors. These PIR sensors can be articulated in the ‘author contributions’ section. mounted on ceiling [20] or walls [21] of a residence and can measure the individuals’ walking Competing Interests: I have read the journal’s speed when they are in the field of view of the sensors. However, walking speed measurement policy and the authors of this manuscript have the based on PIR sensors is limited to confined areas such as hallways. Additionally, such system following competing interest: Provisional US Patent cannot differentiate between multiple residents, limiting its application to independent-living Application No. 62/344,566 filed June 2, 2016 for Systems and Methods for Walking Speed resident homes. Camera-based systems have also been used in the literature for in-home gait Estimation. We confirm that our provisional patent speed measurement [22]. However, camera-based systems can get affected by the lighting con- does not alter our adherence to PLOS ONE policies ditions, and similar to the PIR sensors, they are limited to confined areas and hence more suit- on sharing data and materials. able for clinical settings. Fortunately, with recent advances in MEMS technology and wireless sensor networks, wearable inertial measurement units (IMU) have emerged as powerful devices for portable human motion analysis [23–29]. Being self-contained, wearable inertial sensors can facilitate walking speed measurement in an ambulatory fashion. Considering that the acceleration data from tri- axial accelerometer in a wearable inertial sensor can be integrated to get the velocity, integration-based approaches have been widely used for speed tracking [30]. The main challenge in integration-based approaches is the velocity drift over time that happens as a result of time- varying bias in MEMS-based inertial sensors [31]. To mitigate the drift, some researchers have proposed the detection of periodic foot stance phases during walking to reset the velocity to zero through a process called zero velocity update (ZUPT) [30–34]. However, the need for foot-stance detection requires the wearable sensor to be normally mounted on the leg (ideally on the foot), which is inconvenient for longitudinal walking speed monitoring, particularly indoors. Using waist-worn IMU, some studies have modeled the foot swing in walking as an inverted pendulum to find a 3D walking kinematic model for speed estimation [35]. Addition- ally, using a waist-mounted IMU, linear and nonlinear regression models have shown promising performances for ambulatory walking [13, 36–37] and swimming [38] speed estimation. These regression-based approaches for walking speed estimation are based on mapping the inherent pattern of acceleration and rate of turn information corresponding to the hip rotation in a gait cycle to walking speed. For longitudinal health status monitoring, among the available state-of-the-art inertial sensing-based wearables, wrist-worn devices are the most user-friendly and compliant that do not limit the freedom of movement and do not require specific dressing style (e.g. wearing a belt in the case of waist-worn sensor). Thus, wrist-worn devices have relatively higher potential to be easily incorporated into daily lifestyle and worn for longer hours. Similar to hip rotation in each gait cycle [13], arm swing motion during walking is a periodic motion pattern that is highly correlated to walking speed: the faster the walking speed, the faster the arm swing motion. However, in walking speed estimation based on regression models, free arm motion necessitates the use of more complex algorithms to manipulate the acceleration and rate of turn information and get a variable that is more representative of the arm swing motion. Although extracting this variable is of high importance (because the accuracy of the regression models depends on the set of chosen variables and the extracted features), it has not been addressed in the existing literature. Aiming at accurate walking speed estimation using a wrist-worn IMU, this paper provides a novel processing method based on combined inertial sensor fusion and principal component

PLOS ONE | DOI:10.1371/journal.pone.0165211 October 20, 2016 2 / 16 101 Regression Model-Based Walking Speed Estimation Using Wrist-Worn Inertial Sensor

analysis (PCA) for variable extraction. Experimental results show that the extracted variable can improve the accuracy of wrist-based walking speed estimation.

Theoretical Method Problem Definition Considering that walking is represented by a set of features, this section is focused on formulat- ing a mapping from walking-related features (predictors) to walking speed (response value) using a regression model. In a regression problem, a training set (S) consisting of N-number

of D-dimensional predictors xi and noisy observations of the response value yi is given N (S ¼ fðxi; yiÞgi¼1). The goal of a regression model is to find the best-fit function f(xi) that predicts the response values. The Gaussian process regression (GPR) and regularized least squares regression using least absolute shrinkage and selection operator (LSR-Lasso) models are the two candidate regression methods used in this study.

Gaussian Process Regression The objective of GPR, a well-known non-parametric regression technique, is to model the dependency as follows [39]:

yi ¼ f ðxiÞ þ εi ð1Þ

2 where εi = N(0, σn ) is the independent and identically, normally distributed noise terms. GPR has two main advantages compared to conventional regression methods [40]: 1. It is a non-parametric regression method and the model structure is determined from data. 2. It uses a probabilistic approach that can model the prediction uncertainty.

A Gaussian process is completely identified by its mean μ(xi) and covariance function Σ(xi, xj). The covariance function used here is a parameterized squared exponential (SE) covariance function [39]: ! 1 ð ; Þ ¼ s 2 À ð Þ Σ xi xj f exp À 1 T 2 2ðxi À xjÞW ðxi À xjÞ

where σf is the signal variance and W = diag(l1, ..., lD) is the diagonal matrix of length-scale parameters. This covariance function implements automatic relevance determination (ARD) as the length-scale values determine the effect of each predictor on the regression. Assuming N training samples are available, for a new input, xÃ, the covariance matrix in Eq (2) can be partitioned into two blocks [39]: ! Σ Σ Ã Σ ¼ N N ð Þ Nþ1 T 3 ΣNÃ a

where ΣN is the N × N covariance matrix of the training samples, ΣNÃ includes the elements of Ã Ã Ã Σ(xi, x ) and α represents Σ(x , x ). A posteriori mean estimation and related variance can be given respectively by

Ã À 1 mðx Þ ¼ ΣNÃ ΣN y ð4Þ

2 Ã T À 1 s ðx Þ ¼ a À ΣNÃ ΣN ΣNÃ ð5Þ

T where y = [y1, ..., yN] .

PLOS ONE | DOI:10.1371/journal.pone.0165211 October 20, 2016 3 / 16 102 Regression Model-Based Walking Speed Estimation Using Wrist-Worn Inertial Sensor

GPR is chosen herein because of its superior performance compared to other regression models in [36] where a waist-worn IMU is used to estimate walking speed.

Regularized Least Squares Regression Using Lasso Lasso is the shrinkage and selection method for regularized linear regression. LSR-Lasso, a well-known parametric regression technique, minimizes the usual sum of squared errors, with a bound on the sum of the absolute values of the coefficients to deliver a sparse solution, i.e. a set of estimated regression coefficients in which only a small number is non-zero [41]. Given a

linear regression, the LSR-Lasso solves the ℓ1-penalized regression to minimize [41]: 1 SN ðy À b À x T βÞ2 þ lSD jb j ð6Þ 2 i¼1 i 0 i j¼1 j

for unknown parameters β0 and β = [β1, ..., βD]. The second term in Eq (6) is the penalty function balancing the fit of the model with its complexity with the non-negative parameter λ governing this trade-off [41]. The value of λ is chosen based on 10-fold cross-validation in this study. LSR-Lasso is chosen in this study to provide a performance baseline for GPR with the SE-ARD covariance function.

Experimental Method Participants Fifteen young (nine males, six females) self-reported healthy students from Simon Fraser Uni- versity participated in this study. The participants had an average age of 27±4 years, average height of 1.69±0.08 m, average weight of 6510±10 kg, and average body mass index (BMI) of 23.07±2.3 kg/m2. Informed written consent was obtained from the participants and the experimental protocol (No. 2013s0750) was approved by the Office of Research Ethics of Simon Fra- ser University.

Hardware and Experimental Protocol Raw inertial and magnetic data are collected from tri-axial accelerometers, gyroscopes, and magnetometers at the rate of 100 Hz. The sensor is Xsens MTw worn by human subjects on the wrist (Fig 1). Each subject is asked to walk for a distance of 30 m in indoor environment for three different self-selected walking speed regimes: slow, normal, and fast. The subjects are asked to keep their walking speed constant during each 30 m trial and each trial is repeated four times per chosen speed regime, resulting in 12 trials per subject. To get the ground truth average walking speed, the floor is divided into three segments of 10 m long (accurately measured by a laser distance measuring tool with sub-centimeter accuracy), as shown in Fig 1, and the time it takes for the subject to pass each segment is measured using a stopwatch with an accuracy of 0.01 s. The criterion of line passage is when the subject’s right foot passes the line and a human observer always walked with the subject to ensure a perfect sagittal plane view. For the purpose of demonstrating and further evaluating the performance of the proposed walking speed estimation method in a real-world setting, five subjects (four males, one female) are asked to perform a 12-min outdoor walking trial that includes 2 min of fast walking, 4 min of normal walking, and 6 min of slow walking. In these outdoor trials, Xsens MTi- G-700 [consisting of tri-axial accelerometers, tri-axial gyroscopes, tri-axial magnetometers, and the Global Positioning System (GPS)] is worn by the subjects on their left wrist and the

PLOS ONE | DOI:10.1371/journal.pone.0165211 October 20, 2016 4 / 16 103 Regression Model-Based Walking Speed Estimation Using Wrist-Worn Inertial Sensor

Fig 1. Experimental set-up. Left: a subject wearing MTw units; right: MTw unit and schematic of the test field. doi:10.1371/journal.pone.0165211.g001

reference walking speed is obtained by GPS/IMU fusion using our existing Kalman filter- based fusion algorithm previously presented in [31]. Compared to the indoor trials, these outdoor trials cover longer walking distances and durations and the subjects have the freedom to change their walking direction.

Analysis Variable Computation The raw data are used to calculate three different variables: magnitude of 3D acceleration (acc), magnitude of external acceleration (ext-acc), and external acceleration in the principal axis (pca-acc). The idea here is to start from raw acceleration data and apply step-by-step increasingly more advanced signal processing techniques to process the raw inertial data to get a variable that is more representative of the arm swing during walking. The above-mentioned three variables are explained in the following: acc. The norm (square root of the sum of squares) of acceleration components: qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi s s 2 s 2 s 2 acc ¼ j aj ¼ a x þ a y þ a z ð7Þ

s where ai,i = x,y,z is the acceleration measured by each axis of the accelerometer in the sensor frame (s-frame: a coordinate frame attached to the sensor). ext-acc. This variable is the norm of gravity compensated acceleration (also known as external acceleration). Removing the gravity component from the tri-axial accelerometer data results in a variable that represents the pure acceleration of the arm during walking. The following steps are taken to get the ext-acc variable (Fig 2a): • Orientation is obtained by fusing the tri-axial accelerometer, gyroscope, and magnetometer using our previous Kalman filter-based sensor fusion algorithm in [42–44].

PLOS ONE | DOI:10.1371/journal.pone.0165211 October 20, 2016 5 / 16 104 Regression Model-Based Walking Speed Estimation Using Wrist-Worn Inertial Sensor

Fig 2. Variables calculation method. (a) ext-acc variable and (b) pca-acc variable. doi:10.1371/journal.pone.0165211.g002

n s • The rotation matrix (s R )[42] is calculated to represent the acceleration in the navigation frame (na). The navigation frame (n-frame) is a local-level frame with its x- and y-axis in the horizontal plane and its z-axis aligned with the gravity vector). • The gravity component of the acceleration is then removed:

n n s n a ext ¼ s R a À g ð8Þ

n n where aext and g are the tri-axial external acceleration vector and the gravity vector, respectively, both in the n-frame. • Finally, the ext-acc is calculated as qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi n n 2 n 2 n 2 ext À acc ¼ j a extj ¼ a ext;x þ a ext;y þ a ext;z ð9Þ

pca-acc. This variable is the horizontal external acceleration in the direction of its highest variations. Considering the problem of walking speed estimation based on arm swing motion, n one of the main shortcomings of the above 3D external acceleration ( aext) is its dependency on the direction of arm swing motion in the navigation frame. The first issue is the inter-subject variability of the arm swing angle (i.e. for the same walking speed, the direction of arm swing with respect to the forward direction of motion varies between individuals). The second issue is the intra-subject variability for different walking directions (i.e. for the same walking speed, any changes in the walking direction result in a change in the absolute direction of the arm swing in the navigation frame). In the above-mentioned two scenarios, variations in the n direction of arm swing result in changes in the components of aext. This will affect the magnitude of the ext-acc, which in the regression model will be interpreted as a change in the walking speed. However, for a constant speed, the direction of arm swing should not affect the estimation of walking speed ideally. Thus, the pca-acc is proposed in here as a direction-independent variable. To obtain the pca-acc (Fig 2b), PCA [45] is applied on the first two components (the n horizontal components) of aext to find the direction of the highest acceleration variation in the horizontal plane, which is aligned with the direction of arm swing. The pca-acc variable is sim- ply the acceleration in this direction (the first principal component).

PLOS ONE | DOI:10.1371/journal.pone.0165211 October 20, 2016 6 / 16 105 Regression Model-Based Walking Speed Estimation Using Wrist-Worn Inertial Sensor

Feature Extraction The sensor data from the IMU are low-pass filtered using a Butterworth filter with a cut-off fre-

quency of 20 HZ considering that activities of daily living (ADL) fall in the frequency range of 0.1 to 10 HZ [40]. Each of the above-mentioned three variable streams is divided into 5-s epochs. The 5-s window is selected based on the window size proposed in [36] and that the periodicity of the signal should be captured in this snapshot. Within each epoch, the following time-domain (TD) and frequency-domain (FD) features are calculated. TD features. Eight TD features are used in this study including the statistical features consisting of mean, standard deviation (SD), median, mode, and mean of absolute values plus XN other features such as the number of mean crossing, signal magnitude area ( jx½nj), and n¼1 XN energy ( x2½n). n¼1 FD features. A 512-point fast Fourier transform (FFT) is used within each epoch to obtain frequency information. The first 40 coefficients of the single-sided amplitude spectrum are used as FD features. These 40 coefficients correspond to the frequency range of less than 8 Hz. This frequency range is selected based on the inspection of the Fourier transforms from the 15 subjects.

Training versus Testing Two modeling approaches have been compared in this study: subject-specificmodel versus generalized model. Considering N subjects, to train and test the models, for each subject, 20% of the subject-specificdata set (of the short indoor tests) was randomly sampled and partitioned into test data, the remaining fraction constituting training data. The subject-specific model for subject n was trained using subject n’s training data and was tested on subject n’s test data. The generalized model for subject n was trained using all data from the remaining subjects and predictions were made on subject n’s test data. For each model, the process was repeated for 10 times for different randomly sampled data and the results were averaged. The final reported error is the error averaged across all participants. For the generalized models, along with TD and FD feature sets, two anthropomorphic parameters including height and weight of the subjects are included in the input features. It is shown that these two features can potentially improve the generalizability of the models [46].

Results and Discussion Effect of the PCA on External Acceleration Signal Fig 3 shows the horizontal components of the external acceleration in the directions of x- and y-axis (of the n-frame) in addition to the direction of the principal axis (pca-acc) during 7 s of a 12-min outdoor walking trial. In this figure, the amplitude of the acceleration in the x-axis grows, whereas the one in the y-axis shrinks (happening when the walking direction changes). However, because the pca-acc signal always captures the acceleration in the principal axis (pointing toward the direction of motion), the amplitude of the pca-acc variable remains constant when the walking speed is constant, but the direction changes (see Fig 3). Thus, the pca- acc variable is expected to provide better estimates of walking speed compared to the external acceleration signal in each axis. Similar observation has also been made in [40]; when raw 3D acceleration data are used to estimate energy expenditure, the x-axis acceleration is coinci- dently aligned with the forward direction of motion, providing better accuracy compared to the y- and z-axis. On the contrary, the magnitude of 3D external acceleration (ext-acc) is another variable that will get affected less by the changes in walking direction compared to the

PLOS ONE | DOI:10.1371/journal.pone.0165211 October 20, 2016 7 / 16 106 Regression Model-Based Walking Speed Estimation Using Wrist-Worn Inertial Sensor

Fig 3. TD comparison between the variables. External acceleration signal in the directions of x- and y-axis of the n-frame and the principal axis (pca-acc) during 7 s of outdoor walking trial. doi:10.1371/journal.pone.0165211.g003

acceleration in individual axes. In Fig 4, the pca-acc and ext-acc signals are compared to each other in the FD. This figure shows the Fourier transform magnitude for both pca-acc and ext- acc variables for the three different walking speed regimes (slow, normal, and fast) based on the data set collected from one subject. Based on this figure, compared to ext-acc, the pca-acc variable shares a relatively clearer pattern of peaks between the three walking regimes with the corresponding peaks moving to the higher frequencies and growing in amplitude as the walking speed gets faster. This clear pattern has the potential to provide relatively more accurate estimation of the walking speed.

Performance of the Generalized GPR Model Table 1 shows the walking speed estimation accuracy of the generalized GPR model for the three different variables: pca-acc, ext-acc, and acc. In this table, the reported mean absolute error (MAE) and root mean square error (RMSE) are used as the measures of accuracy, and the SD shows the precision. Based on this table, when using the pca-acc variable, MAE and RMSE are about 5.9±4.7% and 7.9±5.6 cm/s, respectively. Compared to the ext-acc and acc variables, employing pca-acc results in significantly better estimation accuracy. Using analysis of variance (ANOVA), p<0.01 shows that the results are statistically significant. Also shown in Table 1 are the walking speed estimation errors when FD features are not being used (the two anthropomorphic features are used in both cases) in the generalized GPR model. As it can be seen, the effect of removing the FD features on the accuracy obtained by the ext-acc and acc variables is negligible (changes in MAE is <1%), whereas, for pca-acc, the accuracy is changed from 5.9% to 15.6%. This shows that the variations in the frequency spectrum of the pca-acc variable (which in turn is correlated to the changes in frequency and amplitude of arm swing)

PLOS ONE | DOI:10.1371/journal.pone.0165211 October 20, 2016 8 / 16 107 Regression Model-Based Walking Speed Estimation Using Wrist-Worn Inertial Sensor

Fig 4. FD comparison between the variables. Fourier transform magnitude for the pca-acc and ext-acc signals from one subject. doi:10.1371/journal.pone.0165211.g004

is highly correlated to walking speed. The regression analysis and Bland-Altman plots for the predicted walking speed values based on the GPR generalized model using the pca-acc variable (and combined FD and TD features) are shown in Fig 5a and 5b, respectively. The black line in Fig 5a shows the line of best fit (y = 0.903x+11.7) and the gray line shows the ideal line (y = x) representing the perfect correlation between the reference and GPR model predicted walking speed. Although the line of best fit slightly deviates from the ideal line due to the prediction errors, the analysis shows a very strong linear correlation between the predicted and reference walking speed values (Pearson’s r = 0.9742, p<0.001). The Bland-Altman plot shows that, except for a few outliers (mainly at the lower speed range), the error is mainly kept within 95% limits of agreement and that there is no significant systematic dependence of the estimation error on the walking speed. The obtained accuracy using the pca-acc variable is better than the MAE of 6.96% in [20] using ceiling-mounted RF transceivers and the RMSE of 8 to 15 cm/s in [21] using wall-mounted RF transceivers for longitudinal in-home walking speed monitoring is used for the early detection of MCI. The positive results herein demonstrate the potential of the proposed wrist-worn method for the monitoring of walking speed as an early marker of

Table 1. Walking speed estimation error (different variables). Variable Combined FD and TD features TD features only MAE (%) SD (%) RMSE (cm/s) SD (cm/s) MAE (%) SD (%) RMSE (cm/s) SD (cm/s) pca-acc 5.9 4.7 7.9 5.6 15.6 7.0 17.3 7.5 ext-acc 12.13 6.97 14.57 7.78 11.8 6.98 14.5 8.28 acc 14.4 8.26 16.53 8.49 14.24 6.89 14.3 7.66

Evaluation of the walking speed estimation error based on the generalized GPR model for various variables and feature sets. doi:10.1371/journal.pone.0165211.t001

PLOS ONE | DOI:10.1371/journal.pone.0165211 October 20, 2016 9 / 16 108 Regression Model-Based Walking Speed Estimation Using Wrist-Worn Inertial Sensor

Fig 5. Regression and Bland-Altman plots. (a) Regression analysis for the generalized GPR model and (b) Bland-Altman plot for comparison of the reference walking speed values and the predicted values from the generalized GPR model using the pca-acc variable. The upper and lower horizontal lines show the 95% limits of agreement and the middle horizontal line shows the bias. doi:10.1371/journal.pone.0165211.g005

health issues. Compared to the systems based on the ceiling and wall-mounted sensors in [20, 21], which are only applicable to confined hallways in single resident homes, the proposed system offers the advantage of being self-contained and can be easily used indoor/outdoor environments. As the best walking speed estimation accuracy for the GPR model is obtained using the pca- acc variable and a combination of FD and TD features, the reported results in the following sections are based on the same variables and feature sets.

Comparison Between the Generalized GPR and Subject-Specific GPR Models Table 2 shows the walking speed estimation errors for the generalized and subject-specificGPR models in various walking speed regimes: slow (50–100 cm/s), normal (100–150 cm/s), and fast (150–200 cm/s). Based on this table, the generalized and subject-specificmodels have very

Table 2. Walking speed estimation error (different models). Speed regime Generalized model Subject-speci®c model MAE (%) SD (%) RMSE (cm/s) SD (cm/s) MAE (%) SD (%) RMSE (cm/s) SD (cm/s) Slow 8.9 5.6 7.5 4.2 2.6 3.3 2.6 2.6 Normal 4.4 4.3 6.6 5.3 4.5 5.5 7.3 7.0 Fast 4.5 4.1 9.5 7.3 4.5 3.2 8.1 5.5

Comparison between generalized GPR and subject-speci®c GPR models in walking speed estimation for various speed regimes: slow (50±100 cm/s), normal (100±150 cm/s), and fast (150±200 cm/s).

doi:10.1371/journal.pone.0165211.t002

PLOS ONE | DOI:10.1371/journal.pone.0165211 October 20, 2016 10 / 16 109 Regression Model-Based Walking Speed Estimation Using Wrist-Worn Inertial Sensor

similar performances for normal and fast walking speed regimes. However, for slow walking regime, the RMSE of the generalized model is about 7.5 cm/s (MAE of 8.9%), whereas the one for the subject-specificmodel is about 2.6 cm/s (MAE of 2.6%). This can be explained as follows: the generalized model has to fit the model simultaneously across subjects and within each subject. Compared to normal and fast walking, both the frequency and amplitude of arm swing is very weak in slow walking, and for most subjects, the arm swing differs the most at their low- est walking speeds. Intuitively, given that the generalized model has to trade off between overall accuracy and subject-specificaccuracy, the model is optimized over the most similar input points, which correspond to arm swing motion in normal and fast walking regimes. To shed further light on this issue, a separate generalized model is trained for the slow walking regime by excluding the data points that correspond to the velocities of above 100 cm/s. The results show that this new model can reduce the RMSE to 5.1 cm/s (and the MAE to 6.2%). This observation suggests that, if one is interested in estimating slow walking speeds (e.g. tracking walking speed of the elderly), a model that is trained specifically for the speed regime of interest may provide a better accuracy.

Performance of the Generalized LSR-Lasso Model Fig 6 shows the Bland-Altman plots for the predicted walking speed values based on the generalized LSR-Lasso model. Similar to the generalized GPR model, outliers are mainly in the lower speed range and the error is mainly kept within 95% limits of agreement. The RMSE of the estimated walking speed is about 10.7 cm/s (SD = 7 cm/s) and the MAE is 12.78% (SD = 7.7%). Compared to the generalized GPR model, the LSR-Lasso model has a larger prediction error (p<0.01). This comparison shows that the data-driven estimation of the model

Fig 6. Bland-Altman plot. Comparison of the reference walking speed values and the predicted values based on the generalized LSR-Lasso model. doi:10.1371/journal.pone.0165211.g006

PLOS ONE | DOI:10.1371/journal.pone.0165211 October 20, 2016 11 / 16 110 Regression Model-Based Walking Speed Estimation Using Wrist-Worn Inertial Sensor

structure in the GPR model can more effectively capture the influence of each input feature on the output walking speed. A better performance of the GPR model compared to the LSR model has also been observed in [38], where the generalized GPR and LSR models are used to estimate swimming velocity using a waist-worn IMU. In general, once the model is learned, the computational complexity of a nonparametric approach such as GPR for a new estimation depends on the number of training data points (N) and is of order O(N3); whereas the one for the parametric approaches such as LSR-Lasso depends on the dimension of the input data space (d) and is of order O(d3)[40]. The computational cost can be an important factor when implementing these algorithms in resource-constrained platforms such as wearable devices.

Testing of the Generalized GPR Model on Outdoor Free Walking Data In this part, the experimental data from the five outdoor free walking trials are used to examine how well the trained generalized GPR model (based on short indoor walking trials that are limited to straight-line walking) will perform for wrist-based walking speed estimation in real- world environment (where the walking path is not limited to a straight line). Fig 7 shows the estimated outdoor walking speed along with the reference speed from our previously verified GPS-IMU fusion algorithm [31], which is more accurate than using GPS alone, for slow, normal, and fast walking speed regimes during a sample 12-min trial from one subject. It can be seen that the generalized GPR model can clearly differentiate between the three walking speed regimes. The variations of walking speed within each speed regime are expected considering the various turns in the walking path (Fig 7, inset) and the natural variations in free walking speed over time. Comparing the predicted walking speed from the five outdoor trials to the GPS walking speed shows a high correlation between the two measurements (Pearson’s r = 0.916, p<0.001).

Limitations of the Results The experimental results presented in this paper may have some limitations. The first part of the limitations is with regard to the measurement errors of the reference system. For the indoor experiments, although the accuracy of the stopwatch is 0.01 s, the human response time for pressing the stopwatch button is in the order of 0.1 s. Additionally, the proposed method is based on the assumption of free arm swing during walking. In reality, this assumption would not be satisfied in occasions such as carrying a bag, putting hand in the pocket, and walking with walkers. However, although there is no arm swing in these situations, because the arm is now fixed to the trunk, the acceleration profile of the wrist will be similar to that of the trunk. Regression model-based walking speed estimation using trunk acceleration is already addressed in the literature [13, 36]. Thus, in real-world applications, these situations can be identified using a proper activity classification algorithm and a separate regression model should be trained for walking speed estimation in such cases.

Conclusion A regression model-based human walking speed estimation algorithm is presented, which uses the inertial data from a wrist-worn IMU. The arm swing motion is represented by a novel variable called pca-acc, which is highly correlated to walking speed in terms of both temporal and frequency characteristics. Experimental results from 15 young subjects showed that using the proposed pca-acc variable will result in significantly better walking speed estimation accuracy compared to the use of raw acceleration variables (p<0.01). Using combined TD and FD features of the pca-acc variable, a generalized GPR model resulted in accuracy and precision of

PLOS ONE | DOI:10.1371/journal.pone.0165211 October 20, 2016 12 / 16 111 Regression Model-Based Walking Speed Estimation Using Wrist-Worn Inertial Sensor

Fig 7. Outdoor walking speed. Estimated walking speed based on the generalized GPR model and the one from GPS-IMU fusion during a 12-min outdoor walking trial. doi:10.1371/journal.pone.0165211.g007

about 5.9% and 4.7%, respectively. Based on the experimental results, the generalized and subject-specificGPR models tend to perform similarly, except for the slow walking regime (speed <100 cm/s) where a subject-specificmodel provided better estimation accuracy. Compared to the generalized LSR-Lasso, the generalized GPR model performed significantly better (p<0.01) for wrist-based walking speed estimation. Experimental results from a 12-min outdoor walking trial demonstrated the feasibility of using the proposed method for wrist-based walking speed estimation in a real-world environment. In the future, by undertaking a larger study and collecting data across a range of anthropomorphic parameters such as height, weight, and BMI, the generalizability of the proposed method will be further evaluated. Also, the authors plan to carry out clinical investigations to collect real-world data from elderly subjects with diverse ranges of age, weight, and height and to fine tune the regression model for longitudinal walking speed estimation in older adults.

PLOS ONE | DOI:10.1371/journal.pone.0165211 October 20, 2016 13 / 16 112 Regression Model-Based Walking Speed Estimation Using Wrist-Worn Inertial Sensor

Author Contributions Conceptualization: SZ EJP. Formal analysis: SZ. Investigation: SZ. Methodology:SZ EJP. Validation: SZ. Visualization: SZ EJP. Writing – original draft: SZ EJP. Writing – review & editing: SZ EJP.

References 1. Akl A, Taati B, Mihailidis A. Autonomous unobtrusive detection of mild cognitive impairment in older adults. IEEE Trans. Biomed. Eng. 2015; 62(5):1383±94. doi: 10.1109/TBME.2015.2389149 PMID: 25585407 2. Dodge HH, Mattek NC, Austin D, Hayes Tl, Kaye JA. In-home walking speeds and variability trajectories associated with mild cognitive impairment. Neurology. 2012; 78(24):1946±52. doi: 10.1212/WNL. 0b013e318259e1de PMID: 22689734 3. Akl A, Mihailidis A. Estimating in-home walking speed distributions for unobtrusive detection of mild cognitive impairment in older adults. Proc. IEEE Eng. Med. Biol. Soc. 2015 Aug; Milan, Italy. p. 5175± 78. 4. Heesen C, BoÈhm J, Reich C, Kasper J, Goebel M, Gold SM. Patient perception of bodily functions in multiple sclerosis: gait and visual function are the most valuable. Mult. Scler. 2008; 14(7):988±91. doi: 10.1177/1352458508088916 PMID: 18505775 5. Graham JE, Ostir GV, Fisher SR, Ottenbacher KJ. Assessing walking speed in clinical research: a systematic review. J. Eval. Clin. Pract. 2016; 14(4):552±62. 6. Bayle N, Patel AS, Crisan D, Guo LJ, Hutin E, Weisz DJ, et al. Contribution of step length to increase walking and turning speed as a marker of Parkinson's disease progression. PLoS One. 2016; 11(4): e0152469. doi: 10.1371/journal.pone.0152469 PMID: 27111531 7. Bautmans I, Jansen B, Keymolen BV, Mets T. Reliability and clinical correlates of 3D-accelerometry based gait analysis outcomes according to age and fall-risk. Gait Posture. 2011; 33(3):366±72. doi: 10. 1016/j.gaitpost.2010.12.003 PMID: 21227697 8. Abe Y, Matsunaga A, Matsuzawa R, Kutsuna T, Yamamoto S, Yoneki K, et al. Determinants of slow walking speed in ambulatory patients undergoing maintenance hemodialysis. PLoS One. 2016; 11(3): e0151037. doi: 10.1371/journal.pone.0151037 PMID: 27018891 9. Robertson DA, Savva GM A, King-Kallimanis BL, Kenny RA. Negative perceptions of aging and decline in walking speed: a self-fulfilling prophecy. PLoS One. 2015; 10(4):e0123260. doi: 10.1371/ journal.pone.0123260 PMID: 25923334 10. Fritz S, Lusardi M. Walking speed: the sixth vital sign. J. Geriatr. Phys. Ther. 2009; 32(2):46±9. PMID: 20039582 11. Studenski S, Perera S, Patel K, Rosano C, Faulkner K, Inzitari M, et al. Gait speed and survival in older adults. J. Am. Med. Assoc. (JAMA). 2011; 305(1):50±8. 12. Maggio M, Ceda GP, Ticinesi A, De Vita F, Gelmini G, Costantino C, et al. Instrumental and non-instrumental evaluation of 4-meter walking speed in older individuals. PLoS One. 2016; 11(4):e0153583. doi: 10.1371/journal.pone.0153583 PMID: 27077744 13. Schimpl M, Lederer C, Daumer M. Development and validation of a new method to measure walking speed in free-living environments using the Actibelt® platform. PLoS One. 2011; 6(8):e23080. doi: 10. 1371/journal.pone.0023080 PMID: 21850254 14. Greene BR, Kenny RA. Assessment of cognitive decline through quantitative analysis of the timed up and go test. IEEE Trans. Biomed. Eng. 2012; 59(4):988±95. doi: 10.1109/TBME.2011.2181844 PMID: 22207634

PLOS ONE | DOI:10.1371/journal.pone.0165211 October 20, 2016 14 / 16 113 Regression Model-Based Walking Speed Estimation Using Wrist-Worn Inertial Sensor

15. Vaney C, Blaurock H, Gattlen B, Meisels C. Assessing mobility in multiple sclerosis using the River- mead Mobility Index and gait speed. Clin. Rehabil. 1996; 10(3):216±26. 16. Enright PL, McBurnie MA, Bittner V, Tracy RP, McNamara R, Arnold A, et al. The 6-min walk test a quick measure of functional status in elderly adults. Chest J. 2003; 123(2):387±98. 17. Schimpl M, Moore C, Lederer C, Neuhaus A, Sambrook J, Danesh J, et al. Association between walking speed and age in healthy, free-living individuals using mobile accelerometryÐa cross-sectional study. PLoS One. 2011; 6(8):e23299. doi: 10.1371/journal.pone.0023299 PMID: 21853107 18. Jiang Y, Song H, Wang R, Gu M, Sun J. Data-Centered Runtime Verification of Wireless Medical Cyber-Physical System. IEEE Trans. on Ind. Informat. 2016. doi: 10.1109/TII.2016.2573762 19. Jiang Y, Tan P, Song H, Wan B, Hosseini M, Sha L. A Self-Adaptively Evolutionary Screening Approach for Sepsis Patient. IEEE International Symposium on Computer-Based Medical Systems (CBMS). 2016 Aug; Dublin, Ireland. p. 60±65. 20. Hagler S, Austin D, Hayes TL, Kaye J, Pavel M. Unobtrusive and ubiquitous in-home monitoring: a methodology for continuous assessment of gait velocity in elders. IEEE Trans. Biomed. Eng. 2010; 57 (4):813±20. doi: 10.1109/TBME.2009.2036732 PMID: 19932989 21. Jacobs PG, Wan EA, Schafermeyer E, Adenwala F. Measuring in-home walking speed using wall- mounted RF transceiver arrays. Proc. IEEE Eng. Med. Biol. Soc. 2014 Aug; Chicago, IL. p. 914±17. 22. Wang F, Stone E, Skubic M, Keller JM, Abbott C, Rantz M. Toward a passive low-cost in-home gait assessment system for older adults. Proc. IEEE J. Biomed. Heal. Inform. 2013; 17(2):346±55. 23. Zihajehzadeh S, Park EJ. A novel biomechanical model-aided IMU/UWB fusion for magnetometer- free lower body motion capture. IEEE Trans. Syst. Man Cybern. Syst. 2016; doi: 10.1109/TSMC.2016. 2521823 24. Loh D, Lee TJ, Zihajehzadeh S, Hoskinson R, Park EJ. Fitness activity classification by using multi- class support vector machines on head-worn sensors. Proc. IEEE Eng. Med. Biol. Soc. 2015 Aug; Milan, Italy. p. 502±5. 25. Lee JK, Park EJ. A fast quaternion-based orientation optimizer via virtual rotation for human motion tracking. IEEE Trans. Biomed. Eng. 2009; 56(5):1574±82. doi: 10.1109/TBME.2008.2001285 PMID: 19473934 26. Elhoushi M, Georgy J, Noureldin A, Korenberg MJ. Motion mode recognition for indoor pedestrian navigation using portable devices. IEEE Trans. Instrum. Meas. 2016; 65(1):208±221. 27. Zhang Y, Sun L, Song H, Cao X. Ubiquitous WSN for Healthcare: Recent Advances and Future Pros- pects. IEEE Internet Things J. 2014; 1(4):311±8. 28. Ligorio G, Sabatini AM. A novel Kalman filter for human motion tracking with an inertial-based dynamic inclinometer. IEEE Trans. Biomed. Eng. 2015; 62(8):2033±43. doi: 10.1109/TBME.2015.2411431 PMID: 25775483 29. Altini M, Vullers R, Van Hoof C, Van Dort M, Amft O. Self-calibration of walking speed estimations using smartphone sensors. Proc. IEEE Int. Conf. Pervasive Comput. Commun. 2014 Mar; Budapest, Hungary. p. 10±8. 30. Laudanski A, Yang S, Li Q. A concurrent comparison of inertia sensor-based walking speed estimation methods. Proc. IEEE Eng. Med. Biol. Soc. 2011 Aug; Boston, MA. p. 3484±87. 31. Zihajehzadeh S, Loh D, Lee TJ, Hoskinson R, Park EJ. A cascaded Kalman filter-based GPS/MEMS- IMU integration for sports applications. Measurement. 2015; 73:200±210. 32. Foxlin E. Pedestrian tracking with shoe-mounted inertial sensors. IEEE Comput. Graph. Appl. 2005; 25(6):38±46. PMID: 16315476 33. Meng X, Zhang ZQ, Wu JK, Wong WC. Hierarchical information fusion for global displacement estimation in microsensor motion capture. IEEE Trans. Biomed. Eng. 2013; 60(7):2052±63. doi: 10.1109/ TBME.2013.2248085 PMID: 23446028 34. Skog I, Peter H, Nilsson J, Rantakokko J. Zero-velocity detectionÐan algorithm evaluation. IEEE Trans. Biomed. Eng. 2010; 57(11):2657±66. 35. Hu JS, Sun KC, Cheng CY. A kinematic human-walking model for the normal-gait-speed estimation using tri-axial acceleration signals at waist location. IEEE Trans. Biomed. Eng. 2013; 60(8):2271±79. doi: 10.1109/TBME.2013.2252345 PMID: 23529073 36. Vathsangam H, Emken BA, Spruijt-Metz D, Sukhatme GS. Toward free-living walking speed estimation using Gaussian process-based regression with on-body accelerometers and gyroscopes. Proc. Pervasive Comput. Technol. Healthc. 2010 Mar; Munich, Germany. p. 1±8. 37. Zihajehzadeh S, Park EJ. Experimental evaluation of regression model-based walking speed estimation using lower body-mounted IMU. Proc. IEEE Eng. Med. Biol. Soc. 2016 Aug; Orlando, Florida.

PLOS ONE | DOI:10.1371/journal.pone.0165211 October 20, 2016 15 / 16 114 Regression Model-Based Walking Speed Estimation Using Wrist-Worn Inertial Sensor

38. Dadashi F, Millet GP, Aminian K. A Bayesian approach for pervasive estimation of breaststroke velocity using a wearable IMU. Pervasive Mob. Comput. 2015; 19:37±46. 39. Rasmussen CC, Williams CK. Gaussian Processes for Machine Learning. Boston, MA: MIT Press, 2006. 40. Vathsangam H, Emken A, Schroeder ET, Spruijt-Metz D, Sukhatme GS. Determining energy expenditure from treadmill walking using hip-worn inertial sensors: an experimental study. IEEE Trans. Biomed. Eng. 2011; 58(10):2804±15. doi: 10.1109/TBME.2011.2159840 PMID: 21690001 41. Tibshirani R. Regression selection and shrinkage via the Lasso. J. R. Statist. Soc. B 1996; 58(1):267± 288. 42. Zihajehzadeh S, Loh D, Lee TJ, Hoskinson R, Park EJ. A cascaded two-step Kalman filter for estimation of human body segment orientation using MEMS-IMU. Proc. IEEE Eng. Med. Biol. Soc. 2014 Aug; Chicago, IL. p. 6270±73. 43. Zihajehzadeh S, Lee TJ, Lee JK, Hoskinson R, Park EJ. Integration of MEMS inertial and pressure sensors for vertical trajectory determination. IEEE Trans. Instrum. Meas. 2015; 64(3):804±14. 44. Lee JK, Park EJ, Robinovitch SN. Estimation of attitude and external acceleration using inertial sensor measurement during various dynamic conditions. IEEE Trans. Instrum. Meas. 2012; 61(8):2262±73. PMID: 22977288 45. Jolliffe IT. Principal Component Analysis. New York: Springer-Verlag, 2002. 46. Vathsangam H, Emken BA, Schroeder ET, Spruijt-Metz D, Sukhatme GS. Towards a generalized regression model for on-body energy prediction from treadmill walking. Proc. Pervasive Comput. Tech- nol. Healthc. 2011 Jun; Dublin, Ireland. p. 168±75.

PLOS ONE | DOI:10.1371/journal.pone.0165211 October 20, 2016 16 / 16 115