Infrastructure-Free Indoor Pedestrian Tracking with Smartphone Acoustic-Based Enhancement

sensors

Article Infrastructure-Free Indoor Pedestrian Tracking with Smartphone Acoustic-Based Enhancement

Chao Liu *, Sining Jiang, Shuo Zhao and Zhongwen Guo

Department of Information Science and Engineering, Ocean University of China, Qingdao 266100, China; [email protected] (S.J.); snickerﬁ[email protected] (S.Z.); [email protected] (Z.G.) * Correspondence: [email protected]; Tel.: +86-186-602-26768

Received: 16 April 2019; Accepted: 27 May 2019; Published: 29 May 2019

Abstract: Indoor pedestrian tracking has been identiﬁed as a key technology for indoor location-based services such as emergency locating, advertising, and gaming. However, existing smartphone-based approaches to pedestrian tracking in indoor environments have various limitations including a high cost of infrastructure constructing, labor-intensive ﬁngerprint collection, and a vulnerability to moving obstacles. Moreover, our empirical study reveals that the accuracy of indoor locations estimated by a smartphone Inertial Measurement Unit (IMU) decreases severely when the pedestrian is arbitrarily wandering with an unstable speed. To improve the indoor tracking performance by enhancing the location estimation accuracy, we exploit smartphone-based acoustic techniques and propose an infrastructure-free indoor pedestrian tracking approach, called iIPT. The novelty of iIPT lies in the pedestrian speed reliability metric, which characterizes the reliability of the pedestrian speed provided by the smartphone IMU, and in a speed enhancing method, where we adjust a relatively less reliable pedestrian speed to the more reliable speed of a passing by “enhancer” based on the acoustic Doppler effect. iIPT thus changes the encountered pedestrians from an“obstacle” into an “enhancer.” Extensive real-world experiments in indoor scenarios have been conducted to verify the feasibility of realizing the acoustic Doppler effect between smartphones and to identify the applicable acoustic frequency range and transmission distance while reducing battery consumption. The experiment results demonstrate that iIPT can largely improve the tracking accuracy and decrease the average error compared with a conventional IMU-based method.

Keywords: infrastructure-free indoor pedestrian tracking; acoustic signal; Doppler effect; Inertial Measurement Unit; arbitrary wandering

1. Introduction Considered as the last step of mobile service, indoor pedestrian tracking (IPT) has been widely utilized by researchers for the study of indoor location-based applications and services. For example, shopping mall navigation provides convenient shopping guidance and marketing services for customers. Hospitals leverage IPT systems to locate their specific patients in case of sudden accidents. Without IPT applications, people can get lost when they are trying to find their car in a large parking lot. Due to the building materials, GPS is unavailable for IPT, leading to a constant emerging of various smartphone-based tracking methods that have the capability of obtaining location information via Wi-Fi [1], Bluetooth [2,3], LED [4,5], and other techniques. Each smartphone-based IPT method has its pros and cons, and how to develop a low-cost, reliable, and accurate IPT system is still an outstanding research problem. Many attempts have been made in recent years. Yin et al. [6] utilized radio and visual features of a diagram of Wi-Fi fingerprints to realize IPT, but this still requires intensive labor for fingerprint collection. By leveraging additional beacons, Xiang et al. [7] combined Bluetooth and crowdsourcing to estimate pedestrian locations.

Sensors 2019, 19, 2458; doi:10.3390/s19112458 www.mdpi.com/journal/sensors Sensors 2019, 19, 2458 2 of 16

In [8], a generic indoor localization framework is proposed based on existing lighting infrastructure, which is sensitive to moving obstacles. Limited by the demand for extra infrastructures, these methods suffer a relatively high cost. In order to realize infrastructure-free IPT, researchers leverage smartphone in-built sensors, such as inertial measuring units (IMUs), to estimate a pedestrian’s location. IMUs in commercial smartphones have been widely utilized for pedestrian dead-reckoning (PDR) [9], which contain multiple sensors such as accelerometers, gyroscopes, and sometimes magnetometers, and are capable of tracking the movement of pedestrians by detecting steps, estimating stride lengths, and the directions of motion. However, the tracking accuracy of PDR decreases severely when the pedestrian walks around arbitrarily, causing accumulative error. To improve PDR accuracy, researchers [10,11] focus on the enhancement of a pedestrian’s steps detection and stride length estimation, but they still cannot satisfy the actual accuracy requirement due to the complex indoor environment and the pedestrian’s arbitrary walking. Recently, acoustic-based IPT [12] has shown its unique advantages of accurate positioning and direction finding. To be more specific, ultrasonic-based IPT systems [13,14] perform well at estimating location with a high degree of accuracy. Ultrasonic frequency is normally 20–44 kHz (inaudible for human) and easily removes noise. Due to the low velocity of ultrasonic wave traveling in air, high accuracy time of flight (ToF) measurement is allowed. Moreover, the Doppler effect has been widely utilized to track mobile objects [15,16]. However, there are two main limitations that affect the performance of ultrasonic-based IPT. Firstly, similar to other smartphone-based IPT methods, extra infrastructures and self-designed devices are required to guarantee the tracking accuracy by emitting and receiving ultrasonic signals, which is time-consuming and has a high cost. Secondly, ultrasonic signals can be easily blocked by mobile obstacles due to its physical features, causing reflection and multi-path problems, ultimately decreasing tracking accuracy. According to [17], smartphones can emit and receive ultrasonic signals between each other. What if we utilized smartphones only to accomplish IPT by combining the advantages of ultrasonic and IMU methods? In this paper, we propose iIPT, an infrastructure-free pedestrian tracking approach, by combining smartphone-based acoustic and IMU techniques. A metric called pedestrian speed reliability is presented to characterize the reliability of the pedestrian speed provided by the smartphone IMU. We adjust a relatively less reliable pedestrian speed to a more reliable speed of a passing by ”enhancer“ based on the acoustic Doppler effect, thus changing the encountered pedestrians from an “obstacle“ into an ”enhancer“. Extensive real-world experiments in indoor scenarios have been conducted to verify the feasibility of realizing the acoustic Doppler effect between smartphones and to identify the applicable acoustic frequency range and transmission distance while reducing battery consumption. The experiment results demonstrate that iIPT can largely improve the tracking accuracy and decrease the average error compared with a conventional IMU-based method. We have made several contributions as follows:

• We present an infrastructure-free indoor pedestrian tracking approach by combining both smartphone-based acoustic and IMU techniques. • We present a pedestrian speed reliability metric that characterizes the reliability of the real-time pedestrian speed provided by smartphone IMU and reﬂects the arbitrariness of the pedestrian walking pattern. • We determine the capability and precision of measuring speed using sonic Doppler from a smartphone by designing a robot car whose speed could be controlled to compare the real speed. • We leverage the acoustic Doppler effect to adjust the relatively less reliable pedestrian speed to a more reliable speed of a passing by “enhancer” measured by IMU. • We implement comprehensive experiments to identify the applicable acoustic frequency range, transmission distance, and battery consumption and demonstrate that iIPT can largely improve the tracking accuracy and decrease the average error, compared with PDR.

The remainder of this paper is organized as follows: Section2 illustrates our motivation and challenges for proposing iIPT. In Section3, an overview of iIPT is presented and Section4 elaborates Sensors 2019, 19, 2458 3 of 16 our indoor pedestrian tracking approach. Section5 evaluates the performance of iIPT in real situations. In Section6, we discuss about the limitations and unsolved problems for iIPT. Finally, Section7 provides a brief conclusion.

2. Motivation and Challenges In a common indoor environment, a pedestrian’s attention can be easily drawn by many places of interest, leading to different walking patterns. For indoor pedestrian tracking with smartphone IMUs, a sudden stop or speed up will increase the complexity of walking detection and lower the tracking accuracy. In this scenario, we present two important observations. Observation 1. A pedestrian usually walks arbitrarily with an unstable speed in an indoor environment. According to research [18,19] on human mobility models in indoor environments, pedestrian mobility can be inﬂuenced by “social forces,” including certain motion requirements, other, disturbing pedestrians, and various attractive effects. For example, customers are commonly attracted by clothing shops or restaurants when shopping in a mall. Visitors occasionally concentrate on gorgeous art work at exhibitions when they have great interest in it. These kinds of cases happen quite often in indoor environments and cause problems for IMU-based indoor pedestrian tracking, such as PDR, a common IPT method that is capable of tracking pedestrians by estimating their stride frequency, stride length, and walking direction. Observation 2. The indoor pedestrian tracking accuracy of the IMU-based method decreases severely when a pedestrian walks arbitrarily. As shown in Figure1, when a pedestrian walks normally with uniform speed, the value of acceleration sensed by accelerometers follows a regular pattern so that a pedestrian’s stride frequency can be easily obtained and we can directly utilize their average stride length to estimate the walking distance. The result (see Figure 2(a)) depicts that PDR performs well in tracking a pedestrian with arbitrary walking. The ground truth represents the pedestrian’s actual walking distance, which is obtained by a manual method. After making ﬁxed marks on the ground, we utilize a stopwatch to record the time that a pedestrian steps on the marks. However, Figure 2(b) shows that the accumulative distance bias of PDR keeps increasing if we utilize a conventional PDR method to analyze the acceleration data caused by arbitrary walking. The estimated stride frequency will be larger than the actual value, and the average stride length will not make any sense. Some researchers [10,11] have used a machine learning algorithm to optimize PDR, but these methods require plenty of pedestrian history trajectory as training data and the PDR accuracy relies too much on the quality of the training result.

0.6

0.4

0.2

−0.2

Acceleration(g) −0.4 Normal Walking Arbitrary Walking −0.6

−0.8 0 5 10 15 20 25 30 35 Time(s)

Figure 1. A value change of acceleration sensed by smartphone inertial measuring units (IMUs) under different pedestrian walking patterns.

Based on our observations mentioned above, we have done relevant research, as follows. Indoor Pedestrian Dead-Reckoning. IMU is widely utilized in scenarios of indoor PDR. Jimenez et al. [20] proposed a self-designed IMU attached to the foot of a person for pedestrian Sensors 2019, 19, 2458 4 of 16 tracking, leveraging accelerometers, gyroscopes, and magnetometers to detect step detection, stride length, and heading. Other researchers leverage smartphone-based IMUs to accomplish PDR. In [10,11], researchers use a machine learning algorithm to optimize PDR, but these methods require plenty of pedestrian history trajectory as training data, and the PDR accuracy relies too much on the quality of the training result. Other attempts [21–23] have also been made to improve PDR accuracy. However, how to improve its accuracy is still an open issue that needs to be addressed.

2 Arbitrary Walking Ground Truth 0 0 5 10 15 20 Time (s) (a) Normal walking (b) Arbitrary walking

Figure 2. Accumulative distance bias caused by pedestrian dead-reckoning (PDR) with different pedestrian walking patterns.

Ultrasonic Applications on Smartphones. Basic research [24] has been made to evaluate the innate ability of mobile phone speakers to produce ultrasonic signals under different volume levels, but the ultrasonic emit/receive ability, the transmission distance, and the battery consumption of smartphones are not mentioned. A few researchers have taken advantage of ultrasonic physical features to accomplish high precision measurements. Peng et al. [25] designed a high-accuracy acoustic-based ranging system, which can be easily applied to smartphones and relies on the TOA and TDOA methods to calculate distance. Other researchers [26–29] are working on indoor pedestrian or robot localization and tracking at long distance (more than 20 m) by leveraging ultrasonic signals with smartphones. However, most of them require a self-designed ultrasonic emitter or receiver array based on their own demand, which is time-consuming and has a high cost. Moreover, the tracking accuracy can be easily influenced by obstacles between the emitter and the receiver. In a word, all these methods cannot be directly utilized in our scenario. Ultrasonic Doppler Effect. The Doppler effect has been widely utilized in many scenarios of mobile object tracking with smartphones. Wang et al. [15] proposed a device-free method to track gestures of a hand/finger. Based on the Doppler effect, they analyzed the acoustic phase to obtain movement direction and distance measurements. In order to provide an enjoyable user experience, Mao et al. [30] developed a high-precision acoustic tracker to replace a traditional mouse and allow a user to play VR games by moving a smartphone in the air. However, both of these are operated at short distances (from 1 cm to 2 m). Similar to our scenario, DopEnc [17] is an acoustic-based encounter profiling system on smartphones, using the Doppler effect method to calculate relative speed and build an encounter profile. Swadloon [12] performs an accurate acoustic direction finding scheme, according to the arbitrary pattern of phone shaking in a rough horizontal plane. We are motivated to deal with pedestrian arbitrary walking with unstable speeds and to leverage the smartphone acoustic Doppler effect to enhance the tracking accuracy of the IMU-based method. In this paper, we consider that the acoustic signals with frequencies from 17 to 20 kHz is extremely Sensors 2019, 19, 2458 5 of 16 similar to ultrasonic signals and thus call it sub-ultrasonic, which represents the particular acoustic signals we apply throughout the paper. The elaboration of sub-ultrasonic signals is shown as follows. Ultrasonic has been widely utilized in high accuracy detection and measurement. However, according to our experiment result (see Table1), commercial smartphones can only emit and receive acoustic less than 20 kHz, and more than 17 kHz is inaudible for young babies and other, sensitive people. More detail is illustrated in Section 4.2. In the next section, we present an overview of iIPT.

Table 1. Frequency range of acoustic from commercial smartphones.

Smartphone OS Frequency Reference HTC G1 Android HTC Hero Android 17–22 kHz Ref [9] HiPhone 3GS ios Nokia 6210 Navigator Android Samsung Galaxy S5 Android 17–23 kHz Ref [7] HiPhone 6S ios Samsung Galaxy S7 Android Our research iPhone 7 ios 17–20 kHz (phone-to-phone) iPhone x ios

3. Overview In this section, we ﬁrst present the basic idea for iIPT and then give a brief introduction for IMU-based PDR and the acoustic-based Doppler effect. Moreover, we identify three key issues for iIPT, which will be addressed in Section 3.4.

3.1. Basic Idea Unlike existing ultrasonic IPT systems, iIPT requires no extra infrastructure and only utilizes smartphones to improve tracking accuracy. In our indoor scenario (see Figure3), the pedestrians are separated into two groups: users and encounters. In most cases, the encounter will be treated as an “obstacle” who has the possibility of decreasing the tracking accuracy. For example, encounters nearby may change the received signal strength indication (RSSI) so that Wi-Fi signal ﬁngerprints becomes unreliable. In quite the opposite, we take the most advantage of encounters and consider them as an “enhancer” for iIPT instead.

((((

Figure 3. Indoor pedestrian encounter model for infrastructure-free indoor pedestrian tracking (iIPT). Sensors 2019, 19, 2458 6 of 16

When the user walks arbitrarily with unstable speed, IMU data becomes unreliable, leading to accumulative distance bias by PDR. The user discontinuously emits a sub-ultrasonic signal, and once an encounter with more reliable IMU data receives the signal, iIPT begins to calculate their relative speed based on the sub-ultrasonic Doppler effect. The user’s absolute speed will be obtained by taking both the relative speed and the encounter’s speed from IMU into account. Finally, the user’s walking distance can be estimated by integrating their walking speed during a certain period. Due to the sub-ultrasonic physical feature (straight line traveling and low penetration), we only focus on the face-to-face pedestrian encounter scenario in a hallway-like indoor space.

3.2. Shortcomings of PDR IMU-based PDR plays a vital role in indoor pedestrian tracking. Embedded in smartphones, accelerometers and gyroscopes can detect pedestrian stride frequency and heading direction so that walking trajectory (or position) can be estimated. However, IMU-based PDR relies too much on a pedestrian’s walking pattern having a uniform speed and is sensitive to the change in a pedestrian’s walking gestures in practical situations. There are two main reasons causing unacceptable accumulating errors that grow with the path length: (1) gravity inﬂuence; (2) the drift problem. To minimize this error, plenty of attempts have been made, but mostly by leveraging additional infrastructures.

3.3. Basics of Doppler Effect As an observer moves relative to a wave source, the frequency of the wave changes, which is called the Doppler effect [17]. In our case, due to the relative motion between two pedestrians, the sub-ultrasonic emitted by one smartphone takes slightly less or more time to reach the other smartphone, which mainly depends on whether the two pedestrians are moving closely or far away. The frequency offset is determined by the relative velocity between two pedestrians (as shown in Equation (1)). ∆v ∆ f = × f (1) c e where ∆v represents the relative velocity between two pedestrians. c is the speed of acoustic traveling in air, which is 340 m/s. fe denotes the emitted frequency of acoustic, and frequency offset ∆ f can also be calculated through ∆ f = fr − fe once we obtain the received frequency fr.

3.4. Key Issues In order to realize an infrastructure-free indoor pedestrian tracking approach with a smartphone sub-ultrasonic-based Doppler effect, we have to deal with three key issues listed as follows: Key Issue 1. In our scenario, iIPT adjusts a relatively less reliable pedestrian speed to a more reliable speed of a passing by “enhancer” based on the sub-ultrasonic Doppler effect. In terms of the acceleration data sensed by accelerometers, how to deﬁne a proper metric to characterize the reliability of the pedestrian speed provided by smartphone IMU is investigated. Key Issue 2. Little research has been done on the sub-ultrasonic Doppler effect between two smartphones, so other relevant methods cannot be directly leveraged in our scenario. Due to the variety of commercial smartphones (Android or iOS), their ability to emit and receive sub-ultrasonic may be different from each other. It is a key issue to identify the applicable sub-ultrasonic frequency range and transmission distance for the Doppler effect while taking battery consumption into consideration. Key Issue 3. When two pedestrians are moving face-to-face, their walking speed is unknown and the speed value calculated through the Doppler effect is relative rather than absolute. Without a baseline, how to enhance the user’s walking speed to ﬁnally improve the pedestrian tracking accuracy becomes a key issue. This section presents a big picture for iIPT, including the elaboration of our basic idea, IMU-based PDR, the Doppler effect, and three key issues. We detail our core method in Section4. Sensors 2019, 19, 2458 7 of 16

4. Proposed Approach Starting with the problem statement, this section elaborates our proposed method by leveraging both experimental and theoretical analysis.

4.1. Problem Statement There are only two pedestrians in our encounter model: one is the “user” and the other is the “encounter”. To better illustrate our proposed method, we ﬁrstly make several deﬁnitions as follows:

Deﬁnition 1. (DopE Direction). DopE direction is deﬁned as the sub-ultrasonic propagating direction, which is along the straight line from the sub-ultrasonic emitter to the receiver.

Deﬁnition 2. (DopE Speed). DopE speed ∆v is deﬁned as the relative speed in the DopE direction between user and encounter, calculated by the sub-ultrasonic Doppler effect:

f − f ∆v = c × r e (2) fe where c is the speed of sub-ultrasonic traveling in air, which is 340 m/s, fe denotes the emitted frequency of sub-ultrasonic, and fr represents the received frequency.

Deﬁnition 3. (DopE Angle). The DopE angle θ is deﬁned as the angle between the pedestrian’s walking direction and the DopE direction: L sin θ = (3) D where L represents the width of the hallway, and D represents the shortest distance between user and encounter (shown in Figure4).

We assume that the two pedestrians’ walking direction is parallel in most cases of indoor hallway walking situations. Therefore, θ = β.

Deﬁnition 4. (Pedestrian Speed Reliability). Pedestrian speed reliability ρ is deﬁned as the variance value of acceleration sensed by smartphone IMU.

y encounter x 훽

D L

user 휃

Figure 4. Pedestrian encounter model for the calculation of the DopE angle and the DopE speed. Sensors 2019, 19, 2458 8 of 16

The reliability is lower when the pedestrian is walking arbitrarily. Therefore, we consider ∆ρ as a threshold and utilize the encounter’s speed as a benchmark when ∆ρ is below a certain value.

∆ρ = ρencounter − ρuser. (4)

4.2. Sub-Ultrasonic Doppler Identiﬁcation Due to the variety of commercial smartphones (Android or iOS), their ability to emit and receive sub-ultrasonic are different from each other. In order to obtain the sub-ultrasonic frequency range (especially the upper limit) and a reliable communication distance, we carried out a few experiments in a practical situation.

4.2.1. Sub-Ultrasonic Frequency Scientifically, ultrasonic is defined as sound waves with frequencies higher than the upper audible limit of human hearing (20 kHz). According to [15], commercial smartphones can emit and receive 17–22 kHz acoustic, but no further description is presented in these studies. Moreover, these studies are implemented in a phone-to-infrastructure situation, which is different from a phone-to-phone scenario. To find more information, we carried out an experiment on an acoustic frequency range under a phone-to-phone situation. By leveraging both Android and iOS smartphones, we continuously emitted and received acoustic from 17 to 22 kHz, and the result is depicted in Figure5, showing that both Android and iOS smartphones have the ability to emit and receive acoustic from 17 to 20 kHz under a phone-to-phone situation.

300

250

200

150 Energy 100

0 0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 Frequency(Hz) 104 (a) Sub-ultrasonic spectrum diagram (Android) (b) Sub-ultrasonic time-frequency diagram (iOS)

Figure 5. Identiﬁcation of smartphone-based sub-ultrasonic frequency after Fourier transformation.

4.2.2. Sub-Ultrasonic Transmission Distance According to [29], the sub-ultrasonic transmission distance under a phone-to-infrastructure situation is at least 20 m (even higher, depending on the infrastructure type), and usually the smartphone works as a receiver and the infrastructure works as an emitter so that a longer transmission distance is guaranteed. In this paper, both the emitter and receiver are smartphones, leading to new requirements for the study on phone-to-phone sub-ultrasonic scenarios. The sub-ultrasonic transmission distance is mainly determined by a smartphone’s volume level. The higher the volume we utilize, the longer the distance it will transmit, but it will also generate a more noisy sound at the same time. We can see from Figure6 that a smartphone sub-ultrasonic signal can be unreliably detected by another phone from 8 m away. Moreover, as distance increases, the success rate of receiving sub-ultrasonic with different volumes drops. To be more detailed, 25% of the volume of sub-ultrasonic performs terribly, even in a short distance. Within 4 m, the success rate of volume level 12 and level 16 are both 100%, and as the distance increases, a higher volume sub-ultrasonic transmits more reliably. However, when the distance reaches 8 m, volume level 16 performs better than level 8 because of its higher power, but its performance is worse than level 12 due to the more noisy sound generated. Taking the density of indoor pedestrians into account, this sub-ultrasonic transmission Sensors 2019, 19, 2458 9 of 16 distance analysis provides a fundamental reference for the sub-ultrasonic Doppler effect in the indoor pedestrian encounter scenario.

100

Successful Rate (%) 30 volume level 4 20 volume level 8 volume level 12 volume level 16 10

0 0 1 2 3 4 5 6 7 8 Distance (m)

Figure 6. The relationship between the success rate of receiving sub-ultrasonic by smartphones and sub-ultrasonic transmission distance with different smartphone volumes.

4.2.3. Pedestrian Moving Detection In most cases, the Doppler effect is utilized to fulfill high precision movement detection in a short distance [15,30]. To demonstrate the feasibility of leveraging the Doppler effect to obtain an encounter profile, we design a standstill-slow-faster mode experiment based on our experiment analysis above. At first, the pedestrian begins to leverage the smartphone to emit 19.5 kHz sub-ultrasonic but has to remain standstill for a while. Then they start to walk slowly towards the other phone and afterwards walk fast. We store the received audio data in a PCM file, we cut the audio files into several parts, which are 0.2 s long, and the frequency of each audio file is calculated by Fourier transform one by one. In this experiment, the length of every audio file is a key parameter. We hope to minimize the length of each separated files as much as possible so that every frequency calculated by Fourier can be regarded as the instantaneous frequency. However, at a fixed sampling rate, while reducing the cutting interval, the accuracy of the frequency will be decreased. Correspondingly, the accuracy of velocity will be cut down. After our repeated experiments, we discover that 0.2 s is a decent choice for balancing the accuracy and instantaneity, which achieves a 0.1 m/s velocity error. However, for the accuracy deduced by a 0.1 s file, the error will be above 0.3 m/s. As shown in Figure7, the frequency of the received sub-ultrasonic becomes higher after the pedestrian speeds up. In a word, the Doppler effect is able to detect pedestrian walking in an indoor pedestrian encountering scenario.

Figure 7. Pedestrian walking detection by utilizing the Doppler effect with 19.5 kHz sub-ultrasonic. Sensors 2019, 19, 2458 10 of 16

4.2.4. Robot Car Moving Detection Since we cannot measure the exact instantaneous speed of the volunteer at a certain instant, we designed a robot car (shown in Figure 9c). We ﬁxed the mobile phone on the robot car, which was controlled by Arduino. It can be programmed to move forward at an accurate speeds of 0.3, 1, 2, and 3 m/s. We carried out experiments in two experimental environments, that is, in a hallway (shown in Figure 9b) and outside Figure 9b. In the standstill, slight, slow, normal, and fast moving states of the car, we compared the real speed and the measured speed of the robot car after the robot car’s speed was stable. The result is shown in Figure8. Compared with the indoor environment, the outdoor environment measurement results are more unstable.

4 4 Actually speed Actually speed Calculate speed Calculate speed 3.5 3.5

3 3

2.5 2.5

2 2 Speed(m/s) Speed(m/s) 1.5 1.5

1 1

Slight Slight 0.5 0.5 Standstill Slow Normal Fast Standstill Slow Normal Fast 0 0 0 2 4 6 8 10 12 14 16 18 20 0 2 4 6 8 10 12 14 16 18 20 Time(s) Time(s)

(a) Hallway (b) Outside

Figure 8. Comparisons of the measured speed and the actual speed.

4.3. Walking Speed Enhancement In this paper, we assume that the two pedestrians’ walking direction is parallel in most cases of the indoor hallway walking situations. Therefore, the relative angle between the user’s (or the encounter’s) walking direction and the DopE direction is calculated by the triangle we built. Afterwards, we leveraged the encounter’s walking speed and the DopE speed to ﬁnally obtain the user’s speed. More detail is shown in Algorithm1. In the end, based on the DopE speed ∆v, the DopE angle θ, and the encounter’s walking speed vencounter, we built a mathematical model for computing the user’s walking speed, which is shown in Equation (5). p ∆v − v × 1 − sin θ2 v = encounter . (5) user cos θ

Algorithm 1: Walking speed enhancement. Input: hallway width L acoustic speed c Output: User walking speed vuser 1 Emit fe kHz sub-ultrasonic; 2 Receive fr kHz sub-ultrasonic; fr− fe 3 ∆v = c × ; fe 4 D ← sub-ultrasonic TDOA method; L 5 sin θ = D ; 6 The velocity component of vencounter on DopE direction; 0 p 2 7 vencounter = vencounter × 1 − sin θ ; 8 The velocity component of vuser on DopE direction; 0 0 9 vuser = ∆v − vencounter; 10 return vuser; Sensors 2019, 19, 2458 11 of 16

The proposed method is presented in this section. In next section, we implement a few more experiments to demonstrate that iIPT can largely improve the tracking accuracy and decrease the average error compared with a conventional IMU-based PDR method.

5. Performance Evaluation In order to demonstrate the feasibility and accuracy of iIPT, we implemented several experiments in a teaching building on campus. By using certain commercial smartphones, we veriﬁed the Doppler effect’s accuracy of a pedestrian’s moving detection and compared the iIPT performance with that of a conventional IMU-based PDR method. At last, we illustrate our research on battery consumption.

5.1. Indoor Environment Description Smartphone. In terms of our experiment result in Section4, commercial smartphones have a similar ability to emit and receive sub-ultrasonic. Taking our own situation into account, we utilized an iPhone 7 to implement our experiment and leveraged pulse code modulation (PCM) ﬁles to store and analyze the sub-ultrasonic data. Besides, the format of the PCM is a mono-channel, little endian. The sampling frequency is 96,000 Hz, and the quantization accuracy is 16 bit. Hallway environment. The experiment site is a hallway in our college building that is 2 m wide, which is shown in Figure 9(b). Because the temperature is up to 31 ◦C, we adopted 349 m/s as the sub-ultrasonic velocity. We measured the pedestrian and the robot car in the same hallway environment.

(a) Smartphone (b) Hallway

Figure 9. Indoor experiment environment. Sensors 2019, 19, 2458 12 of 16

Robot car. We ﬁxed the mobile phone on this robot car, which is shown in Figure 9(c). The robot car is controlled by Arduino and has high speed accuracy. Outside environment. The outside environment is on the road, which is shown in Figure 9(d).

5.2. Tracking Accuracy To verify iIPT’s capability of improving pedestrian tracking accuracy when the pedestrian walks arbitrarily, we implemented a set of 6 m pedestrian encountering plots, utilized iIPT to estimate the walking distance, and compared the tracking result with IMU-based PDR methods. Four different kinds of PDR methods were selected as the contract: 1. the Stastic method; 2. the Kim method; 3. the Weiberg method; 4. the Scarlet method [31]. In general, the IMU-based PDR experiment will take a traveled distance of over 10 m. However, as shown in Figure6, the ultrasonic Doppler effect’s performance is reliable only within 6 m because of the characteristic of its fast energy attenuation. Therefore, to extend the traveled distance of ilPT to make it comparable to the PDR method, the number of experiments was increased, and we accumulated errors that can be regarded as one long distance experiment. We compared the total average error of our tracking method with these four PDR methods mentioned above. In order to eliminate the accidental error, every group of experiment for different methods was constructed 20 times. The average experiment result before subtracting the systematic error is shown in Figure 10(a). It is obvious that the accuracy of the static PDR method is not good enough to satisfy the short distance tracking situation. Besides, the accumulative distance bias of the Kim, Weiberg, and Scarlet PDR methods is 0.75 m at 10 m and increases up to over 1.25 and 1.9 m at 20 and 30 m, respectively, while iIPT maintains a stable bias around 0.5 m every 10 m. In addition,it is inevitable that the experimental data always contain a systematic error due to various factors. Therefore, we supplement an experiment that removed the systematic error. In the course of our experiment, we found that the frequency of ultrasonic smartphones received always has a 0.5 Hz ﬁxed deviation, possibly caused by the hardware. Similarly, the PDR methods have a different systematic error with a certain traveled distance. The experiment result after subtracting the systematic error is shown as Figure 10(b). The iIPT obviously provides a smaller error compared to the four PDR methods. The average error can be controlled within 2%.

2.5 4

Static PDR Method Static PDR Method Kim PDR Method Kim PDR Method 3.5 Weiberg PDR Method Weiberg PDR Method 2 Scarlet PDR Method Scarlet PDR Method ilPT 3 ilPT

2.5 1.5

2 Error(m) Error(m) 1 1.5

1 0.5

0.5

0 0 10 20 30 10 20 30 Travelled Distance(m) Travelled Distance(m) (a) Original data (b) After error treatment

Figure 10. Comparison of tracking accuracy between PDR and iIPT.

In addition, we classiﬁed the pedestrian speed reliability ρ into three levels: low (0 <= ρ < 0.005), mid (0.005 <= ρ < 0.01), and high (0.01 <= ρ). Tracking distance bias changes with different levels, and the comparison between iIPT and PDR is shown in Figure 11. We can see from the experiment result that the average distance bias of PDR increases severely with a higher ρ. On the contrary, iIPT is Sensors 2019, 19, 2458 13 of 16 less sensitive to ρ, although the bias still slightly increases. In a word, iIPT performs better than PDR at tracking a pedestrian with arbitrary walking.

2 PDR 1.8 iIPT 1.6

1.4

1.2

0.8

0.6 Average distance bias(m) 0.4

0.2

0 low mid high Pedestrian speed reliability level

Figure 11. Comparison of tracking distance bias between PDR and iIPT under different pedestrian speed reliability levels.

5.3. Battery Consumption For iIPT, the smartphone’s battery consumption is mainly determined by a sub-ultrasonic emitting volume level, which has a vital inﬂuence on sub-ultrasonic transmission distance. As depicted in Figure 12, it is obvious that the battery consumption increases with a higher volume level. From level 1 to level 4, battery consumption increases by 50%, and as the level increases, it increases smoothly up to level 12. From level 12 to level 16 (max volume), the battery consumption increases severely again. Moreover, taking transmission distance into account at the same time, we consider a volume around level 12 to be suitable for the sub-ultrasonic Doppler effect in an indoor environment.

30 Current(mA) 20

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 Volume Level Figure 12. Smartphone’s battery consumption when emitting 19.5 kHz sub-ultrasonic with different volume levels. Sensors 2019, 19, 2458 14 of 16

According to the experiment result, iIPT performs better than the IMU-based tracking method under arbitrary walking circumstance. However, there are still several unsettled problems, and we have a brief discussion of those problems in Section6.

6. Discussion One-on-N encounter model. In our scenario, we only consider a one-on-one pedestrian encounter model so that there is only one user and one encounter. However, in actual situations, there will be multiple encounters around the user, increasing the complexity of the Doppler effect analysis and the amount of ultrasonic reflection. A one-on-N pedestrian encounter model is essential for the multiple encounter scenario, but how to select proper encounters to build the local model becomes a huge problem. Moreover, the one-on-N model causes serious multi-path problems and increases the complexity of analyzing ultrasonic signals due to additional noise. Relative angle calculation. In this paper, we set our indoor background as a hallway, and the relative angle between the user (or the encounter) and the DopE direction was obtained by calculating a triangle, which was based on the assumption that the width of the hallway can be known ahead of time. In addition, the relative angle between the walking direction of the user and the encounter becomes larger when the two pedestrians are walking more closely to each other. Due to the complex indoor environment, how to extend iIPT to deal with more indoor situations is another key problem. Encounter benchmark selection. We consider that the variance of acceleration sensed by smartphones becomes higher when pedestrians walk arbitrarily, which represents the reliability of the pedestrian walking speed from IMUs. Therefore, we chose an encounter with lower variance to modify the user’s speed. However, it is still difficult to find metrics for measuring pedestrian speed reliability, which should be defined more specifically so that iIPT will be able to deal with different indoor pedestrian encounter situations.

7. Conclusions In this paper, we propose an infrastructure-free pedestrian tracking approach by combining smartphone-based sub-ultrasonic and IMU techniques. A metric called pedestrian speed reliability is presented to characterize the reliability of the pedestrian speed provided by smartphone IMU. We adjust a relatively less reliable pedestrian speed to a more reliable speed of a passing by ”enhancer“ based on the sub-ultrasonic Doppler effect, thus changing encountered pedestrians from an ”obstacle“ into an ”enhancer“. Extensive real-world experiments in indoor scenarios have been conducted to verify the feasibility of realizing the acoustic Doppler effect between smartphones and to identify the applicable sub-ultrasonic frequency range and transmission distance while reducing battery consumption. The experiment results demonstrate that iIPT can largely improve the tracking accuracy and decrease the average error compared with a conventional IMU-based method. For further research, we plan to extend the encounter model to multiple situations and use crowd sensing technology to solve the encounter selection problem.

Author Contributions: C.L. and S.J contributed equally to this work and should be regarded as co-first authors. C.L. and S.J. designed the enhancement algorithm named iIPT. C.L., S.J, S.Z and Z.G contributed to the interpretation of the results and writing of the manuscript. All authors read and approved the final manuscript. Funding: This work was supported by the National Key R&D Program 2016YFC1401900, the China Postdoctoral Science Foundation 2017M620293, the Fundamental Research Funds for the Central Universities 201713016, Qingdao National Labor for Marine Science and Technology Open Research Project QNLM2016ORP0405, and the Natural Science Foundation of Shandong No. ZR2018BF006. Conflicts of Interest: The authors declare no conflict of interest. Sensors 2019, 19, 2458 15 of 16

References

1. Hossain, A.K.M.M.; Soh, W.S. A survey of calibration-free indoor positioning systems. Comput. Commun. 2015, 66, 1–13. [CrossRef] 2. Chawathe, S.S. Beacon Placement for Indoor Localization Using Bluetooth. In Proceedings of the ITSC 2008: 11th International IEEE Conference on Intelligent Transportation Systems, Beijing, China, 12–15 October 2008. 3. Bruno, R.; Delmastro, F. Design and Analysis of a Bluetooth-Based Indoor Localization System. In Proceedings of the Personal Wireless Communications: Ifip-tc6 8th International Conference, PWC, Venice, Italy, 23–25 September 2003. 4. Zhang, W.; Kavehrad, M. A 2-D indoor localization system based on visible light LED. In Proceedings of the 2012 IEEE Photonics Society Summer Topical Meeting Series, Seattle, WA, USA, 9–11 July 2012. 5. Jimenez, A.R.; Zampella, F.; Seco, F. Light-matching: A new signal of opportunity for pedestrian indoor navigation. In Proceedings of the IPIN 2014: 5th International Conference on Indoor Positioning and Indoor Navigation, Busan, Korea, 27–30 October 2014. 6. Yin, Z.; Wu, C.; Yang, Z.; Liu, Y. Peer-to-Peer Indoor Navigation Using Smartphones. IEEE J. Sel. Areas Commun. 2017, 35, 1141–1153. [CrossRef] 7. Xiang, L.; Tai, T.Y.; Li, B.; Li, B. Tack: Learning Towards Contextual and Ephemeral Indoor Localization with Crowdsourcing. IEEE J. Sel. Areas Commun. 2017, 35, 863–879. [CrossRef] 8. Zhao, Z.; Wang, J.; Zhao, X.; Peng, C.; Guo, Q.; Wu, B. NaviLight: Indoor localization and navigation under arbitrary lights. In Proceedings of the INFOCOM 2017-IEEE Conference on Computer Communications, Atlanta, GA, USA, 1–4 May 2017. 9. Yang, Z.; Wu, C.; Zhou, Z.; Zhang, X.; Wang, X.; Liu, Y. Mobility Increases Localizability: A Survey on Wireless Indoor Localization Using Inertial Sensors. Acm Comput. Surv. 2015, 47, 1–34. [CrossRef] 10. Rai, A.; Chintalapudi, K.K.; Padmanabhan, V.N.; Sen, R. Zee: Zero-effort crowdsourcing for indoor localization. In Proceedings of the IEEE International Conference on Mobile Computing and Networking, Istanbul, Turkey, 22–26 August 2012; pp. 293–304. 11. Siirtola, P.; Röning, J. User-Independent Human Activity Recognition Using a Mobile Phone: Offline Recognition vs. Real-Time on Device Recognition. In Distributed Computing and Artificial Intelligence; Springer: Berlin/Heidelberg, Germany, 2012; Volume 151, pp. 617–627. 12. Huang, W.; Xiong, Y.; Li, X.; Lin, H.; Mao, X.; Yang, P. Shake and Walk: Acoustic Direction Finding and Fine-grained Indoor Localization Using Smartphones. In Proceedings of the INFOCOM 2014-IEEE Conference on Computer Communications, Toronto, ON, Canada, 27 April–2 May 2014. 13. Ward, A.; Jones, A.; Hopper, A. A new location technique for the active office. IEEE Pers. Commun. 1997, 4, 42–47. [CrossRef] 14. Priyantha, N.B.; Chakraborty, A.; Balakrishnan, H. The Cricket location-support system. In Proceedings of the International Conference on Mobile Computing and Networking, Boston, MA, USA, 6–11 August 2000; pp. 32–43. 15. Wang, W.; Liu, A.X.; Sun, K. Device-free gesture tracking using acoustic signals. In Proceedings of the International Conference on Mobile Computing and Networking, New York City, NY, USA, 3–7 October 2016; pp. 82–94. 16. Aguilera, T.; Paredes, J.A.; Alvarez, F.J.; Suarez, J.I.; Hernandez, A. Acoustic local positioning system using an iOS device. In Proceedings of the IPIN 2014: 5th International Conference on Indoor Positioning and Indoor Navigation, Busan, Korea, 27–30 October 2014. 17. Zhang, H.; Du, W.; Zhou, P.; Li, M.; Mohapatra, P. DopEnc: Acoustic-based encounter profiling using smartphones. In Proceedings of the International Conference on Mobile Computing and Networking, New York, NY, USA, 3–7 October 2016; pp. 294–307. 18. Helbing, D.; Molnar, P. Social force model for pedestrian dynamics. Phys. Rev. E: Stat. Nonlinear Soft Matter Phys. 1998, 51, 4282. [CrossRef] 19. Tang, B.; Jiang, C.; He, H.; Guo, Y. Probabilistic human mobility model in indoor environment. In Proceedings of the International Joint Conference on Neural Networks, Vancouver, BC, Canada, 24–29 July 2016. 20. Jimenez, A.R.; Seco, F.; Prieto, C.; Guevara, J. A Comparison of Pedestrian Dead-Reckoning Algorithms Using a Low-Cost MEMS IMU. In Proceedings of the IEEE International Symposium on Intelligent Signal Processing, Budapest, Hungary, 26–28 August 2009; pp. 37–42. Sensors 2019, 19, 2458 16 of 16

21. Hong, F.; Chu, H.; Wang, L.; Feng, Y.; Guo, Z. Pocket Mattering: Indoor Pedestrian Tracking with Commercial Smartphone. In Proceedings of the International Conference on Indoor Positioning and Indoor Navigation, Sydney, Australia, 13–15 November 2012. 22. Jiménez, A.R.; Seco, F.; Zampella, F.; Prieto, J.C.; Guevara, J. Improved heuristic drift elimination with magnetically-aided dominant directions (MiHDE) for pedestrian navigation in complex buildings. J. Locat. Based Serv. 2012, 6, 186–210. [CrossRef] 23. Ruiz, A.R.J.; Granja, F.S.; Honorato, J.C.P.; Rosas, J.I.G. Accurate Pedestrian Indoor Navigation by Tightly Coupling Foot-Mounted IMU and RFID Measurements. IEEE Trans. Instrum. Meas. 2011, 61, 178–189. [CrossRef] 24. Filonenko, V.; Cullen, C.; Carswell, J. Investigating ultrasonic positioning on mobile phones. In Proceedings of the IPIN 2010: 2010 International Conference on Indoor Positioning and Indoor Navigation, Zurich, Switzerland, 15–17 September 2010; pp. 1–8. 25. Peng, C.; Shen, G.; Zhang, Y.; Li, Y.; Tan, K. BeepBeep: A high accuracy acoustic ranging system using COTS mobile devices. In Proceedings of the International Conference on Embedded Networked Sensor Systems, SENSYS 2007, Sydney, Australia, 6–9 November 2007; pp. 1–14. 26. Saad, M.M.; Bleakley, C.J.; Ballal, T.; Dobson, S. High-Accuracy Reference-Free Ultrasonic Location Estimation. IEEE Trans. Instrum. Meas. 2012, 61, 1561–1570. [CrossRef] 27. Sanchez, A.; Castro, A.D.; Elvira, S.; Glez-De-Rivera, G.; Garrido, J. Autonomous indoor ultrasonic positioning system based on a low-cost conditioning circuit. Measurement 2012, 45, 276–283. [CrossRef] 28. Urena, J.; Gualda, D.; Hernandez, A.; Garcia, E. Ultrasonic local positioning system for mobile robot navigation: From low to high level processing. In Proceedings of the IEEE International Conference on Industrial Technology, Seville, Spain, 17–19 March 2015; pp. 3440–3445. 29. Gualda, D.; Ureña, J.; García, J.C.; Lindo, A. Locally-referenced ultrasonic-LPS for localization and navigation. Sensors 2014, 14, 21750–21769. [CrossRef][PubMed] 30. Mao, W.; He, J.; Qiu, L. CAT: High-precision acoustic motion tracking. In Proceedings of the International Conference on Mobile Computing and Networking, New York, NY, USA, 3–7 October 2016; pp. 69–81. 31. Pratama, A.R.; Widyawan; Hidayat, R. Smartphone-based Pedestrian Dead Reckoning as an Indoor Positioning System. In Proceedings of the International Conference on System Engineering and Technology, Bandung, Indonesia, 11–12 September 2012.

© 2019 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).