ORE Open Research Exeter
TITLE Smartphone placement within vehicles
AUTHORS Wahlstrom, J; Skog, I; Handel, P; et al.
JOURNAL IEEE Transactions on Intelligent Transportation Systems
DEPOSITED IN ORE 22 July 2020
This version available at
http://hdl.handle.net/10871/122083
COPYRIGHT AND REUSE
Open Research Exeter makes this work available in accordance with publisher policies.
A NOTE ON VERSIONS
The version presented here may differ from the published version. If citing, you are advised to consult the published version for pagination, volume/issue and date of publication 1
Smartphone Placement within Vehicles Johan Wahlstrom,¨ Isaac Skog, Peter Handel,¨ Bill Bradley, Samuel Madden, and Hari Balakrishnan
Abstract—Smartphone-based driver monitoring is quickly a) gaining ground as a feasible alternative to competing in-vehicle and aftermarket solutions. Today, the main challenges for data analysts studying smartphone-based driving data stem from the mobility of the smartphone. In this study, we use kernel-based k-means clustering to infer the placement of smartphones within vehicles. All in all, trip segments are mapped into fifteen different placement clusters. As part of the presented framework, we b) discuss practical considerations concerning e.g., trip segmenta- tion, cluster initialization, and parameter selection. The proposed method is evaluated on more than 10 000 kilometers of driving data collected from approximately 200 drivers. To validate the in- c) terpretation of the clusters, we compare the data associated with different clusters and relate the results to real world knowledge of driving behavior. The clusters associated with the label “Held by hand” are shown to display high gyroscope variances, low Fig. 1. We consider the problem of inferring the smartphone’s placement in maximum speeds, low correlations between the measurements a vehicle, i.e., whether it is placed a) in a car mount; b) on the passenger from smartphone-embedded and vehicle-fixed accelerometers, seat; c) on the center console; d) in the driver’s pants (not illustrated); or e) and short segment durations. held by hand (not illustrated). Index Terms—Telematics, inertial sensors, smartphones, kernel-based k-means clustering. Over the same period, the market share of smartphone-based solutions within insurance telematics has increased from 2% to 29%. With four out of five millennials positive to sharing I. INTRODUCTION their recent driving data, the industry can expect further growth One of the primary challenges of the intelligent transporta- in the near future [10]. There are today several companies, tion systems (ITS) society lies in modifying current implemen- such as Cambridge Mobile Telematics, TrueMotion, and The tations so that they can be based on the sensing, computation, Floow, whose primary focus is to provide smartphone-based and connectivity capabilities of mobile devices such as smart- telematics solutions to insurance companies around the world. phones. This field within ITS is referred to as smartphone- Smartphones may also be used for road-condition monitoring based vehicle telematics, and emerged after the birth of the [11], real-time ridesharing [12], vehicular ad hoc networks modern smartphone, about a decade ago [1]. Smartphones (VANETs) [13], and value-added services such as vehicle- enable end-user centric and vehicle-independent telematics finder services and fuel and routing optimization. solutions that complement vehicle-centric factory-installed Despite the many advantages of smartphone solutions, telematics systems. Generally, smartphone-based implementa- smartphone-based driver behavior profiling is generally chal- tions benefit from seamless software updates, intuitive user lenging and does in many situations not reach the same interfaces, short development cycles, and low development performance as competing data collection methods. Due to costs [2]–[7]. One example application where smartphones the mobility of the smartphone, the data processing is funda- have sparked a major industry disruption is insurance telem- mentally different from traditional sensor fusion using vehicle- atics. In insurance telematics, driving data is used to adjust fixed sensors. Recent studies have for example focused on automotive insurance premiums and provide various value- how to classify data from smartphone-embedded sensors into added services to policyholders. By utilizing smartphone- classes associated with different transportation modes [14]– embedded sensors rather than e.g., in-vehicle sensors, many [16], as well as how to estimate the orientation [17] and insurers can both lower maintenance costs and increase cus- the position [18], [19] of the smartphone with respect to the tomer engagement [8]. As of March 2018, there were sixty- vehicle frame. Further, the utility and characteristics of vehicle nine active mobile-based insurance telematics programs [9]. data recorded from smartphones can be expected to depend on This represents an increase of more than 100 % in two years. whether the smartphone is rigidly mounted on a cradle, placed on the passenger seat, held by hand, etc. J. Wahlstrom¨ is with the Dept. of Computer Science, University of Oxford, Despite an increasing interest in smartphone-based driver Oxford, UK (e-mail: [email protected]). behavior profiling, there is still a big gap between academia I. Skog is with the Dept. of Electrical Engineering, Linkoping¨ University, Linkoping,¨ Sweden (e-mail: [email protected]). and industry: While most academic studies assume that the P. Handel¨ is with the Dept. of Information Science and Engineering, KTH smartphone is fixed to the vehicle, anyone who studies large- Royal Institute of Technology, Stockholm, Sweden (e-mail: [email protected]). scale industry data collected from smartphone-embedded and B. Bradley, S. Madden, and H. Balakrishnan are with Cambridge Mobile Telematics, Cambridge, MA 02142 (email: {wfbradley, sam, orientation-dependent sensors such as accelerometers, gyro- hari}@cmtelematics.com). scopes, and magnetometers, will inevitably have to consider 2 the effects of e.g., users picking up their smartphones during Illustration of segmentation trips. As illustrated in Fig. 1, this article attempts to bridge c this gap by examining the problem of unsupervised inference 3 Segment 1 c2 Segment 2 Segment 3 on the smartphone’s conceptual placement1 within a vehicle. States c Knowledge of the smartphone’s placement is important for 1 several reasons. For instance, it can be used to assess the | reliability of collected data (inertial measurements collected t¯ t¯ from smartphones held by hand are e.g., less suitable for the detection of harsh acceleration events since hand motions ω¯ may interfere with the measurements). Hence, detected harsh Angular velocity braking events could be considered more or less reliable | Time depending on during which smartphone placement they were detected. Moreover, it may be used for accident reconstruc- Fig. 2. Illustration of the trip segmentation described in Section II-A based on typical gyroscope measurements. The true placements over the considered tions (to answer e.g., “Was the driver interacting with his trip are assumed to be c2, c3, and c1. smartphone before the accident?”) and assessments of driver distraction (drivers interacting with their mobile phone are at measurements exceeded ω¯. Likewise, these periods were con- a significantly increased risk of being involved in an accident sidered to end a time period t¯ after the last sampling instance [20]). Recently, there has been several efforts to reduce driver at which the gyroscope measurements exceeded the same distraction by tracking the amount of time that drivers spend threshold. All data within these periods was then discarded, interacting with their smartphone using smartphone-embedded and the data before and after was used to form two new sensors and then use this to encourage more attentative driving independent trip segments. Segments that were shorter than [21]. t¯, or where the speed did not exceed s¯, were also discarded. The following approach is taken in this article: Trip seg- The segmentation algorithm is illustrated in Fig. 2. ments are mapped into fifteen different states using kernel- based k-means clustering. The features are computed using B. Segmentation Parameter Selection data from smartphone sensors and a vehicle-fixed sensor tag The threshold of ω¯ = 100 [ /s] was chosen so that it will equipped with an accelerometer triad. Within the inference ◦ typically not be reached during normal vehicle maneuvers2 framework we make use of classical mechanics, established (when the smartphone is fixed to the vehicle), but so that it methods for inertial measurement unit (IMU) alignment, will be reached in most of the cases when the smartphone knowledge of the relation between the smartphone-to-vehicle is, by hand, moved from one place to another within the orientation and the smartphone’s placement in the vehicle, and vehicle. Further, the lower limits on the duration and speed of behavioral statistics published by the National Highway Traffic a segment of interest were set to t¯= 10 [s] and s¯ = 10 [km/h], Safety Administration (NHTSA). The results are interpreted by respectively. studying the feature distributions of the different clusters and relating this to typical driver behavior characteristics. Last, we demonstrate the effect that the smartphone’s placement has C. Accelerometer-based Segmentation on the efficiency of accelerometer-based detection of harsh For the trips where gyroscope measurements were un- accelerations. available (some older smartphones are not equipped with gyroscopes) we instead used smartphone-based accelerometer II. TRIP SEGMENTATION measurements to estimate the angular velocity. First, the Assume that the smartphone at each time instant can be accelerometer measurements were low-pass filtered using a said to be in one and only one of a discrete number of Butterworth filter of order five and with a cutoff frequency states describing its conceptual placement. Instead of jointly of 2 [Hz] (to suppress noise and make the number of trip estimating the smartphone placement at all time instants in a segments similar to that obtained with gyroscope measure- given trip, we will first attempt to identify the time points at ments). Second, instantaneous estimates of the smartphone’s which the state can change (such as when the driver pulls roll and pitch angles were computed by assuming that the out the smartphone from his pocket and mounts it on the filtered signals only reflect the accelerometer measurements 3 dashboard). Assuming that all true state changes have been due to gravity [23]. Obviously, this approach also relies on detected, all driving segments stretching from one detected 2One way to assess the range of angular velocities that are encountered in state change to the next can then be clustered by means of everyday driving is to consider the lateral acceleration of a vehicle driving in standard clustering methods. an horizontal circle with constant speed. Thus, assuming a minimum turning radius of 5 [m] and a coefficient of friction of 1, it follows that the vehicle cannot drive with an angular velocity higher than p9.8/5 · 180/π [◦/s] ≈ A. Gyroscope-based Segmentation 80 [◦/s] without losing its grip of the road surface. Here, we use that the no- sliding condition under the stated assumptions is ω2r/g < µ, where ω, r, g, Periods at which the smartphone state can change were and µ denote the vehicle’s angular velocity, the turning radius, the gravity considered to start when the Euclidean norm of the gyroscope force, and the coefficient of friction, respectively [22]. 3The smartphone frame was defined to have its three coordinate axes 1Note that we talk of placement in a more abstract sense than simply pointing to the right (as seen from a user facing the display), pointing upwards referring to the smartphone’s position with respect to the vehicle. along the display, and in the direction of the display. 3 the assumption that eventual sensor biases can be neglected. roll Third, we used the estimated roll and pitch angles to compute ∆ | pitch the matrix δCk = Ck+1Ck at each sampling instance k. Here, Ck is the rotation matrix corresponding to the roll and pitch estimates at sampling instance k and a zero yaw angle (the yaw angle can be estimated if magnetometer measurements are available). Under the small angle approximation, δC f , k · s where fs is the sampling rate, is equal to a skew symmetric matrix from which estimates of the smartphone’s angular yaw velocity can be extracted [23]. Once these estimates have been extracted, the segmentation can be performed in the same way as when gyroscope measurements are available. Fig. 3. Illustration of smartphone and vehicle when the roll, pitch, and yaw angles of the smartphone-to-vehicle orientation are zero. III. KERNEL-BASEDK-MEANSCLUSTERING The objective of kernel-based k-means clustering is to map with the median accelerometer measurements) that had the the N data points x1,..., xN into the k different classes { } largest variance in the accelerometer measurements, and as- k suming that this was the vehicle’s forward/backward direction ∆ X X 2 Ω1,..., Ωk = arg min h(xi) mc . (1) [27]. We then separated the vehicle’s forward and backward { } Ω1,...,Ωk k − k c=1 xi Ωc ∈ directions by assuming that the accelerometer measurements Here, ∆ P and ∆ P denote in the forward direction were positively correlated with the mc = xi Ωc h(xi)/ Ωc Ωc = xi Ωc 1 the centroid and the∈ cardinality| of| the|cth| cluster,∈ respectively, differentiated GNSS measurements of speed. As should be while h( ) is the basis function vector. Further, denotes obvious, the computations rely on the assumption that while the Euclidean· norm, and the kernel function is definedk · k as the smartphone may vibrate or be affected by minor hand movements, the smartphone-to-vehicle orientation is not sub- ∆ | k(xi, xj) = h(xi) h(xj). (2) ject to any significant changes during the course of a given Typically, one would employ iterative methods to find one segment (i.e., that the segmentation described in Section II or several local minimums to the optimization problem (1) has been successful). [24]. The k-means algorithm was chosen due to its simplicity, 2) Accelerometer variance: The accelerometer variance was computational efficiency, and familiarity to practitioners. Refer first computed separately along each of the three spatial to [25] for a discussion on regularized k-means clustering and dimensions. We then computed the sum of the three variance how it can be used to prevent overfitting. measures and took the logarithm of the resulting value to compress order-of-magnitude variations. 3) Gyroscope variance A. Features : The gyroscope variance was com- puted in the same way as the accelerometer variance. This subsection describes the features that were ex- 4) Maximum speed: The maximum speed of the vehicle tracted from the segments. The features were derived from during the segment as indicated by GNSS measurements. smartphone-based inertial and global navigation satellite sys- 5) Tag-smartphone correlation: The correlation between the tem (GNSS) measurements, information on the smartphone’s accelerometer measurements from the external accelerometer screen state, and accelerometer data from a vehicle-fixed tag and the smartphone (both rotated to the vehicle frame) sensor tag. The measurements from the tag were only used was first computed separately along each spatial dimension. to compute the fifth feature, the tag-smartphone correlation. The feature was defined as the sum of the three correlations. 1) Smartphone-to-vehicle orientation: Methods for estimat- ing the smartphone-to-vehicle orientation have previously been 6) Segment duration: The logarithm of the temporal length reviewed in [1] (see also [26]). In this study, we first computed of a segment. the median values (over the segment) of the accelerometer 7) Screen state: The percentage of time that the screen was measurements in each direction. The roll and pitch angles of on during the segment. the smartphone-to-vehicle orientation were then estimated by 8) Initial screen state: The percentage of time that the screen assuming that these median values only reflect the accelerome- was on during the first ten seconds of the segment. ter measurements due to gravity, and that the vehicle’s roll and All features except the smartphone-to-vehicle orientation pitch angles could be approximated as zero4 (the smartphone- were normalized over all segments by means of linear rescal- to-vehicle orientation is illustrated in Fig. 3) [23]. Finally, ing, and after the normalization took on values between 1 − the yaw angle was estimated by finding the direction in the and 1. If a feature could not be computed for a given segment horizontal plane (perpendicular to the direction of the vector due to missing data (for example, some vehicles did not have a sensor tag installed) the feature value was set to the 4The vehicle frame was defined as a forward-right-down frame. The Euler median value (as taken over the remaining segments) of the angles describing the smartphone-to-vehicle orientation were defined to be the angles of rotations that applied to the yaw-pitch-roll axes (in that order) same feature. In commercial deployments intended for a large rotated the vehicle frame to the smartphone frame. number of users, low-dimensional features could be computed 4
TABLE I INITIALIZATION FOR K-MEANSCLUSTERING.
State c Placement label d Description of (likely) placement φ [rad] θ [rad] ψ [rad] Initial screen state 1 1) Seat Lap or driver/passenger seat π 0 0 off 2 1) Seat Lap or driver/passenger seat π 0 π/2 off 3 1) Seat Lap or driver/passenger seat π 0 π off 4 1) Seat Lap or driver/passenger seat π 0 3π/2 off 5 2) Cup holder Cup holder 3π/2 0 0 off 6 2) Cup holder Cup holder or car mount 3π/2 0 π/2 off 7 2) Cup holder Cup holder 3π/2 0 π off 8 2) Cup holder Cup holder 3π/2 0 3π/2 off 9 2) Cup holder Cup holder or car mount 3π/2 0 π/2 on 10 3) Pants Pants pocket 0 0 π/2 off 11 4) Console Dashboard or center console π 0 π/2 on 12 5) Held by hand Held by hand (call, left ear) 3π/2 − 0 on 13 5) Held by hand Held by hand (call, right ear) 3π/2 − − π on 14 5) Held by hand Held by hand (e.g., text messaging) 5π/4 0 π/2 on Here, φ, θ, and ψ denote the roll, pitch, and yaw angles (of the smartphone-to-vehicle orientation), respectively. Further, denotes the angle 10 · π/180 [rad]. using local resources on the smartphone and then sent to the cloud for further processing. 1 2, 11 3 4
B. Kernel 5 6, 9 7 8 10 The clustering was made using the kernel