Behavioral Biometrics for Continuous Authentication in the Internet Of

This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/JIOT.2020.3004077, IEEE Internet of Things Journal IEEE INTERNET OF THINGS 1

Behavioral Biometrics for Continuous Authentication in the Internet of Things Era: An Artiﬁcial Intelligence Perspective Yunji Liang, Sagar Samtani, Bin Guo, and Zhiwen Yu

Abstract—In the Internet of things (IoT) era, user authentication is an essential to ensure the security of connected devices and the customization of passive services. However, conventional knowledge-based and physiological biometric-based authentication systems (e.g., password, face recognition, and fingerprints) are susceptible to shoulder surfing attacks, smudge attacks and heat attacks. The powerful sensing capabilities of IoT devices including smartphones, wearables, robots and autonomous vehicles enable the continuous authentication (CA) based on behavioral biometrics. Artificial Intelligence (AI) approaches hold significant promise in sifting through large volumes of heterogeneous biometrics data to offer unprecedented user authentication and user identification capabilities. In this survey paper, we outline the nature of continuous authentication in IoT applications, highlight the key behavioral signals, and summarize the extant solutions from an AI perspective. Based on our systematic and comprehensive analysis, we discuss the challenges and promising future directions to guide the next generation of AI-based continuous authentication research. Index Terms—Behavioral Biometric, Continuous Authentica- tion, Data Mining, Internet of Things, Artificial Intelligence, Con- strained Devices, Body Area Networks, Cyber-Physical Systems;

I.INTRODUCTION ITH the flourishing of IoT, our daily life is being W transformed by ambient intelligence [1] along with massive connected IoT devices ranging from smartphones and wearables to robots, autonomous vehicles and drones [2], [3]. Fig. 1. An overview of credentials for user authentication and identification The broad penetration of IoT devices in consumer market and their applications makes user authentication critically important to secure users have the appropriate right to access IoT devices [2] and to avoid the devastating damages caused by one attack occurring in the local vulnerable spots [4]. Apart from the security Due to the importance of user authentication, researchers concerns, user authentication is beneficial for passive and cus- and industries are increasingly studying the development of tomized services when the user switching occurs. For example, sophisticated methods to verify and recognize user identities. for one autonomous car shared among family members, the As shown in Fig. 1, authentication systems can be divided driving habits among family members differ significantly. To into three categories: knowledge-based, physiological biomet- assist the drivers, different assistance strategies can be applied ric based, and behavioral biometric based solutions [2], [5]. based on user identities [3]. Thus, user authentication can Knowledge-based authentication explicitly requests user to protect crucial information against potential attacks and offer enter credentials such as password, personal identification customized services for improved user experience. number (PIN) and graphical PIN to confirm the identity of an individual. Physiological biometric based authentication Y. Liang, B. Guo, and Z. Yu are with the School of Computer Sci- uses biological traits (e.g. fingerprint, iris, and facial images) ence, Northwestern Polytechnicial University, Xi’an, Shaanxi, China. E-mail: [email protected] and employs the machine learning methods to discriminate S. Samtani is with the Operations and Decision Technologies Department user identities. Behavioral biometrics including walking gait, in the Kelley School of Business, Indiana University, USA. keystroke and touchscreen dynamics are used for user authen- Copyright (c) 20xx IEEE. Personal use of this material is permitted. However, permission to use this material for any other purposes must be tication as well. Authentication systems can be classified into obtained from the IEEE by sending a request to [email protected]. two sub-categories: user authentication to detect whether the

2327-4662 (c) 2020 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information. Authorized licensed use limited to: University of South Florida. Downloaded on June 28,2020 at 15:53:57 UTC from IEEE Xplore. Restrictions apply. This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/JIOT.2020.3004077, IEEE Internet of Things Journal IEEE INTERNET OF THINGS 2

user is one unauthorized visitor or genuine user and user Obtrusiveness: Existing solutions require explicit inputs or identification to recognize whom the current user is. actions, which are obtrusive for users by requiring extra user The essence of authentication systems is to build the map- attention. They also cause a distraction from the undergoing ping relationship between users and objectives. According to tasks [20], [21]. For example, iris and facial recognition the object-user mapping relationship, authentication systems require users to stare at the camera in specific angles, which can be categorized as Fig. 2. Among them, one-to-one map- is unnatural and uncomfortable for users. ping aims to verify whether the user is the genuine user In recent years, the rapid proliferation of IoT devices such or imposter for one privately-owned device (such as mobile as smartphones, wearable devices and facility cameras has phones and laptops) or one mobile application. One-to-many made it possible to seamlessly sense and track user behaviors. mapping provides the appropriate access control among mul- The analysis and mining of behavior fingerprints offer new tiple users for one object shared within a group of persons. In opportunities for continuous authentication [22], [23]. In this IoT systems, numerous smart devices are connected to provide paper, we provide a systematic overview about the continuous pervasive services for one user (such as smart home and authentication based on behavioral biometrics from the per- vehicle-to-vehicle systems [6]). In the dynamic environment spective of AI. Our contributions in this paper are as follows: participants need to finish one session across shared IoT • We provide a systematic overview of the key components devices where complex and robust authentication schemes and differentiators between user authentication and iden- are needed [7]. The many-to-one mapping and many-to- tification; many mapping fit well for the user authentication in complex • We summarize the key elements of behavioral biometrics; dynamic environment. • We provide a summary of the emerging types of sensing technologies being integrated into emerging IoT technologies, with a specific focus on how the data they generate and common representations of these data; • We present a general framework on how future researchers can develop innovative AI-based approaches for continuous user authentication and identification. • We summarize emerging directions for future AI-based research in the aforementioned areas. The remainder of this paper is organized as follows. In Section II, we characterize the nature of behavioral biometrics. In Section III, we propose a general framework for continuous user authentication from sensing and computing perspectives. Sections IV and V provide one systematic survey about data Fig. 2. The mapping relationship between devices and users sensing and inference methods respectively. Finally, Section VI presents the open issues and challenges in CA based on Although numerous user authentication and identification behavioral biometrics and Section VII concludes this paper. methods are proposed, prior methods have several key drawbacks as it pertains to their fit with IoT applications: Vulnerability: Prior systems are prone to a diverse range II.CHARACTERIZING BEHAVIORAL BIOMETRICS of attacks. For knowledge-based authentication, imposters can Behavioral biometrics refer to the unique behavioral traits capture inputs by shoulder surfing and recording attacks [8]– that can be used for human authentication. Unlike the [10], thermal attack [11], [12] and smudge attacks [13], knowledge-based credentials and physiological biometrics [14]. For facial recognition, an adversary could conquer the shown in Fig. 1, behavioral biometrics identify people by how facial detection through legitimate users’ facial photos. The a user conducts the specified activity rather than by static fingerprint can be conquered by smudge attack [13]–[15] and information or physical characteristics. User authentication forged by deep learning methods [16]. The automated speaker based on behavioral biometrics is characterized as secure, verification based on the personal characteristics of voices is continuous, transparent, and cost effective. subject to replay attacks [17], [18]. Secure: In contrast to knowledge-based credentials and Discreteness: In general, user identification and authentica- physiological biometrics, behavioral biometrics provide a dy- tion is executed once at the beginning of a session. If the au- namic modality that is completely passive and works in the thentication information is stolen or compromised, imposters background, making it impossible to copy or steal. Behavioral can fully control the hacked accounts or IoT devices, resulting biometric data are extracted when users are performing one in devastating damages consequentially. In addition, the one- specified activities. Unlike the static authentication informa- time user authentication is insecure in some scenarios. For tion, the nature of behavioral biometric data ensures that they example, in the ride-sharing platforms, registered drivers may cannot be forgotten, exchanged, and stolen. Moreover, the subcontract the ride-assignments or share their registration dynamics of activities makes it very difficult to forge behav- to an unauthorized person, which could be dangerous for ioral biometrics. Authentication systems based on knowledge- the riders [19]. Thus, the one-time authentication method is based credentials and physiological biometrics are vulnerable insecure and cannot provide seamless protection. to a variety of cyberattacks including shoulder surfing attacks,

smudge attacks, replay attacks, thermal attacks and adversary attacks [8]–[10], [13], [14]. Systems based on behavioral biometrics are secure and robust to the aforementioned cyberattacks. Continuous: In the IoT era, user authentication is one crucial task to secure the connected devices, and it should not be a one-off event but rather a constant process. Unlike the one-time authentication that is enforced at the beginning of a session or login, continuous user authentication is an essential requirement to verify that users are who they claim to be on an ongoing basis. In order to achieve this goal, behavioral biometrics continuously profile a user’s behavior based upon the natural interactions without having to constantly interrupt users. The continuity of behavior makes it a nature way for continuous authentication with no distraction for users. Fig. 3. An abstract framework of continuous authentication and identification based on behavioral signals Unobtrusive: Unobtrusive sensing aims to monitor physical activities and behaviors continuously via sensors embedded in ambient environment or wearable sensors [24], and maximize the user experience to avoid disturbing users from the undergo- movements, touchscreen dynamics are suitable for continuous ing tasks [25]. Behavioral data can be sampled when users are authentication. interacting with IoT devices or ambient environments with no Unobtrusive sensing summarizes the available sensors and explicit input. Moreover, user authentication can be performed feasible sensing strategies to capture the behavioral signals. in a transparent and unobtrusive way with no distraction The sensing modalities for behavioral signals are diverse. In for users [26]. Previous attempts to continuously authenticate the IoT era, the unprecedented sensing capability brings the may have been too disruptive (e.g., prompts mid-session), but opportunities to sense the behavior in different granularities now by using unobtrusive sensing techniques users can be with diverse sensors. For example, walking gaits can be cap- continuously authenticated without interruption. This feature tured by facility cameras, accelerometers built-in wearables, is beneficial for the enhancement of user experience, and and WiFi signals that bounce off the walking individuals. IoT provides more secure protection for IoT devices. devices such as smartphones and wearables are infused into Cost Effective: Physiological biometrics usually rely on our daily life and can provide transparent, unobtrusive and customized hardware for information acquisition [27], [28]. continuous behavior sensing without additional attentions and This is often expensive in terms of costs and impedes the actions required. widespread adoption of physiological biometrics for user Continuous computing authentication. In contrast, behavioral biometrics can be ob- highlights the workflow for contin- served and sampled with embedded sensors in IoT devices uous authentication. The goals of CA based on behavioral detect whether the user has the right (e.g., microphone, touchscreen, acceleremeter in smart phone biometrics are to to access the IoT device or not (authentication) and wearable devices) or public facilities (e.g., WiFi access , and to recognize whom the current user is (identification) point and surveillance camera). The widespread availability . of IoT devices makes it possible to sense behaviors without Accordingly, continuous authentication based on behavioral extra hardware, which improves the acceptance of behavioral biometrics can be divided into two categories: anomaly de- biometrics with low cost and ease of use. tection and classification. Anomaly detection methods can determine the abnormal patterns from the regular ones. For user identification, one predictive model is trained to maximize III.OVERVIEW OF CONTINUOUS AUTHENTICATION the inter-class differences (i.e., legitimate users vs. outliers). To guide the readers to understand the core concepts in Applications are typical scenarios where continuous authen- this survey paper, we provide an overview of continuous tication based on behavioral biometrics are applied. According authentication systems to illustrate what components should to the property of devices, user authentication can be applied in be included in one behavior-based continuous authentication three categories of IoT devices or scenarios including private system. Specifically, we present an abstract framework of devices, shared devices, and open application environments. continuous authentication and identification based on behav- Apart from IoT device security issues for access control, ioral signals to highlight the primary components for user behavioral biometrics can be applied for customized services authentication. As shown in Fig. 3, it consists of four layers: in smart space. For example, for one autonomous car shared behavioral signals, unobtrusive sensing, continuous computing among family members, insurance companies can design tai- and applications. lored insurance policies according to individual’s driving pat- Behavioral signals are the collection of distinctive behav- terns, where the continuous user identification based on driving ioral patterns or traits that can be used by one decision-making patterns is the premise. For smart spaces, understanding the system to decide individual’s identity. A large number of pilot presence of users in the buildings is significantly important for studies show that dynamics of keystroke, walking gaits, eye providing more responsive and customized services [29].

IV. SENSINGOF BEHAVIORAL BIOMETRICS C. Eye Movement Numerous behavioral traits have been explored for continu- Driven by the internal interaction relationship between ous authentication. In this section, we analyze the commonly muscles and brain neural, eye movements including gaze and used behavioral traits for use authentication, and conduct com- blinking are significantly different for individuals and are prehensive comparison from different dimensions including difficult to be mimicked and duplicated. Authentication based vulnerability, discretenss, obtrusiveness, and privacy. on eye movements can be divided into two categories in terms of data signals. Bioelectrical signals caused by eye movements and blinks are studied, and found that the accompanying A. Keystroke Dynamics electro-oculogram signals extracted from eye blinking were Keystroke dynamics characterize the typing rhythm such unique and rational as the biometrics for identification recog- as keystroke length, distance between consecutive strokes, the nition tasks [36]. The dynamics of eye movements including pressure exerted on each key when the individual types char- pupillary response to stimuli, pupil size, velocity, acceleration, acters, and others. To date, keystroke-powered authentication and spatial/geometric features are recorded and analyzed from has been broadly explored for devices equipped with physical video. Several studies demonstrated those patterns were intrin- keyboards. With the emergence of touchscreens, when users sic and could be applied for user identification [37]–[39]. enter characters via touchscreens, subtle changes of built-in However, eye-movement based solutions often use expen- sensors including accelerometer and gyroscope occur. Jointly sive and specialized monitor-mounted gaze trackers. Such combined with the status of built-in sensors, the keystroke explicit authentication methods may cause vigilance of the timing, touch-typing, and keystroke pressure are distinctive imposter and cover the camera of the tracker. These au- features for user identification [30], [31]. Similarly, mouse thentication methods also have high energy consumption for usage dynamics have also been shown to serve as potential the continuous video recording and realtime video analysis. authentication cues [32], [33]. Furthermore, eye-movement based user identification can be The advantages of analyzing keystroke dynamics include the obtrusive to an individual’s privacy. unobtrusive data collection and continuous monitoring of typing behaviors when users interact with devices simultaneously. D. Walking Gait However, the keystroke dynamics vary in different scenarios such as walking, holding at hand, and putting on table [34]. As a result, keystroke-based user authentication is scenario- dependent, which requires the understanding of user scenarios and build the appropriate algorithms accordingly.

B. Touchscreen Dynamics With the prevalence of touchscreen in IoT devices, sophisticated interaction patterns including pressure intensity and sliding dynamics when users interact with touchscreen enable the detection of user identification in an unobtrusive way [35]. One type of studies combine the touch patterns with conventional authentication method such as PIN codes or shaped-based drawing when individuals are running the log-in session. Even though more patterns are extracted to protect the devices against potential attacks, the authentication method is still static. For continuous authentication, Sitova´ et Fig. 4. User authentication and identification based on walking gaits via different sensors including cameras, wearables, smart floor, and device-free al. analyzed the micro-movement and orientation dynamics sensing. resulting from how a user grasps, holds and taps on the smart phone and leveraged the context and touchscreen dynamics for Identifying and authenticating based on walking gaits is user authentication [23]. an emerging biometric technology which recognizes users’ The authentication based on touch operations provides one identities by analyzing walking patterns [26]. Based on the natural way to collect user interaction data. Moreover, it makes strategies of data acquisition, the sensing strategies of gait the continuous user identification possible with no extra sen- signals can be grouped as: facility cameras, floor sensors and sors and low computational load. However, touch operations wearables. Fig. 4 illustrates the interplay and relationships vary among applications. Therefore, systematically studying among the three components. application dependent touch patterns can help protect against Vision based solutions record an individual’s gait patterns unauthorized access of crucial mobile applications. This is when walking via facility cameras. Then background segmen- especially true when individuals are likely to possess more tation techniques are used to extract features from recorded than one mobile devices. When they interact with different images to verify user identities [40], [41]. However, the devices, whether the touch dynamics are identical and can be vision-based solutions are subject to environments including transferred among different devices are still open questions. illumination and camera angle [40], [42]. Furthermore, the

high computation consumption and privacy concerns make vision-based, floor-based, and WiFi based solutions achieve vision-based solutions infeasible for continuous authentication. sub-optimal performances for multi-person scenarios. For floor-sensor based solutions, dense press sensors are deployed under floor to track the pressure dynamics or acoustic E. Body Gesture patterns when walking on the floor [43]. Its advantages include high resolution in terms of performance and unobtrusiveness With the popularity of wearable sensors, body gestures for user interaction [44]. However, floor-sensor based solutions especially hand gestures have been widely studied for user are ideal for CA for two reasons. First, they often have authentication. Majority of user authentication based on body sophisticated system design and high costs. Second, they only gestures attempt to verify or recognize the user identity based work in the enclosed environment with limited users and on a specified gesture performed. Among them, Li et al. [67] do not work in the open space with low scalability. Taken found that a person’s head movement patterns are unique when together, these limitations often result in floor-based systems stimulated by music beats, and implemented the Headbanger, being difficult to deploy. an authentication system that can authenticate uses by sensing Wearable sensor based solutions rely on sensors attached head movements when listening to music beats based on the on different spots of the body (such as waist, hip, pocket) built-in accelerator in Google glasses. In addition, hand gesture to capture the accompanying signals when walking, there- and in-air-handwriting are studied for user authentication as fore enabling continuous verification of user identity [45]– well. Matsuo et al. [66] designed one authentication system [49]. However, primary studies are conducted in the labo- based on the acceleration signals during the arm sweep action. ratories with cumbersome prototype systems and expensive Lu et al. [69] proposed a multifactor user authentication customized devices. Recently, more studies focus on the framework using both the motion signal of a piece of in-air- user verification based on off-the-shelf devices (e.g., mobile handwriting and the geometry of hand skeleton captured by phones) [50]–[52]. The main advantage of using a wearable a depth camera. However, user authentication based on body accelerometer sensor for gait recognition is that it provides gestures are obtrusive as the users are required to perform unobtrusive verification without requiring user explicit actions. certain movements or actions. Especially, the accelerometer sensor has characters of small volume, low cost and can be easily integrated into the hardware F. Others of wearable devices. Chewing renders the changes of muscle tension with ac- Lately, walking gait recognition based on WiFi, millimeter companying chewing sounds. Zou et al. proposed a human wave and radio frequency identification (RFID) is gaining authentication mechanism that utilized the sounds generated attention [53]. These studies assume that the unique walk- by dental occlusion (i.e., tooth click) to unlock the mobile ing gaits and body shapes entail distinctive disturbances in devices [70]. The prototype system, BiLock, relies on the signals that can be used for user verification [54]. Wang et microphone in mobile devices to record the sounds of dental al. provided one comprehensive survey about the device-free occlusion, and verifies whether the current user is legitimate sensing based on WiFi signals [55]. Several user verification or not. Although BiLock is easy to use and requires no systems based on WiFi signals are available including WiFiID extra sensing unit, its performance is sensitive to the scenario. [56], WiWho [57], WiFiU [58], and FreeSense [59]. Among Furthermore, BiLock requires users to put the mobile device them, FreeSense [59] is an unobtrusive system for indoor 5cm to 15cm away from lips, which is obtrusive for users and human identification based on the disturbed WiFi channel does not support the transparent sensing. Similarly, Bodybeat state information (CSI) signals when individuals walk through leverages the non-speech body sounds caused by food intake, the line-of-sight (LOS) path between the source and the breath, laughter, and cough for user identification [71]. receiver of WiFi signals. FreeSense captures the disturbed In addition, breathing is used for user authentication as well waveforms when the user is walking across the LOS path, by characterizing the subtle vibration caused by respiratory. and applies the discrete wavelet transform and principal com- For example, BreathPrint employs deep-learning models to ponent analysis to extract shape features. The performance of effectively expression the acoustic features caused by breath- FreeSense declines from 94.5% to 75.5% while the number of ing for the user authentication on resource-constrained devices participants increases from two to nine. Luo et al. used RFID [72]. To provide unobtrusive continuous authentication, Liu et for gait recognition by monitoring the interruptions to RFID al. proposed a continuous user verification system based on signals when one target user is blocking the signals between unique human respiratory-biometric characteristics extracted transmitter and receivers [53]. from the off-the-shelf WiFi signals [73]. However, CA based on walking gaits has several challenges. First, prior studies were mainly conducted in a controlled environment. The robustness should be further evaluated in G. Summary of Behavioral Biometrics the physical world. Second, a person’s walking gaits can be As shown in Table I, each behavioral biometric has its pros altered by many factors, i.e, drunkenness, aging, carrying a and cons, and no single biometric is expected to effectively load, and shoe type [60]–[62]. The model trained with the meet all the needs of any scenarios and applications [74]. dataset collected in one situation may introduce bias when The advantages of most user authentication systems based it is applied into other situations [63]. Third, prior studies on behavioral biometrics include non-obtrusiveness with no mainly focus on the single-person gait detection. However, extra user attentions required and non-vulnerability against

TABLE I SUMMARY OF BEHAVIORAL SIGNALSFOR USER AUTHENTICATION

Category Signal Description Vulnerability√ Discreteness Obtrusiveness Privacy√ Keystroke Dynamics typing [30]–[33] typing characters via keyboard or × × clicking via touchscreen when entering information √ √ Touch Operation screen touch [23], finger movements and screen pressure × × [35] on touchscreen √ √ biosignal [36] bioelectronic signals caused by eye × × Eye Movement movements √ √ √ temporal [37]–[39] dynamics of eye movement √ × √ camera [40], [41] capture the walking styles from video × × floor sensor [43], features caused by users walking on × × × × [44] the pressure sensors embedded in floor √ √ wearable [45]–[49] motions patterns tracked by wearable × × Walking Gait sensors √ WiFi [55], [56] Perturbations caused by unique walk × × × styles when walking through the WiFi spectrum field √ millimeter wave [64], interrupted radar signals between × × × [65] transmit antennas and receivers √ RFID [53] Perturbations caused by unique walk × × × styles when walking through the RFID spectrum field √ √ arm swing [66] motion signals generated when a fixed × × behavior is performed Body Gesture √ head movement [67] head motions measured by head-worn × × × devices √ mouse movement Doppler profiles of acoustic signals × × × [68] caused by users’ speaking mouths √ √ √ handwriting [69] motion signals generated by × in-air-handwriting √ √ √ √ dental occlusion [70] unique acoustic patterns of teeth click √ √ Sounds of Behavior body sounds [71] acoustic patterns caused by food × × intake, breath, and laugh √ breathing [72], [73] acoustic features caused by breathing × × ×

cyberattacks. However, the existing authentication systems do not take advantage of the nature of behavioral traits to support the continuous authentication and do not consider the issue of privacy protection.

V. AI-BASEDSOLUTIONSFOR USER AUTHENTICATION In this section, we summarize the AI-based methods that are employed to recognize user identities based on behavioral fingerprints. In CA, intelligent algorithms including machine learning and deep learning are capable of determining the access control of IoT devices by checking user identities. As shown in Fig. 5, the pipeline of AI-based methods for CA consists of following major components: data pre-processing, Fig. 5. General workflow of machine learning-based user identification and feature extraction and classification algorithms. We further authentication describe each component in the following sub-sections. data [56], [57]. Similarly, BiLock uses a 6-order butterworth A. Data Pre-Processing filter to remove the out-of-band interference of dental clicks, Data pre-processing is a critical procedure to distill high and employs the wavelet de-noising to improve the signal to quality data out of the raw data that are generally incomplete, noise radio [70]. noisy, inconsistent, and redundant. As the inputs of continuous For the continuous authentication, data segmentation seeks human identification based on behavioral patterns are sequen- the effective regions from sequential data. The rule-based tial data, data filtering and data segmentation are necessary to solutions including fixed threshold or fixed-size window are reduce the noisy data and align the inputs. used over sequential data for segmentation. In WiFi-ID [56], For data filtering, numerous filters are applied to the se- two frequency bands are used to separate WiFi signals im- quential data to remove data that can be repetitive, irrelevant pacted by walking gaits from wavelet domain. The fixed-size or even sensitive [56], [70]. For example, butterworth filter is window is applied [75], [76] to segment the dynamic swipe applied to WiFi CSI data to remove the high frequency noisy behavior and video records of walking gaits into fixed-length

TABLE II SUMMARY OF WORKS IN CONTINUOUS BEHAVIORAL AUTHENTICATION AND IDENTIFICATION

Type Work Signal Features Algorithm # of Users Performance Touchstroke phone-holding behavior, statistical features from Bayesian Network 12 Fusion of hand [30] keystroke dynamics four built-in sensors, Classiﬁer movements and keystroke keystroke dynamics of dynamics can improve touch-typing timing authentication accuracy. HMOG [23] accelerometer, gyroscope, HMOG features (grasp scaled Manhattan, scaled 100 EER=7.16% for walking; and magnetometer resistance and grasp Euclidian, One-class EER=10.05% for sitting readings when a user stability features); 11 SVM tapping on the screen touchscreen features; keystroke dynamics

uthentication Omar et al. mouse dynamics, statistical features of a statistical classiﬁer 274 EER=2.11%

A [32] short-term Memory, mouse dynamics based on Weighted-Sum visual scan and detection capability BiLock [70] chewing sounds 13-order Melfrequency SVM with radial basis 100 FAR=1.1%, FRR=5.5% Cepstral coefficients function (RBF) kernel (MFCCs) from occlusion sounds Headbanger head movement with — dynamic time warping 95 Accuracy=95.57%, [67] external rhythmic stimuli (DTW) distance FAR=4.43% GlassGuard touch behavior, voice MFCCs, touchscreen One-class SVM, 32 Accuracy=93%, FAR=3% [78] features, sensor data dynamics Threshold Random Walking DeepAuth Motion Sensors frequency domain signals long short-term memory 47 Accuracy=96.7% [75] Song et al. Keystroke dynamics key hold time, key isolated forest 51 Accuracy=98% [79] latency time, key duration time Hong et al. Wave gesture based on singular value One-class SVM 8 Accuracy=92.83%; FPR [80] accelerometer decomposition (SVD) (false positive rate)= 3.67% George et al. Eye movement fixation features and Radial Basis Function 153 EER=2.59% [38] saccade features Network WiFi-ID WiFi signals of walking 7 time domain features Sparse Approximation 20 average accuracy of 93% [56] gaits and 3 frequency domain Classification to 77% from a group of 2 features to 6 people WiWho [57] WiFi signals of walking time domain and decision tree 20 average accuracy of 92% gaits frequency domain to 80% from a group of 2 features to 6 people Batchuluun thermal images of — convolutional neural 80 EER=0.77%,

Identiﬁcation et al. [76] walking gaits network (CNN) Accuracy=99.9% Mondal et swipe gesture action duration, Artiﬁcial Neural Network 41 Accuracy=95.45% al. [81] coordinate, distance, (ANN), movement variability, Counter-Propagation orientation, velocity ANN Abo-Zahhad eye blinking amplitude, position, area, linear discriminant 25 Accuracy=97.3%; et al. [36] energy, slope, duration et analysis EER=3.7% al.

slices respectively. However, rule-based solutions are sensitive deviation, and variance. Statistical features characterize the to inputs, and tightly rely on the prior knowledge. Dynamic overall patterns of the given samples from macro perspectives. segmentation strategies are introduced to split the stream data Due to the dynamics of user behaviors over time, frequency dynamically. For example, BehaveSense [77] recognizes four domain patterns of behaviors are of significance for describing touchscreen operations to separate the effective samples from the dynamics of signals. To obtain frequency domain patterns, stream data, and utilizes the sequential patterns of touchscreen sampled data are transformed by spectrum analysis to learn the operations for user identification. In addition, dynamic time frequency-domain features. As shown in Table II, DeepAuth warping (DTW) is widely used to find out the cycle of characterizes the frequency domain representation of motion behaviors [57], [59], [67]. sensors [75]. GlassGuard [78] and BiLock [70] employ the mel-frequency cepstral coefficients (MFCCs) to characterize B. Feature Extraction the dynamics of vocal signals. Feature engineering is to extract features of value that can Generally, hand-crafted feature engineering heavily relies represent users’ behavior comprehensively from the training on the knowledge of domain experts and is time consuming dataset. The extracted features depend on sensors and ap- to construct one complete feature set. As a result, it is the plications. Statistical features refer to the measurements of bottleneck of classification oriented tasks. In addition, as interpreting both quantitative and qualitative data with stan- not all hand-crafted features are preeminently contributive to dard statistics such as the root mean square, mean, standard the verification of user identity, feature selection is optional

to rebuild one subset of attributes with least data loss. To mitigate the behavioral variability of mouse dynamics, Cai et al. proposed a uniﬁed framework of employing dimensionality reduction methods to extract predominant characteristics from the original feature space for enhanced performance, and found that variability reduction in feature engineering could enhance the authentication mechanisms [82]. On the other hand, due to these drawbacks of feature engineering, many researchers in this ﬁeld have turned to deep learning based methods. Unlike conventional machine learning that relies on feature engineering, deep learning approaches utilize the complex neural network architecture to learn the representation of behaviors [76].

Fig. 6. General workflow of machine learning-based user identification and C. Anomaly-based User Authentication authentication [88] For the privately-owned IoT devices, a large amount of positive examples from the legitimate users are available, the observation that anomalies are few in number and much while negative examples from imposters are rare. Therefore, different from the rest of the data [85]. Specifically, iForest supervised classification algorithms do not fit well to train constructs an ensemble of binary search trees (named iTrees) predictive models when few negative examples are available in which anomaly points are isolated closer to the root of the [83]. To secure the privately-owned devices, anomaly detection tree. Each node in iTree has either two children or a leaf node solutions are applied to check whether the current user is one p with no child. For one p-dimension sample xi ∈ < from authorized user or one imposter. Behavioral examples from im- dataset D = {x1,x2, ...,xi, ...,xn}, one feature ai ∈ [1, p] and posters are referred as anomalies or outliers, and the behavioral its split value V 0 are randomly selected. According to feature examples from genuine users are normal. Identifying outliers Vai ,k for each input data Xk , the Vai ,k which is less than the or anomaly detection is referred as one-class classification. For V 0 is classified into left children set, the rest is classified into the user authentication task, one-class classification algorithms right children set. This process is repeated for the instances of including one-class support vector machines (SVM) [84] and left and right children nodes until (i) the incoming data set D isolated forest [85] are widely adopted for imbalanced datasets has only one record or all data in D have the same value; (ii) with severely skewed class distributions. the tree reaches the height limit l [85], [89]. One-class SVM [84] is a semi-supervised classification Isolated forest is applied for user authentication due to the algorithm, and aims to find a hyperplane to enclose the ma- following reasons. First, the feature values extracted corre- jority of positive examples from the origin with the maximum n sponding to the anomalies in original data are few and differ- margin [86]. Given the training vectors xi ∈ < , the problem ent. Second, iForest works well when handling extremely large is formulated as below [83]: data size and high-dimensional problems and in situations 1 T 1 Õ where training set does not contain any anomalies. Finally, min w w + ξi − ρ w,ξ,ρ 2 v` since iForest has linear time complexity, a fast anomaly i (1) detection on resource-constrained IoT devices is crucial to subject to wT · φ(x ) ≥ ρ − ξ , ξ ≥ 0 i i i report unauthorized access immediately. where w is the normal vector of the separating hyperplane, and ξ are slack variables. The parameter v ∈ (0,1] controls i D. Classification Algorithms for User Identification the trade-off between w and slack variables ξi. When w and ρ are solved by solving Eq. 1, the decision function User identification aims to recognize whom the current Í f (x) = sgn( i αi k(xi,x) − ρ) will be positive for majority user is, and further determine whether the current user has examples, where k(xi,x) is a kernel function [87]. In the the legitimate right to access the IoT devices or applications. context of behavior-based user authentication, the workflow of Formally, behavior-based user identification can be formulated one-class SVM is shown in Fig. 6 to highlight how the outliers as follows. Given a dataset {(x1, y1), ...,(xn, yn)}, where xi = 1 m are separated from the origin. An outlier is any data instance [xi , ..., xi ] is the m-dimensional feature vector of samples; that lies outside the support of the training data. The original yi ∈ C is the corresponding class of one specific user; C refers high-dimensional data can be projected into one feature space to the set of classes. The goal of user identification task is to via one kernel function, where the hyperplane w separates the learn a mapping function that predicts the label information training data from the origin by a maximal margin ρ/kwk (Fig. for one given behavior sequence with least biases. According 6). Data mapped to the same side of the origin will be given a to the adopted classification algorithms, the extant behavior- negative one-class SVM value, whereas those mapped to the based user identification systems can be divided into two side of the training data will have positive values [88]. categories: conventional and deep learning-based solutions. Isolated forest (iForest) [85], [89] is one unsupervised 1) Conventional classification: As shown in Table II, nu- algorithm for anomaly detection. Its main idea is based on merous hand-crafted features such as statistical features of

Fig. 7. Diagram of selected prevailing deep neural networks. (a) the architecture of Convolutional Neural Network (CNN); (b) the architecture of a Long Short Term Memory (LSTM) cell; (d) the diagram of Generative Adversarial Network (GAN).

mouse dynamics, keystroke dynamics of touch-typing timing, where each input is connected to all the neurons. For one and even frequency domain signals are proposed and a variety classification task, a softmax layer follows the final fully- of supervised classification algorithms including SVM [70], connected layer immediately to limit the output of the function random forest [64], Na¨ıve Bayes [30], and artificial neural into the range 0 to 1, which can be interpreted directly as a network [81] are employed to bridge the mapping between probability of multiple classes. feature sets and labels. However, user identification based CNN is widely adopted in user authentication and identi- on conventional classification algorithms relies on feature fication systems to detect personal patterns from fingerprints engineering, which involves computing explicit features spec- [91] and eyes [92], [93]. CNN is also commonly applied to ified by experts, resulting in algorithms designed to detect detect the liveness of biometrics against presentation attacks specific indictors. Hand-crafting feature is time consuming, [94]–[99]. For the security of smart vehicles, Xun et al. labor intensive, and not suitable for rapidly evolving domains. designed one driver fingerprinting device for the continuous Moreover, supervised learning algorithms are sensitive for the user authentication of automobiles. The driver fingerprinting training dataset. Usually, the features used in those approaches device is deployed in automobiles to collect the real-time are based on samples made from existing dataset. Due to driving data from on-board diagnostic port, and uses the the intra-user variation of behavioral biometrics [90], these CNN model to extract driver behavioral characteristics from solutions are not robust when faced with the changes of user the driving data. Finally, the extracted driving features are behaviors, and cannot adjust the parameters accordingly. fed to SVM for illegal driver detection [100]. To secure the 2) Deep Learning-based classification: To improve the smart home, Qin et al. extracted the time and frequency performance of identification systems, deep learning solutions features of sensors via CNN for each time slot, and fed are gaining popularity. Deep learning-based solutions can be the deep representation to RNN for user identification [101]. divided into three categories according to the types of neu- Batchuluun et al. [76] applied CNN to identify human identity ral networks: convolutional neural network (CNN), recurrent based on the walking gaits extracted from videos. Unlike gait neural network (RNN), and generative adversarial network recognition in the controlled environment, Zhou et al. studied (GAN). the user identification based on the waling gait in the wild, and As shown in Fig. 7(a), a typical CNN consists of convolu- proposed a hybrid model of CNN and RNN to learn robust tional layer, pooling layer, fully-connected layer, and softmax gait feature representation including space and time domains layer. In the convolutional layer, a convolution is a linear features [102]. operation that involves the multiplication of a set of weights ht = tanh(Whht−1 + Wyxt ) (also referred as filter) with the input. The convolutional layer (2) creates one feature map by applying the same filter on the yt = Wyht input repetitively to summarize the presence of a specific type RNN is a typical neural network to handle with sequence of features in the input. The pooling layer operates upon data, and the output of the prior state is forwarded as the each feature map and provides an approach to down sampling input to the current state. This process can be formulated as feature maps by summarizing the presence of features in Eq. 2, where xt is the input at time t; ht and ht−1 are the patches of the feature map. Two common pooling methods current state and previous state, respectively; Wh and Wx are are average pooling and max pooling. For the fully-connected feed-forward and recurrent weight matrices, respectively. The layer, all the neurons in this layer are connected to every output yt at time t is produced by combining the hidden state activation unit of the next layer operates on a flattened input ht with the weight matrix Wy. However, RNNs are subject to

gradient vanishing and exploding problems [103]. In addition, ical examples generated by G(z), and attempts to predict the training of RNN is time-consuming and energy intensive whether the input is real or generated. The adversary learning [104]. To address those problems, long short term memory of G(z) and D(x) can be represented mathematically as

(LSTM) is proposed. The architecture of one cell in LSTM is min max Ex∼pd at a (x)[logD(x)] + Ez∼pz (z)[log(1 − D(G(z)))] shown in Fig. 7(b). One LSTM cell consists of forget gate, [109]. GAN has been widely adopted by the adversary to input gate, and output gate. The forget layer, denoted as ft , generate high fidelity human biometrics including fingerprints controls whether the previous hide state ht−1 is forwarded to [110], facial and vocal biometrics [16], [111] to bypass the the current state ht , where σ is a sigmoid activation function. authentication systems. For example, DeepMasterPrints [110] The input gate it determines to what extent new memory is employed GAN to generate synthetic image-level fingerprints. added into the cells state, and an output gate ot regulates how In DeepMasterPrints, two generator networks are trained via gates at the next step will be affected by the previous cell state the Wasserstein GAN algorithm based on fingerprints scanned ht−1 and current input xt . with a capacitive sensor and a dataset of inked and rolled fingerprints. Experimental results show that DeepMasterPrints ft = σ(Wf ·[ht−1,xt ] + bf ) is able to generate fingerprints that can easily bypass the popular commercial fingerprint matching systems. it = σ(Wi ·[ht−1,xt ] + bi) cet = tanh(Wc ·[ht−1,xt ] + bc) (3) E. Evaluation ct = ft ∗ ct−1 + it ∗ cet For the evaluation of a verification system, the false accep- ot = σ(Wo[ht−1,xt ] + bo) tance rate (FAR) and the false rejection rate (FRR) are two h = o ∗ tanh(c ) t t t types errors. FAR refers the likelihood that an unauthorized Numerous user authentication and identification studies user is mistakenly accepted as a legitimate user; While FRR based on behavioral biometrics show that RNN and its vari- indicates the probability that a legitimate user is incorrectly ants are promising to process the sequential behavior data rejected as an imposter. Verification systems should avoid with overwhelming performance. For example, Zhang et al. those two error types. To balance the two error types in extracted discriminate features from walking gaits monitored a verification system, equal error rate (EER) is introduced by smartphones and LSTM-based model for user identification to predetermine the threshold value where FAR is equal to [105]. Due to the limitation of resource-constrained devices FRR. EER is a commonly accepted overall measure of system in memory and computation, Chauhan et al. proposed one performance. The lower the EER, the higher the accuracy of efficient authentication system based on breathing acous- the verification system. tics. They introduced model compression solutions including weight quantization and fully connected layer factorization to VI.OPPORTUNITIES AND CHALLENGES reduce the complexity of LSTM, and found that compressed The continuous authentication based behavioral traits is an LSTM outperformed other baselines with smaller model size, emerging paradigm facing several challenges. In this section, lower inference time, and more accurate [106]. Luo et al. used we enumerate the opportunities and challenges from the AI- RFID for gait recognition by monitoring the interruptions to perspective. RFID signals and introduced attention mechanism in LSTM for robust user identification [53]. Amini et al. [75] proposed a LSTM-based authentication framework that leveraged a user’s A. Evolution of Behavioral Biometrics behavior captured by motion sensors while shopping online The performance of continuous authentication based behav- to continuously re-authenticate the user, providing security ioral biometrics could be impacted by scenarios and applica- without compromising usability. DeepAuth [107] uses the tions. For instance, alcohol, mood and carrying a backpack unique motion patterns when users entering passwords as may affect the person’s gaits [112], [113]. Pupil size and behavioural biometrics, and learns the deep representation eye movement dynamics vary with individuals’ physical status of motion patterns via RNN-based model. Extensive exper- such as stress or fatigue [114]. In addition, touch patterns are iments show that DeepAuth performs well for the security sensitive with screen size and target applications. Obviously, of resource-constrained devices with in both authentication user behaviors can be impaired by many facets including performance and cost [107]. mood, health, and alcohol. How to rule out the impacts of GAN is an unsupervised algorithm to train two com- situations on user behaviors in the CA systems are crucial for petitive neural networks via a cooperative zero-sum game the robustness of behavior-based authentication systems. framework [108]. The GAN model consists of two sub- User behaviors also change over time. For example, Gal- models: a generator model to generate new examples and bally et al. analyzed the effects of age and aging on finger- a discriminator model to classify whether generated exam- prints, and found that fingerprint quality decreased linearly ples are real data or generated examples. As shown in Fig. with age for elders [115]. The touchscreen typing-patterns 7(c), the generator network G(z) takes an random input z are dynamic and impacted by health status [116]. However, with probability distribution p(z) and generates a sample of what kinds of behavioral patterns can be used as behavioral synthetical examples. The discriminator network D(x) takes biometrics for CA has not been studied yet. In addition, the input either the real examples x from pdata(x) or synthet- extant behavior-based systems are static and cannot adjust

themselves with the evolution of user behaviors. Therefore, model to reduce the computational cost by compressing hyper- fundamental studies should be conducted to evaluate the parameters [132] or multi-objective optimization based on impacts of dynamics of behavioral traits on continuous au- accuracy, latency and energy [133]. Parameter sharing [134] thentication, and propose dynamic solutions that fit well with is mainly applied in convolutional layers to reduce the size the changes of behavior evolution. of parameters by the assumption that the input going to be processed by the network is decomposable into a set of local regions with the same nature and thus each of them can be B. Sparsity of Behavioral Biometrics processed with the same set of transformations [131], [134]. The success of machine learning roots in tapping into Edge computing is shedding lights on mobile devices ori- the large-scale training dataset to learn the comprehensive ented deep learning [135], [136]. Via edge intelligence, the representation with the rapid advances of AI-based methods. complex and energy-intensive deep neural networks can be A large number of users are becoming multi-device users partitioned into tiny subtasks, and be distributively executed by interacting with more than one IoT devices [117]. The on neighboring devices or edges [137]–[139]. For example, one-to-many mapping relationship between users and IoT Kang et al. designed a light-weight scheduler to automatically devices poses great challenges for CA based on behavioral balance the computational offload between mobile devices and biometrics. In addition, the sufficient training dataset is not servers by partitioning the neural network layers [137]. Xu et readily available, and the trained models based on the skewed al. proposed DeepWear to optimize the energy consumption of distribution datasets are not robust [118]. wearables by offloading the deep learning tasks among mobile To address the sparsity of behavioral biometrics, one/zero- devices [139]. shot learning aims to build one classifier from one or only Although there are many pilot studies about deep learning a few training samples [119]. The nature of one/zero-shot on resource-constrained IoT devices, they are evaluated in learning is suitable for handling the sparsity problem of behav- laboratory environments. As a result, the performance is not ioral biometrics. Meanwhile, as user preferences of multiple generalizable to other contexts (e.g., smart home) due to the IoT devices are significantly different, the interaction records extreme heterogeneity of IoT environments [140]. In addition, collected from different IoT devices are different. For example, CA on resource-constrained IoT devices is very sensitive for the keystroke rhythm can be captured when interacting with the latency. A novel framework is essential to understand the PC; the touchscreen dynamics are collected via the mobile trade-off among accuracy, critical latency, and efficiency [141]. phone or tablets. To handle the imbalanced classification problem and smoothly conduct the user authentication on D. Emerging Malicious Attacks heterogeneous IoT devices, transfer learning [120], [121] is promising to transfer the trained model based on behavioral CA based on behavioral biometrics cannot be easily attacked records collected on one source device to the target device. by a random attacker. However, the nature of CA based on In addition, to address the sparsity of labeled data, AutoTune behavioral biometrics does not imply that the CA systems are uses the wireless identifier as a supervisory label and learns secure. the association between facial images and wireless identifier First, most authentication systems based behavioral biomet- [122]. rics are prototypes evaluated in constrained laboratory environment with limited participants. Comprehensive evaluations with a large number of participants are needed to investigate C. Deep learning on Resource-Constrained IoT Devices the performance of existing behavior-based authentication In general, IoT devices including smartphones and wear- systems when faced with the potential attacks. ables are resource-constrained with limited memory, power Second, IoT systems are faced with numerous security supply and computing capability [123]. Although deep learn- threats on physical, protocol, communication, and application ing algorithms achieved the state-of-the-art performance, deep layers [28]. For example, in the low power wireless network, learning models are becoming extremely complex with mil- the energy depletion attack can drain the batteries of devices lions of hyper-parameters [124], time-consuming training, rapidly by forcing sensors or actuators to execute energy- significant energy strains [104], [125], and do not work well intensive tasks. Consequently, the entire network could fail on resource-constrained IoT devices by debilitating levels of due to battery exhaustion [142]. For the electrical vehicles, system overhead [123]. To execute the deep learning-based batteries could be attacked (e.g., draining energy) to reduce models on resource-constrained devices, the following two driving range and increase driving range anxiety. Kang et types of solutions are proposed. al. proposed a battery authentication method based on the Model compression of deep learning has become a sig- user habits to identify users that share a vehicle [143]. In nificant problem. Methods to reduce the complexity of deep addition, the thermal attacks rely on the heat transferred neural networks include tensor decomposition [126], [127], from users to interactive devices, and exploits heat traces pruning [125], [128] and parameter sharing [129]–[131]. Ten- in the wake of user interaction with devices to uncover the sor decomposition reduces model complexity by expressing entered credentials [11]. On the communication layer, the a higher-order tensor with a sequence of linear operations on heterogeneous communication protocols are subject to attacks the matrix singular value decomposition [127]. Neural pruning such as eavesdropping, sinkhole, hello flood, collision [144], eliminates the less important connections in one pre-trained [145]. Voice assistants such as Google Assistant, Amazon

Alexa, Facebook Portal, and Apple Siri are vulnerable to signal the speech signal received by the voice assistant’s microphone. injection attacks on microphones based on the photoacoustic By fusing multi-modal data, VAuth shows robust performance effect across large distances and through glass windows 1. against potential attacks including replay attacks, mangled Third, AI is widely applied by imposters to hack the voice attacks, or impersonation attacks [155]. To against authentication and identification systems. For example, AI has the replay attacks, REVOLT exploits the spectral differences been maliciously used by the imposters to infer the password. between original and replayed voice signals, and combines the Snoopy demonstrates that it is possible to infer password breathing rate extracted from WiFi signal while speaking to entered on mobile devices by monitoring both accelerometers detect the liveness [156]. and gyroscopes [146]. Snoopy may fool the potential users as a Multi-view based solutions for user authentication mainly harmless app to continuously monitor the motion sensors when extracted fine-grained feature maps from multi-view images. users are taping the passwords, and uses bidirectional RNN For example, Li et al. employed the multi-view deep represen- for most commonly used password inference and encoder- tation learning to recognize one million celebrities from their decoder architecture with RNN models for universal password face images captured in the real world [157]. On the other interference [146]. Another prevailing example is the usage of hand, multi-view learning can be applied for user authenti- AI by the adversary to reconstruct the biometrics (i.e., replay cation to address the incompletion of obtained biometrics by attack). Deep generative models (DGMs) such as GANs and fusing multiple views [158]. variational autoencoder (VAE) have been widely adopted to In addition, contextual information can enhance the per- generate high fidelity human biometrics including fingerprints formance of user authentication. Hintze et al. introduced [110], facial and vocal biometrics [16], [111] to bypass the dynamic factors like day and time, and location together with authentication systems. multi-modal biometrics to adjust the authentication scheme accordingly [159]. [160] presents one context-based biometric E. Fusion of Behavioral Biometrics authentication model, which chooses the appropriate authentication method dynamically according to the interaction form. Since prior solutions are insufficient to effectively provide secure protection in a broad range of IoT scenarios, multifactor authentication (MFA) provides multiple layers of secu- F. Cross-device Continuous Authentication rity to protect IoT devices against potential attacks through Increasingly, a large number of users are becoming multi- the fusion of behavioral biometrics [2]. device users by interacting with multiple smart devices [117], The early form of MFA integrated multiple authentica- [161]. For instance, more than 70% of online users access tion schemes sequentially. For example, Hu et al. proposed internet across multiple devices. 90% use multiple screens a secure data backup scheme by integrating password and sequentially to accomplish a task over time2. The complex- biometrics to overcome the potential attacks [147]. Multi-view ity of multi-device, multi-user interaction presents significant representation learning for user authentication has emerged challenges for cross-device continuous authentication. Prior as a viable approach to process such data. This paradigm of studies assume the one-to-one mapping between user and de- machine learning aims to fuse multiple views (i.e., feature vice, and mainly focus on the user authentication in the single- sets) to improve the performance [148], [149]. Multi-view device scenario. However, the relationships between users and representation learning can be categorized into two groups: devices in multi-user, multi-device scenarios is many-to-many. multi-modal methods and multi-view methods. Due to the heterogeneity of devices, transferring one pre- Multi-modal solutions extract features from heterogeneous trained user identification model from source domain to target biometrics to build one classifier based on the ensembled domain is rarely studied. There are significant differences features. Kim et al. designed one multi-modal authentication in interaction modality among heterogeneous devices. For system by fusing features obtained from face, teeth and voice devices with similar interaction modalities, transfer learning modalities [150] to secure mobile devices. Crawford et al. is one promising solution [117]. In addition, the co-location utilized the keystroke dynamics and speaker verification to en- information of devices is useful for cross-device authentica- hance the authentication performance on mobile devices with tion. Hintze et al. proposed one multi-modal and cross-device a 67% reduction of explicit authentication [151]. EchoPrint authentication system based on behavioral and physiological emits nearly inaudible sound signals from the earpiece speaker biometrics (e.g., gait, voice, face, and keystroke dynamics) to to illuminate the user’s face. The extracted acoustic features reduce the manual burden of user verification according to the from the echoes are combined with visual facial landmarks context like location, time of day and nearby devices [161]. from the frontal camera to authenticate the user [152]. [153] integrates physiological biometrics, behavioral traits and on- G. Privacy Concerns line activities including search, shopping, and web browsing Behavior-based authentication systems are subject to pri- for user authentication. Kumar et al. utilizes LSTM to model vacy concerns, especially when they are adopted for per- the motion sensors and adopts CNN to extract gait patterns sonalized services. To address the privacy concerns, many from video respectively, and Gray wolf optimizer has been privacy protection solutions are proposed to secure the crucial used to optimize the parameters during fusion [154]. VAuth information in different authentication systems [162]. collects the body vibrations of the user and matches it with 2https://www.readinbrief.com/multi-device-content-consumption-statistics- 1https://lightcommands.com/ trends/

For the standalone authentication systems, although users 61902320, 71472175, 71602184, 71621002, by national sci- are able to control the standalone client, they are likely to ence foundation under grant No. CNS-1850362 and OAC- subject to exposing data to unauthorized third parties [162]. 1917117, and by the fundamental research funds for the central To secure the sensitive data in standalone systems, privacy universities under grant No.:31020180QD140. impact assessment and surveillance impact assessment should be enforced to ensure conformance with legal and regulatory REFERENCES requirements [162]. Moreover, the anonymity and encryption [1] D. J. Cook, J. C. Augusto, and V. R. Jakkula, “Ambient intelligence: are promising to protect against data exposure in privately- Technologies, applications, and opportunities,” Pervasive and Mobile owned devices. For the centralized authentication systems, Computing, vol. 5, no. 4, pp. 277–298, 2009. they may suffer from data exposure in transit and adversary [2] A. Ometov, V. Petrov, S. Bezzateev, S. Andreev, Y. Koucheryavy, and M. Gerla, “Challenges of multi-factor authentication for securing attacks in computation models by carefully crafted adversarial advanced iot applications,” IEEE Network, vol. 33, no. 2, pp. 82–88, samples [163]. Blockchain keeps the sensitive data private March 2019. such that others cannot trace and infer sensitive data stored in [3] S. Gupta, A. Buriro, and B. Crispo, “Driverauth: A risk-based multi- modal biometric-based driver authentication scheme for ride-sharing the block [164]. However, the centralized authentication sys- platforms,” Computers & Security, vol. 83, pp. 122 – 139, 2019. tems may suffer from the bottle problem due to the limitation [4] Y. Sun, B. Wang, S. Li, Z. Sun, H. M. Nguyen, and T. Q. Duong, of a single centralized server [7]. “Manipulation with domino effect for cache- and buffer-enabled social iiot: Preserving stability in tripartite graphs,” IEEE Transactions on For the distributed authentication systems, blockchain and Industrial Informatics, vol. 16, no. 8, pp. 5389–5400, 2020. federated learning [165], [166] show the great potential to [5] A. K. Jain, A. Ross, and S. Pankanti, “Biometrics: a tool for in- provide privacy-preserving authentication in collaborative ap- formation security,” IEEE Transactions on Information Forensics and Security, vol. 1, no. 2, pp. 125–143, June 2006. plications. Federated learning is a machine learning technique [6] L. Tang, Z. Duan, Y. Zhu, J. Ma, and Z. Liu, “Recommendation for that trains an algorithm across multiple decentralized edge ridesharing groups through destination prediction on trajectory data,” devices or servers holding local data samples, without ex- IEEE Transactions on Intelligent Transportation Systems, pp. 1–14, 2019. changing their data samples. This approach contrasts with tra- [7] H. Liu, P. Zhang, G. Pu, T. Yang, S. Maharjan, and Y. Zhang, ditional centralized machine learning techniques where all data “Blockchain empowered cooperative authentication with data trace- samples are uploaded to one server, as well as to more classical ability in vehicular edge computing,” IEEE Transactions on Vehicular Technology, vol. 69, no. 4, pp. 4221–4232, 2020. decentralized approaches which assume that local data samples [8] T. Kwon and J. Hong, “Analysis and improvement of a pin-entry are identically distributed [165]–[167]. The nature of federated method resilient to shoulder-surfing and recording attacks,” IEEE learning not only can prevent the data sharing among devices, Transactions on Information Forensics and Security, vol. 10, no. 2, but also avoid the enormous communication costs. In addition, pp. 278–292, Feb 2015. [9] M. Cagalj,ˇ T. Perkovic,´ and M. Bugaric,´ “Timing attacks on cognitive other privacy-preserving machine learning approaches such as authentication schemes,” IEEE Transactions on Information Forensics multi-party computation, homomorphic encryption are getting and Security, vol. 10, no. 3, pp. 584–596, March 2015. more attraction recently. For intrusted participants, the multi- [10] T. Chen, M. Farcasin, and E. Chan-Tin, “Smartphone passcode prediction,” IET Information Security, vol. 12, no. 5, pp. 431–437, 2018. party computation (MPC) is able to calculate a joint function [11] Y. Abdelrahman, M. Khamis, S. Schneegass, and F. Alt, “Stay cool! in a decentralized network on the premise of ensuring privacy understanding thermal attacks on mobile-based user authentication,” and independence of input [168]. in Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, ser. CHI ’17. New York, NY, USA: ACM, 2017, pp. 3751–3763. [12] D. Li, X. Zhang, M. Hu, G. Zhai, and X. Yang, “Physical password VII.CONCLUSION breaking via thermal sequence analysis,” IEEE Transactions on Infor- mation Forensics and Security, vol. 14, no. 5, pp. 1142–1154, May In IoT era, user authentication and identification are critical 2019. to ensure the security of connected things and the customiza- [13] A. J. Aviv, K. Gibson, E. Mossop, M. Blaze, and J. M. Smith, tion of passive services. However, conventional identification “Smudge attacks on smartphone touch screens,” in Proceedings of the 4th USENIX Conference on Offensive Technologies, ser. WOOT’10. methods suffer from several key drawbacks including dis- Berkeley, CA, USA: USENIX Association, 2010, pp. 1–7. creteness, obtrusiveness, and vulnerability. In this article, we [14] S. Schneegass, F. Steimle, A. Bulling, F. Alt, and A. Schmidt, propose the continuous authentication based on behavioral “Smudgesafe: Geometric image transformations for smudge-resistant user authentication,” in Proceedings of the 2014 ACM International biometrics, characterize the key features of CA based on user Joint Conference on Pervasive and Ubiquitous Computing, ser. Ubi- behaviors (e.g., invulnerability, continuity, unobtrusiveness, Comp ’14. New York, NY, USA: ACM, 2014, pp. 775–786. and convenience), and summarize the existing CA solutions [15] S. Kim, H. Lee, and T. Kwon, “Poster: Rethinking fingerprint identification on smartphones,” in Proceedings of the 2017 ACM SIGSAC from sensing and computing. Based on this taxonomy, we Conference on Computer and Communications Security, ser. CCS ’17. discuss the challenges and open issues from the perspective New York, NY, USA: ACM, 2017, pp. 2515–2517. of AI. [16] A. Roy, N. Memon, and A. Ross, “Masterprint: Exploring the vulnerability of partial fingerprint-based authentication systems,” IEEE Transactions on Information Forensics and Security, vol. 12, no. 9, pp. 2013–2025, Sep. 2017. ACKNOWLEDGMENT [17] Y. Ren, Z. Fang, D. Liu, and C. Chen, “Replay attack detection based on distortion by loudspeaker for voice authentication,” Multimedia This work is supported by the national key research Tools and Applications, pp. 1–14, 2018. and development program of China under grant No.: [18] Y. Wang, W. Cai, T. Gu, W. Shao, Y. Li, and Y. Yu, 2019YFB2102200, by the ministry of health of China under “Secure your voice: An oral airflow-based continuous liveness detection for voice assistants,” Proc. ACM Interact. Mob. Wearable grant No.: 2017ZX10303401-002 and 2017YFC1200302, by Ubiquitous Technol., vol. 3, no. 4, Dec. 2019. [Online]. Available: the natural science foundation of China under grant No.: https://doi.org/10.1145/3369811

[19] S. Gupta, A. Buriro, and B. Crispo, “Driverauth: A risk-based multi- [40] Z. Wu, Y. Huang, L. Wang, X. Wang, and T. Tan, “A comprehensive modal biometric-based driver authentication scheme for ride-sharing study on cross-view gait based human identification with deep cnns,” platforms,” Computers & Security, vol. 83, pp. 122–139, 2019. IEEE Transactions on Pattern Analysis and Machine Intelligence, [20] M. Gil, P. Giner, and V. Pelechano, “Personalization for unobtrusive vol. 39, no. 2, pp. 209–226, Feb 2017. service interaction,” Personal Ubiquitous Comput., vol. 16, no. 5, pp. [41] J. P. Singh, S. Jain, S. Arora, and U. P. Singh, “Vision-based gait 543–561, Jun. 2012. recognition: A survey,” IEEE Access, vol. 6, pp. 70 497–70 527, 2018. [21] Y. Sui, X. Zou, E. Y. Du, and F. Li, “Design and analysis of a highly [42] Y. Liang, X. Zhou, Z. Yu, and B. Guo, “Energy-efficient motion related user-friendly, secure, privacy-preserving, and revocable authentication activity recognition on mobile devices for pervasive healthcare,” Mobile method,” IEEE Transactions on Computers, vol. 63, no. 4, pp. 902– Networks and Applications, vol. 19, no. 3, pp. 303–317, Jun. 2014. 916, April 2014. [43] M. Umair Bin Altaf, T. Butko, and B. Juang, “Acoustic gaits: Gait [22] M. Frank, R. Biedert, E. Ma, I. Martinovic, and D. Song, “Touchalytics: analysis with footstep sounds,” IEEE Transactions on Biomedical On the applicability of touchscreen input as a behavioral biometric Engineering, vol. 62, no. 8, pp. 2001–2011, Aug 2015. for continuous authentication,” IEEE Transactions on Information [44] H. Ma and W. Liao, “Human gait modeling and analysis using a semi- Forensics and Security, vol. 8, no. 1, pp. 136–148, Jan 2013. markov process with ground reaction forces,” IEEE Transactions on [23] Z. Sitova,´ J. Sedˇ enka,ˇ Q. Yang, G. Peng, G. Zhou, P. Gasti, and K. S. Neural Systems and Rehabilitation Engineering, vol. 25, no. 6, pp. Balagani, “Hmog: New behavioral biometric features for continuous 597–607, June 2017. authentication of smartphone users,” IEEE Transactions on Information [45] P. Casale, O. Pujol, and P. Radeva, “Personalization and user verifica- Forensics and Security, vol. 11, no. 5, pp. 877–892, May 2016. tion in wearable systems using biometric walking patterns,” Personal [24] Y. Zheng, X. Ding, C. C. Y. Poon, B. P. L. Lo, H. Zhang, X. Zhou, Ubiquitous Comput., vol. 16, no. 5, pp. 563–580, Jun. 2012. G. Yang, N. Zhao, and Y. Zhang, “Unobtrusive sensing and wearable [46] D. Ma, G. Lan, W. Xu, M. Hassan, and W. Hu, “Unobtrusive user devices for health informatics,” IEEE Transactions on Biomedical verification using piezoelectric energy harvesting,” in Proceedings of Engineering, vol. 61, no. 5, pp. 1538–1554, 2014. the 14th EAI International Conference on Mobile and Ubiquitous [25] M. Weiser, “The computer for the 21st century,” IEEE Pervasive Systems: Computing, Networking and Services, ser. MobiQuitous 2017. Computing, vol. 1, no. 1, pp. 19–25, 2002. New York, NY, USA: ACM, 2017, pp. 541–542. [26] W. Meng, D. S. Wong, S. Furnell, and J. Zhou, “Surveying the [47] D. Baek, P. Musale, and J. Ryoo, “Walk to show your identity: development of biometric user authentication on mobile phones,” IEEE Gait-based seamless user authentication framework using deep neu- Communications Surveys Tutorials, vol. 17, no. 3, pp. 1268–1293, ral network,” in The 5th ACM Workshop on Wearable Systems and 2015. Applications, ser. WearSys ’19. New York, NY, USA: ACM, 2019, [27] N. Sae-Bae, J. Wu, N. Memon, J. Konrad, and P. Ishwar, “Emerging pp. 53–58. nui-based methods for user authentication: A new taxonomy and [48] G. Cola, M. Avvenuti, F. Musso, and A. Vecchio, “Gait-based authen- survey,” IEEE Transactions on Biometrics, Behavior, and Identity tication using a wrist-worn device,” in Proceedings of the 13th Inter- Science, vol. 1, no. 1, pp. 5–31, Jan 2019. national Conference on Mobile and Ubiquitous Systems: Computing, [28] A. Mosenia and N. K. Jha, “A comprehensive study of security of Networking and Services, ser. MOBIQUITOUS 2016. New York, NY, internet-of-things,” IEEE Transactions on Emerging Topics in Com- USA: ACM, 2016, pp. 208–217. puting, vol. 5, no. 4, pp. 586–602, Oct 2017. [49] M. D. Marsico and A. Mecca, “A survey on gait recognition via [29] C. X. Lu, H. Wen, S. Wang, A. Markham, and N. Trigoni, “Scan: wearable sensors,” ACM Comput. Surv., vol. 52, no. 4, pp. 86:1–86:39, Learning speaker identity from noisy sensor data,” in Proceedings of the Aug. 2019. 16th ACM/IEEE International Conference on Information Processing [50] Y. Ren, Y. Chen, M. C. Chuah, and J. Yang, “User verification in Sensor Networks, ser. IPSN ’17. New York, NY, USA: Association leveraging gait recognition for smartphone enabled mobile healthcare for Computing Machinery, 2017, p. 67–78. systems,” IEEE Transactions on Mobile Computing, vol. 14, no. 9, pp. [30] A. Buriro, B. Crispo, F. Del Frari, and K. Wrona, “Touchstroke: 1961–1974, Sep. 2015. Smartphone user authentication based on touch-typing biometrics,” in [51] F. Lin, A. Wang, Y. Zhuang, M. R. Tomita, and W. Xu, “Smart insole: New Trends in Image Analysis and Processing, V. Murino, E. Puppo, A wearable sensor device for unobtrusive gait monitoring in daily life,” D. Sona, M. Cristani, and C. Sansone, Eds., 2015, pp. 27–34. IEEE Transactions on Industrial Informatics, vol. 12, no. 6, pp. 2281– [31] A. J. Aviv, B. Sapp, M. Blaze, and J. M. Smith, “Practicality of 2291, Dec 2016. accelerometer side channels on smartphones,” in Proceedings of the [52] M. Muaaz and R. Mayrhofer, “Smartphone-based gait recognition: 28th Annual Computer Security Applications Conference, ser. ACSAC From authentication to imitation,” IEEE Transactions on Mobile Com- ’12, 2012, pp. 41–50. puting, vol. 16, no. 11, pp. 3209–3221, Nov 2017. [32] O. Hamdy and I. Traore,´ “Homogeneous physio-behavioral visual [53] C. Luo, J. Wu, J. Li, J. Wang, W. Xu, Z. Ming, B. Wei, W. Li, and and mouse-based biometric,” ACM Transactions on Computer-Human A. Y. Zomaya, “Gait recognition as a service for unobtrusive user Interaction, vol. 18, no. 3, Aug. 2011. identification in smart spaces,” ACM Trans. Internet Things, vol. 1, [33] N. Zheng, A. Paloski, and H. Wang, “An efficient user verification no. 1, Mar. 2020. system using angle-based mouse movement biometrics,” ACM Trans- [54] F. Xiao, Z. Guo, Y. Ni, X. Xie, S. Maharjan, and Y. Zhang, “Artificial actions on Information and System Security, vol. 18, no. 3, Apr. 2016. intelligence empowered mobile sensing for human flow detection,” [34] J. Roh, S. Lee, and S. Kim, “Keystroke dynamics for authentication IEEE Network, vol. 33, no. 1, pp. 78–83, 2019. in smartphone,” in 2016 International Conference on Information and [55] Z. Wang, B. Guo, Z. Yu, and X. Zhou, “Wi-fi csi-based behavior recog- Communication Technology Convergence (ICTC), Oct 2016, pp. 1155– nition: From signals and actions to activities,” IEEE Communications 1159. Magazine, vol. 56, no. 5, pp. 109–115, May 2018. [35] L. Lu and Y. Liu, “Safeguard: User reauthentication on smartphones [56] J. Zhang, B. Wei, W. Hu, S. S. Kanhere, and A. Tan, “Human via behavioral biometrics,” IEEE Transactions on Computational Social identification using wifi signal,” in 2016 IEEE International Conference Systems, vol. 2, no. 3, pp. 53–64, Sept 2015. on Pervasive Computing and Communication Workshops (PerCom [36] M. Abo-Zahhad, S. M. Ahmed, and S. N. Abbas, “A novel biometric Workshops), March 2016, pp. 1–2. approach for human identification and verification using eye blinking [57] Y. Zeng, P. H. Pathak, and P. Mohapatra, “Wiwho: Wifi-based person signal,” IEEE Signal Processing Letters, vol. 22, no. 7, pp. 876–880, identification in smart spaces,” in 2016 15th ACM/IEEE International July 2015. Conference on Information Processing in Sensor Networks (IPSN), [37] Y. Zhang, W. Hu, W. Xu, C. T. Chou, and J. Hu, “Continuous April 2016, pp. 1–12. authentication using eye movement response of implicit visual stimuli,” [58] W. Wang, A. X. Liu, and M. Shahzad, “Gait recognition using Proc. ACM Interact. Mob. Wearable Ubiquitous Technol., vol. 1, no. 4, wifi signals,” in Proceedings of the 2016 ACM International Joint pp. 177:1–177:22, Jan. 2018. Conference on Pervasive and Ubiquitous Computing, ser. UbiComp [38] A. George and A. Routray, “A score level fusion method for eye ’16. New York, NY, USA: ACM, 2016, pp. 363–373. movement biometrics,” Pattern Recognition Letters, vol. 82, pp. 207 [59] T. Xin, B. Guo, Z. Wang, P. Wang, J. Lam, V. Li, and Z. Yu, – 215, 2016. “Freesense: A robust approach for indoor human detection using wi-fi [39] S. Eberz, K. B. Rasmussen, V. Lenders, and I. Martinovic, “Looks like signals,” Proceedings of the ACM on Interactive, Mobile, Wearable and eve: Exposing insider threats using eye movement biometrics,” ACM Ubiquitous Technologies, vol. 2, pp. 1–23, 09 2018. Transactions on Privacy and Security, vol. 19, no. 1, pp. 1:1–1:31, Jun. [60] D. Gafurov, E. Snekkenes, and P. Bours, “Gait authentication and 2016. identification using wearable accelerometer sensor,” in 2007 IEEE

Workshop on Automatic Identification Advanced Technologies, June [80] F. Hong, M. Wei, S. You, Y. Feng, and Z. Guo, “Waving authentication: 2007, pp. 220–225. Your smartphone authenticate you on motion gesture,” in Proceedings [61] R. Cham and M. S. Redfern, “Changes in gait when anticipating of the 33rd Annual ACM Conference Extended Abstracts on Human slippery floors,” Gait & Posture, vol. 15, no. 2, pp. 159 – 171, 2002. Factors in Computing Systems, ser. CHI EA ’15. New York, NY, [62] K. Singhal and J. B. Casebolt, “Chapter 8 - aging and gait,” in Nutrition USA: ACM, 2015, pp. 263–266. and Functional Foods for Healthy Aging, R. R. Watson, Ed. Academic [81] S. Mondal and P. Bours, “Continuous authentication and identification Press, 2017, pp. 65 – 74. for mobile devices: Combining security and forensics,” in 2015 IEEE [63] J. Liang, Y. Cao, C. Zhang, S. Chang, K. Bai, and Z. Xu, “Additive International Workshop on Information Forensics and Security (WIFS), adversarial learning for unbiased authentication,” in 2019 IEEE/CVF Nov 2015, pp. 1–6. Conference on Computer Vision and Pattern Recognition (CVPR), [82] Z. Cai, C. Shen, and X. Guan, “Mitigating behavioral variability for 2019, pp. 11 420–11 429. mouse dynamics: A dimensionality-reduction-based approach,” IEEE [64] K. Diederichs, A. Qiu, and G. Shaker, “Wireless biometric individual Transactions on Human-Machine Systems, vol. 44, no. 2, pp. 244–255, identification utilizing millimeter waves,” IEEE Sensors Letters, vol. 1, 2014. no. 1, pp. 1–4, 2017. [83] L. M. Manevitz and M. Yousef, “One-class svms for document classi- [65] N. Zhao, Z. Zhang, M. U. Rehman, A. Ren, X. Yang, J. Zhao, fication,” J. Mach. Learn. Res., vol. 2, p. 139–154, Mar. 2002. W. Zhao, and B. Dong, “Authentication in millimeter-wave body- [84] B. Scholkopf,¨ R. Williamson, A. Smola, J. Shawe-Taylor, and J. Platt, centric networks through wireless channel characterization,” IEEE “Support vector method for novelty detection,” in Proceedings of Transactions on Antennas and Propagation, vol. 65, no. 12, pp. 6616– the 12th International Conference on Neural Information Processing 6623, 2017. Systems, ser. NIPS’99. Cambridge, MA, USA: MIT Press, 1999, p. [66] K. Matsuo, F. Okumura, M. Hashimoto, S. Sakazawa, and Y. Hatori, 582–588. “Arm swing identification method with template update for long term [85] F. T. Liu, K. M. Ting, and Z. Zhou, “Isolation forest,” in 2008 Eighth stability,” in Advances in Biometrics, S.-W. Lee and S. Z. Li, Eds. IEEE International Conference on Data Mining, 2008, pp. 413–422. Berlin, Heidelberg: Springer Berlin Heidelberg, 2007, pp. 211–221. [86] D. Shin and S. Kim, “Nearest mean classification via one-class svm,” [67] S. Li, A. Ashok, Y. Zhang, C. Xu, J. Lindqvist, and M. Gruteser, in 2009 International Joint Conference on Computational Sciences and “Whose move is it anyway? authenticating smart wearable devices Optimization, vol. 1, 2009, pp. 593–596. using unique head movement patterns,” in 2016 IEEE International [87] H. Lukashevich, S. Nowak, and P. Dunker, “Using one-class svm Conference on Pervasive Computing and Communications (PerCom), outliers detection for verification of collaboratively tagged image March 2016, pp. 1–9. training sets,” in 2009 IEEE International Conference on Multimedia [68] L. Lu, J. Yu, Y. Chen, H. Liu, Y. Zhu, L. Kong, and M. Li, “Lip reading- and Expo, 2009, pp. 682–685. based user authentication through acoustic sensing on smartphones,” [88] Y. Guerbai, Y. Chibani, and N. Abbas, “One-class versus bi-class IEEE/ACM Transactions on Networking, vol. 27, no. 1, pp. 447–460, svm classifier for off-line signature verification,” in 2012 International 2019. Conference on Multimedia Computing and Systems, 2012, pp. 206–210. [69] D. Lu, D. Huang, Y. Deng, and A. Alshamrani, “Multifactor user [89] F. T. Liu, K. M. Ting, and Z.-H. Zhou, “Isolation-based anomaly authentication with in-air-handwriting and hand geometry,” in 2018 detection,” ACM Trans. Knowl. Discov. Data, vol. 6, no. 1, Mar. 2012. International Conference on Biometrics (ICB), Feb 2018, pp. 255–262. [90] L. M. Mayron, “Biometric authentication on mobile devices,” IEEE [70] Y. Zou, M. Zhao, Z. Zhou, J. Lin, M. Li, and K. Wu, “Bilock: User Security Privacy, vol. 13, no. 3, pp. 70–73, 2015. authentication via dental occlusion biometrics,” Proc. ACM Interact. [91] R. Das, E. Piciucco, E. Maiorana, and P. Campisi, “Convolutional Mob. Wearable Ubiquitous Technol., vol. 2, no. 3, pp. 152:1–152:20, neural network for finger-vein-based biometric identification,” IEEE Sep. 2018. Transactions on Information Forensics and Security, vol. 14, no. 2, pp. [71] T. Rahman, A. T. Adams, M. Zhang, E. Cherry, B. Zhou, H. Peng, and 360–373, 2019. T. Choudhury, “Bodybeat: A mobile system for sensing non-speech [92] Z. Zhao and A. Kumar, “Improving periocular recognition by explicit body sounds,” in Proceedings of the 12th Annual International Con- attention to critical regions in deep neural network,” IEEE Transactions ference on Mobile Systems, Applications, and Services, ser. MobiSys on Information Forensics and Security, vol. 13, no. 12, pp. 2937–2952, ’14. New York, NY, USA: Association for Computing Machinery, 2018. 2014, p. 2–13. [93] Q. Zhang, H. Li, Z. Sun, and T. Tan, “Deep feature fusion for iris [72] J. Chauhan, S. Seneviratne, Y. Hu, A. Misra, A. Seneviratne, and and periocular biometrics on mobile devices,” IEEE Transactions on Y. Lee, “Breathing-based authentication on resource-constrained iot Information Forensics and Security, vol. 13, no. 11, pp. 2897–2912, devices using recurrent neural networks,” Computer, vol. 51, no. 5, 2018. pp. 60–67, May 2018. [94] E. Park, X. Cui, T. H. B. Nguyen, and H. Kim, “Presentation attack [73] J. Liu, Y. Dong, Y. Chen, Y. Wang, and T. Zhao, “Leveraging breathing detection using a tiny fully convolutional network,” IEEE Transactions for continuous user authentication,” in Proceedings of the 24th Annual on Information Forensics and Security, vol. 14, no. 11, pp. 3016–3025, International Conference on Mobile Computing and Networking, ser. 2019. MobiCom ’18. New York, NY, USA: ACM, 2018, pp. 786–788. [95] R. F. Nogueira, R. de Alencar Lotufo, and R. Campos Machado, [Online]. Available: http://doi.acm.org/10.1145/3241539.3267743 “Fingerprint liveness detection using convolutional neural networks,” [74] A. Alzubaidi and J. Kalita, “Authentication of smartphone users us- IEEE Transactions on Information Forensics and Security, vol. 11, ing behavioral biometrics,” IEEE Communications Surveys Tutorials, no. 6, pp. 1206–1213, 2016. vol. 18, no. 3, pp. 1998–2026, thirdquarter 2016. [96] W. Kang, H. Liu, W. Luo, and F. Deng, “Study of a full-view 3d [75] S. Amini, V. Noroozi, A. Pande, S. Gupte, P. S. Yu, and C. Kanich, finger vein verification technique,” IEEE Transactions on Information “Deepauth: A framework for continuous user re-authentication in mo- Forensics and Security, vol. 15, pp. 1175–1189, 2020. bile apps,” in Proceedings of the 27th ACM International Conference [97] A. Czajka, “Pupil dynamics for iris liveness detection,” IEEE Transac- on Information and Knowledge Management, ser. CIKM ’18. New tions on Information Forensics and Security, vol. 10, no. 4, pp. 726– York, NY, USA: ACM, 2018, pp. 2027–2035. 735, 2015. [76] G. Batchuluun, R. A. Naqvi, W. Kim, and K. R. Park, “Body- [98] O. V. Komogortsev, A. Karpov, and C. D. Holland, “Attack of movement-based human identification using convolutional neural net- mechanical replicas: Liveness detection with eye movements,” IEEE work,” Expert Systems with Applications, vol. 101, pp. 56 – 77, 2018. Transactions on Information Forensics and Security, vol. 10, no. 4, pp. [77] Y. Yang, B. Guo, Z. Wang, M. Li, Z. Yu, and X. Zhou, “Behavesense: 716–725, 2015. Continuous authentication for security-sensitive mobile apps using [99] C. Yuan, Z. Xia, L. Jiang, Y. Cao, Q. M. Jonathan Wu, and X. Sun, behavioral biometrics,” Ad Hoc Networks, vol. 84, pp. 9 – 18, 2019. “Fingerprint liveness detection using an improved cnn with image scale [78] G. Peng, D. T. Nguyen, G. Zhou, and S. Wang, “Poster: A continu- equalization,” IEEE Access, vol. 7, pp. 26 953–26 966, 2019. ous and noninvasive user authentication system for google glass,” in [100] Y. Xun, J. Liu, N. Kato, Y. Fang, and Y. Zhang, “Automobile driver Proceedings of the 13th Annual International Conference on Mobile fingerprinting: A new machine learning based authentication scheme,” Systems, Applications, and Services, ser. MobiSys ’15. New York, IEEE Transactions on Industrial Informatics, vol. 16, no. 2, pp. 1417– NY, USA: ACM, 2015, pp. 487–487. 1426, 2020. [79] K. Song, Y. Zhou, H. Liu, and N. Zhu, “Isolated forest in keystroke [101] Z. Qin, L. Hu, N. Zhang, D. Chen, K. Zhang, Z. Qin, and K. R. Choo, dynamics-based authentication: Only normal instances available for “Learning-aided user identification using smartphone sensors for smart training,” in 2017 2nd IEEE International Conference on Computa- homes,” IEEE Internet of Things Journal, vol. 6, no. 5, pp. 7760–7772, tional Intelligence and Applications (ICCIA), 2017, pp. 63–67. 2019.

[102] Q. Zou, Y. Wang, Q. Wang, Y. Zhao, and Q. Li, “Deep learning-based [122] C. X. Lu, X. Kan, B. Du, C. Chen, H. Wen, A. Markham, N. Trigoni, gait recognition using smartphones in the wild,” IEEE Transactions on and J. Stankovic, “Autonomous learning for face recognition in the Information Forensics and Security, vol. 15, pp. 3197–3212, 2020. wild via ambient wireless cues,” in The World Wide Web Conference, [103] R. Pascanu, T. Mikolov, and Y. Bengio, “On the difficulty of training ser. WWW ’19. New York, NY, USA: Association for Computing recurrent neural networks,” in Proceedings of the 30th International Machinery, 2019, p. 1175–1186. Conference on International Conference on Machine Learning - Volume [123] N. D. Lane, S. Bhattacharya, P. Georgiev, C. Forlivesi, and F. Kawsar, 28, ser. ICML’13. JMLR.org, 2013, p. III–1310–III–1318. “An early resource characterization of deep learning on wearables, [104] E. Strubell, A. Ganesh, and A. McCallum, “Energy and policy consid- smartphones and internet-of-things devices,” in Proceedings of the 2015 erations for deep learning in NLP,” in Proceedings of the 57th Annual International Workshop on Internet of Things Towards Applications, Meeting of the Association for Computational Linguistics. Florence, ser. IoT-App ’15. New York, NY, USA: ACM, 2015, pp. 7–12. Italy: Association for Computational Linguistics, Jul. 2019, pp. 3645– [124] S. Bai, J. Z. Kolter, and V. Koltun, “An empirical evaluation of generic 3650. convolutional and recurrent networks for sequence modeling,” arXiv: [105] H. Zhang, J. Liu, K. Li, H. Tan, and G. Wang, “Gait learning based Learning, 2018. authentication for intelligent things,” IEEE Transactions on Vehicular [125] T. Yang, Y. Chen, and V. Sze, “Designing energy-efficient convolutional Technology, vol. 69, no. 4, pp. 4450–4459, 2020. neural networks using energy-aware pruning,” in 2017 IEEE Confer- [106] J. Chauhan, J. Rajasegaran, S. Seneviratne, A. Misra, A. Seneviratne, ence on Computer Vision and Pattern Recognition (CVPR), July 2017, and Y. Lee, “Performance characterization of deep learning models for pp. 6071–6079. breathing-based authentication on resource-constrained devices,” Proc. [126] N. D. Lane, S. Bhattacharya, P. Georgiev, C. Forlivesi, L. Jiao, ACM Interact. Mob. Wearable Ubiquitous Technol., vol. 2, no. 4, Dec. L. Qendro, and F. Kawsar, “Deepx: A software accelerator for low- 2018. power deep learning inference on mobile devices,” in 2016 15th [107] C. X. Lu, B. Du, P. Zhao, H. Wen, Y. Shen, A. Markham, and ACM/IEEE International Conference on Information Processing in N. Trigoni, “Deepauth: In-situ authentication for smartwatches via Sensor Networks (IPSN), April 2016, pp. 1–12. deeply learned behavioural biometrics,” in Proceedings of the 2018 [127] S. Bhattacharya and N. D. Lane, “Sparsification and separation of ACM International Symposium on Wearable Computers, ser. ISWC deep learning layers for constrained resource inference on wearables,” ’18. New York, NY, USA: Association for Computing Machinery, in Proceedings of the 14th ACM Conference on Embedded Network 2018, p. 204–207. Sensor Systems CD-ROM, ser. SenSys ’16. New York, NY, USA: [108] I. J. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde- ACM, 2016, pp. 176–189. Farley, S. Ozair, A. Courville, and Y. Bengio, “Generative adversarial [128] S. Han, J. Pool, J. Tran, and W. J. Dally, “Learning both weights and nets,” in Proceedings of the 27th International Conference on Neural connections for efficient neural networks,” in Proceedings of the 28th Information Processing Systems - Volume 2, ser. NIPS’14. Cambridge, International Conference on Neural Information Processing Systems - MA, USA: MIT Press, 2014, p. 2672–2680. Volume 1, ser. NIPS’15. Cambridge, MA, USA: MIT Press, 2015, [109] A. Creswell, T. White, V. Dumoulin, K. Arulkumaran, B. Sengupta, pp. 1135–1143. and A. A. Bharath, “Generative adversarial networks: An overview,” [129] R. Appuswamy, T. Nayak, J. Arthur, S. Esser, P. Merolla, J. Mck- IEEE Signal Processing Magazine, vol. 35, no. 1, pp. 53–65, 2018. instry, T. Melano, M. Flickner, and D. Modha, “Structured Convolu- [110] P. Bontrager, A. Roy, J. Togelius, N. Memon, and A. Ross, “Deep- tion Matrices for Energy-efficient Deep learning,” arXiv e-prints, p. masterprints: Generating masterprints for dictionary attacks via latent arXiv:1606.02407, Jun 2016. variable evolution*,” in 2018 IEEE 9th International Conference on [130] Z. Lu, V. Sindhwani, and T. N. Sainath, “Learning compact recurrent Biometrics Theory, Applications and Systems (BTAS), 2018, pp. 1–9. neural networks,” in 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), March 2016, pp. 5960–5964. [111] A. van den Oord, S. Dieleman, H. Zen, K. Simonyan, O. Vinyals, [131] H. Pham, M. Y. Guan, B. Zoph, Q. V. Le, and J. Dean, “Efficient neural A. Graves, N. Kalchbrenner, A. Senior, and K. Kavukcuoglu, architecture search via parameter sharing,” 2018. “WaveNet: A Generative Model for Raw Audio,” arXiv e-prints, p. [132] X. Zhang, X. Zhou, M. Lin, and J. Sun, “Shufflenet: An extremely arXiv:1609.03499, Sep 2016. efficient convolutional neural network for mobile devices,” Computer [112] H. N. Ahmad and T. M. Barbosa, “The effects of backpack carriage Vision and Pattern Recognition, pp. 6848–6856, 2018. on gait kinematics and kinetics of schoolchildren,” Scientific Reports, [133] X. Dai, P. Zhang, B. Wu, H. Yin, F. Sun, Y. Wang, M. Dukhan, vol. 9, no. 3364, pp. 86–95, 2019. Y. Hu, Y. Wu, Y. Jia, P. Vajda, M. Uyttendaele, and N. K. Jha, [113] G. F. D. Greenstein, “Gait and balance deficits in chronic alcoholics: No “Chamnet: Towards efficient network design through platform-aware improvement from 10 weeks through 1 year abstinence,” Alcoholism model adaptation,” in The IEEE Conference on Computer Vision and Clinical & Experimental Research, vol. 37, no. 1, 2012. Pattern Recognition (CVPR), June 2019. [114] R. Z. Marandi, P. Madeleine, O. Omland, N. Vuillerme, and A. Samani, [134] L. Duong, T. Cohn, S. Bird, and P. Cook, “Low resource dependency “Eye movement characteristics reflected fatigue development in both parsing: Cross-lingual parameter sharing in a neural network parser,” young and elderly individuals,” Scientific Reports, vol. 8, no. 13148, in Proceedings of the 53rd Annual Meeting of the Association for 2018. Computational Linguistics and the 7th International Joint Conference [115] J. Galbally, R. Haraksim, and L. Beslay, “A study of age and ageing on Natural Language Processing (Volume 2: Short Papers). Beijing, in fingerprint biometrics,” IEEE Transactions on Information Forensics China: Association for Computational Linguistics, Jul. 2015, pp. 845– and Security, vol. 14, no. 5, pp. 1351–1365, May 2019. 850. [116] V. C. S. B. Z. K. Dimitrios Iakovakis, Stelios Hadjidimitriou and [135] J. Chen and X. Ran, “Deep learning with edge computing: A review,” L. J. Hadjileontiadis, “Touchscreen typing-pattern analysis for detecting Proceedings of the IEEE, vol. 107, no. 8, pp. 1655–1674, Aug 2019. fine motor skills decline in early-stage parkinson’s disease,” Scientific [136] Z. Zhou, X. Chen, E. Li, L. Zeng, K. Luo, and J. Zhang, “Edge Reports, vol. 8, no. 7663, 2018. intelligence: Paving the last mile of artificial intelligence with edge [117] X. Wang, T. Yu, M. Zeng, and P. Tague, “Xrec: Behavior-based computing,” Proceedings of the IEEE, vol. 107, no. 8, pp. 1738–1762, user recognition across mobile devices,” Proc. ACM Interact. Mob. Aug 2019. Wearable Ubiquitous Technol., vol. 1, no. 3, pp. 111:1–111:26, Sep. [137] Y. Kang, J. Hauswald, C. Gao, A. Rovinski, T. Mudge, J. Mars, and 2017. L. Tang, “Neurosurgeon: Collaborative intelligence between the cloud [118] K. Cao, C. Wei, A. Gaidon, N. Arechiga, and T. Ma, “Learning imbal- and mobile edge,” SIGARCH Comput. Archit. News, vol. 45, no. 1, pp. anced datasets with label-distribution-aware margin loss,” in Advances 615–629, Apr. 2017. in Neural Information Processing Systems 32, 2019, pp. 1567–1578. [138] Z. Zhao, K. M. Barijough, and A. Gerstlauer, “Deepthings: Distributed [119] S. Rahman, S. Khan, and F. Porikli, “A unified approach for conven- adaptive deep learning inference on resource-constrained iot edge tional zero-shot, generalized zero-shot, and few-shot learning,” IEEE clusters,” IEEE Transactions on Computer-Aided Design of Integrated Transactions on Image Processing, vol. 27, no. 11, pp. 5652–5667, Circuits and Systems, vol. 37, no. 11, pp. 2348–2359, Nov 2018. Nov 2018. [139] M. Xu, F. Qian, M. Zhu, F. Huang, S. Pushp, and X. Liu, “Deep- [120] S. J. Pan and Q. Yang, “A survey on transfer learning,” IEEE Transac- wear: Adaptive local offloading for on-wearable deep learning,” IEEE tions on Knowledge and Data Engineering, vol. 22, no. 10, pp. 1345– Transactions on Mobile Computing, pp. 1–1, 2019. 1359, Oct 2010. [140] T. Chen, S. Barbarossa, X. Wang, G. B. Giannakis, and Z. Zhang, [121] S. Al-Stouhi and C. K. Reddy, “Transfer learning for class imbalance “Learning and management for internet of things: Accounting for problems with inadequate data,” Knowledge and Information Systems, adaptivity and scalability,” Proceedings of the IEEE, vol. 107, no. 4, vol. 48, no. 1, pp. 201–228, Jul. 2016. pp. 778–796, April 2019.

[141] L. Zeng, E. Li, Z. Zhou, and X. Chen, “Boomerang: On-demand [161] D. Hintze, M. Fuller,¨ S. Scholz, R. D. Findling, M. Muaaz, P. Kapfer, cooperative deep neural network inference for edge intelligence on the E. Koch, and R. Mayrhofer, “Cormorant: Ubiquitous risk-aware multi- industrial internet of things,” IEEE Network, vol. 33, no. 5, pp. 96–103, modal biometric authentication across mobile devices,” Proc. ACM Sep. 2019. Interact. Mob. Wearable Ubiquitous Technol., vol. 3, no. 3, pp. 85:1– [142] V. Nguyen, P. Lin, and R. Hwang, “Energy depletion attacks in low 85:23, Sep. 2019. power wireless networks,” IEEE Access, vol. 7, pp. 51 915–51 932, [162] E. Toch, C. Bettini, E. Shmueli, L. Radaelli, A. Lanzi, D. Riboni, 2019. and B. Lepri, “The privacy implications of cyber security systems: A [143] L. Kang and H. Shen, “Preventing battery attacks on electrical vehicles technological survey,” ACM Comput. Surv., vol. 51, no. 2, Feb. 2018. based on data-driven behavior modeling,” in Proceedings of the 10th [163] Y. X. Marcus Tan, A. Iacovazzi, I. Homoliak, Y. Elovici, and A. Binder, ACM/IEEE International Conference on Cyber-Physical Systems, ser. “Adversarial attacks on remote user authentication using behavioural ICCPS ’19. New York, NY, USA: Association for Computing mouse dynamics,” in 2019 International Joint Conference on Neural Machinery, 2019, p. 35–46. Networks (IJCNN), 2019, pp. 1–10. [144] I. Tomic´ and J. A. McCann, “A survey of potential security issues in [164] Q. He, Y. Xu, Z. Liu, J. He, Y. Sun, and R. Zhang, “A privacy- existing wireless sensor network protocols,” IEEE Internet of Things preserving internet of things device management scheme based on Journal, vol. 4, no. 6, pp. 1910–1923, Dec 2017. blockchain,” International Journal of Distributed Sensor Networks, [145] I. Farris, T. Taleb, Y. Khettab, and J. Song, “A survey on emerging sdn vol. 14, no. 11, 2018. and nfv security mechanisms for iot systems,” IEEE Communications [165] S. Wang, T. Tuor, T. Salonidis, K. K. Leung, C. Makaya, T. He, and Surveys Tutorials, vol. 21, no. 1, pp. 812–837, 2019. K. Chan, “Adaptive federated learning in resource constrained edge [146] C. X. Lu, B. Du, H. Wen, S. Wang, A. Markham, I. Martinovic, computing systems,” IEEE Journal on Selected Areas in Communica- Y. Shen, and N. Trigoni, “Snoopy: Sniffing your smartwatch passwords tions, vol. 37, no. 6, pp. 1205–1221, June 2019. via deep sequence learning,” Proc. ACM Interact. Mob. Wearable [166] G. Xu, H. Li, S. Liu, K. Yang, and X. Lin, “Verifynet: Secure Ubiquitous Technol., vol. 1, no. 4, Jan. 2018. and verifiable federated learning,” IEEE Transactions on Information [147] H. Hu, C. Lin, C. Chang, and L. Chen, “Enhanced secure data backup Forensics and Security, vol. 15, pp. 911–926, 2020. scheme using multi-factor authentication,” IET Information Security, [167] H. Liu, H. Ning, Y. Zhang, and L. T. Yang, “Aggregated-proofs based vol. 13, no. 6, pp. 649–658, 2019. privacy-preserving authentication for v2g networks in the smart grid,” [148] J. Zhao, X. Xie, X. Xu, and S. Sun, “Multi-view learning overview: IEEE Transactions on Smart Grid, vol. 3, no. 4, pp. 1722–1733, 2012. Recent progress and new challenges,” Information Fusion, vol. 38, pp. [168] H. Gao, Z. Ma, S. Luo, and Z. Wang, “Bfr-mpc: A blockchain-based 43–54, 2017. fair and robust multi-party computation scheme,” IEEE Access, vol. 7, [149] Y. Li, M. Yang, and Z. Zhang, “A survey of multi-view representation pp. 110 439–110 450, 2019. learning,” IEEE Transactions on Knowledge and Data Engineering, vol. 31, no. 10, pp. 1863–1883, Oct 2019. [150] D. Kim, K. Chung, and K. Hong, “Person authentication using face, teeth and voice modalities for mobile device security,” IEEE Trans- actions on Consumer Electronics, vol. 56, no. 4, pp. 2678–2685, November 2010. [151] H. Crawford, K. Renaud, and T. Storer, “A framework for continuous, transparent mobile device authentication,” Computers & Security, vol. 39, pp. 127 – 136, 2013. Yunji Liang is an associate professor from Northwestern Polytechnical [152] B. Zhou, J. Lohokare, R. Gao, and F. Ye, “Echoprint: Two-factor au- University, China. He received his Ph.D. degree in computer science from thentication using acoustics and vision on smartphones,” in Proceedings Northwestern Polytechnical University in 2016. During 2012-2017, he worked of the 24th Annual International Conference on Mobile Computing and at the University of Arizona as visiting scholar and post-doctoral researcher Networking, ser. MobiCom ’18. New York, NY, USA: ACM, 2018, respectively. His research interests include pervasive computing, social com- pp. 321–336. puting and intelligent system. [153] H. Gomi, S. Yamaguchi, K. Tsubouchi, and N. Sasaya, “Towards authentication using multi-modal online activities,” in Proceedings of the 2017 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2017 ACM International Symposium on Wearable Computers, ser. UbiComp ’17. New York, Sagar Samtani is an assistant professor and Grant Thornton scholar in NY, USA: ACM, 2017, pp. 37–40. the operations and decision technologies department in the Kelley School [154] P. Kumar, S. Mukherjee, R. Saini, P. Kaushik, P. P. Roy, and D. P. of Business at Indiana University. He received his Ph.D. in Management Dogra, “Multimodal gait recognition with inertial sensor data and video Information Systems in the Artificial Intelligence Lab at the University of using evolutionary algorithm,” IEEE Transactions on Fuzzy Systems, Arizona in 2018. His research interests include cyber threat intelligence, AI vol. 27, no. 5, pp. 956–965, 2019. for cybersecurity, Dark Web analytics, and interpretable deep learning. [155] H. Feng, K. Fawaz, and K. G. Shin, “Continuous authentication for voice assistants,” in Proceedings of the 23rd Annual International Conference on Mobile Computing and Networking, ser. MobiCom ’17. New York, NY, USA: Association for Computing Machinery, 2017, p. 343–355. Bin Guo is a professor from Northwestern Polytechnical University, China. [156] S. Pradhan, W. Sun, G. Baig, and L. Qiu, “Combating replay attacks He received his Ph.D. degree in computer science from Keio University, Japan against voice assistants,” Proc. ACM Interact. Mob. Wearable Ubiqui- in 2009 and then was a post-doc researcher at Institut TELECOM SudParis tous Technol., vol. 3, no. 3, Sep. 2019. in France. His research interests include ubiquitous computing and mobile [157] J. Li, J. Zhao, F. Zhao, H. Liu, J. Li, S. Shen, J. Feng, and T. Sim, “Ro- crowd sensing. bust face recognition with deep multi-view representation learning,” in Proceedings of the 24th ACM International Conference on Multimedia, ser. MM ’16. New York, NY, USA: ACM, 2016, pp. 1068–1072. [158] C. L. P. Lim, W. L. Woo, S. S. Dlay, D. Wu, and B. Gao, “Deep multiview heartwave authentication,” IEEE Transactions on Industrial Informatics, vol. 15, no. 2, pp. 777–786, Feb 2019. Zhiwen Yu is a professor from Northwestern Polytechnical University, [159] D. Hintze, E. Koch, S. Scholz, and R. Mayrhofer, “Location-based China. He has worked as an Alexander Von Humboldt Fellow at Mannheim risk assessment for mobile authentication,” in Proceedings of the 2016 University, Germany from Nov. 2009 to Oct. 2010, a research fellow at Kyoto ACM International Joint Conference on Pervasive and Ubiquitous University, Japan from Feb. 2007 to Jan. 2009. His research interests cover Computing: Adjunct, ser. UbiComp ’16. New York, NY, USA: ACM, ubiquitous computing and HCI. 2016, pp. 85–88. [160] A. Wojtowicz´ and K. Joachimiak, “Model for adaptable context-based biometric authentication for mobile devices,” Personal Ubiquitous Comput., vol. 20, no. 2, pp. 195–207, Apr. 2016.