Syracuse University SURFACE Dissertations - ALL SURFACE August 2019 Inferences from Interactions with Smart Devices: Security Leaks and Defenses Diksha Shukla Syracuse University Follow this and additional works at: https://surface.syr.edu/etd Part of the Engineering Commons Recommended Citation Shukla, Diksha, "Inferences from Interactions with Smart Devices: Security Leaks and Defenses" (2019). Dissertations - ALL. 1060. https://surface.syr.edu/etd/1060 This Dissertation is brought to you for free and open access by the SURFACE at SURFACE. It has been accepted for inclusion in Dissertations - ALL by an authorized administrator of SURFACE. For more information, please contact [email protected]. ABSTRACT We unlock our smart devices such as smartphone several times every day using a pin, password, or graphical pattern if the device is secured by one. The scope and usage of smart devices’ are expanding day by day in our everyday life and hence the need to make them more secure. In the near future, we may need to authenticate ourselves on emerging smart devices such as electronic doors, exercise equipment, power tools, medical devices, and smart TV remote control. While recent research focuses on developing new behavior-based methods to authenticate these smart devices, pin and password still remain primary methods to authenticate a user on a device. Although the recent research exposes the observation-based vulnerabilities, the popular belief is that the direct observation attacks can be thwarted by simple methods that obscure the attacker’s view of the input console (or screen). In this dissertation, we study the users’ hand movement pattern while they type on their smart devices. The study concentrates on the following two factors; (1) finding security leaks from the observed hand movement patterns (we showcase that the user’s hand movement on its own reveals the user’s sensitive information) and (2) developing methods to build lightweight, easy to use, and more secure authentication system. The users’ hand movement patterns were captured through video camcorder and inbuilt motion sensors such as gyroscope and accelerometer in the user’s device. INFERENCES FROM INTERACTIONS WITH SMART DEVICES: SECURITY LEAKS AND DEFENSES by Diksha Shukla M.C.A., Jawaharlal Nehru University, 2011 M.S., Louisiana Tech University, 2014 Dissertation Submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy in Computer & Information Science and Engineering Syracuse University August 2019 Copyright c Diksha Shukla 2019 All Rights Reserved To my family. iv ACKNOWLEDGMENTS I would like to extend my sincere gratitude and thank my advisor, Prof. Vir V. Phoha for all the guidance, encouragement, and support during my PhD study. His patience, insightful feedback, and critical environment in many discussions have helped me formulate my dissertation topic and progress in my PhD research. I thank Prof. Shobha Bhatia, Prof. Wenliang Du, Prof. Zhanpeng Jin, Prof. Sucheta Soundarajan and Prof. Edmund S. Yu for agreeing to be on my research committee. I am grateful to my research committee for their time and efforts and helping me succeed in my PhD research. I have had great opportunity to collaborate with my fellow colleagues at Syracuse University and some great researchers, Prof. Nitesh Saxena and Prof. Zhanpeng Jin, outside of Syracuse University. I thank all my collaborators for the learning experience I had working with them. I would also like to thank my labmates and fellow graduate students for their help and great friendship. I thank all my friends in Syracuse and outside of Syracuse for their constant support and encouragement throughout this journey. Most of all, I would like to thank my parents, my sisters and my brother for their unconditional love and support. v TABLE OF CONTENTS Page ABSTRACT ..................................... i LIST OF TABLES .................................. xi LIST OF FIGURES ................................. xiii 1 Introduction ................................... 1 1.1 Thesis Statement .............................. 6 1.2 Dissertation Roadmap ........................... 9 2 Background ................................... 10 2.1 Attacks on Users’ Pins, Passwords, and Typed Text ............ 10 2.2 Authentication System Using Behavior Biometrics ............ 14 2.3 Attacks on Users’ Behavioral Biometrics ................. 16 2.3.1 Attack Models Using the EEG signals ............... 16 2.3.2 Attack model on smartwatches .................. 17 3 Deciphering Numeric Pins and Alphanumeric Passwords ............ 18 3.1 Introduction ................................ 19 3.1.1 Discussion – Follow-up Potential Adversarial Scenarios ..... 22 3.2 Related Work ............................... 23 3.3 Attack Details ............................... 30 3.3.1 Step 1 - Capturing Videos ..................... 32 3.3.2 Step 2 - Video preprocessing ................... 37 3.3.3 Step 3 -Hand and Phone Anchor Points Tracking ......... 38 3.3.4 Step 4 - Estimating the Fingertip Movement ............ 40 vi Page 3.3.5 Step 5 - Identifying Key Touch Frames .............. 41 3.3.6 Step 6 - Estimating the Location of Touch on Mobile Phone Screen 49 3.3.7 Step 7 - Determining Keypad State ................ 53 3.3.8 Step 8 - Recognizing Touched Keys ................ 57 3.4 Performance Evaluation .......................... 59 3.4.1 Key Touch Frame Detection ................... 59 3.4.2 Key Cluster Prediction ...................... 62 3.4.3 Keypad State Determination - Prediction of Shift and Number Key 63 3.4.4 Character Prediction Accuracy .................. 63 3.4.5 Effect of Various Parameter Settings ................ 66 3.4.6 Effect of Screen Size of the Victim’s Phone ............ 67 3.4.7 Discussion – Attack Performance ................. 68 3.4.8 Discussion – Assumptions, Limitations, and Mitigations ..... 73 3.5 Effect of Screen Size of the Victim’s Phone ................ 74 3.6 Conclusion ................................. 76 4 3D Vision Attack on the Numeric Pins ...................... 77 4.1 Introduction ................................ 77 4.2 Related Work ............................... 81 4.3 Attack Details ............................... 82 4.3.1 Data Collection and Participants’ Recruitment .......... 84 4.3.2 Video Tracking and Feature Extraction .............. 85 4.3.3 Feature Preprocessing ....................... 86 4.3.4 Classification ........................... 87 4.4 Performance Evaluation .......................... 88 4.4.1 Key-No Key Frame Classification ................. 88 4.4.2 Touched Key Classification .................... 89 vii Page 4.4.3 Overall Pin Prediction ....................... 91 4.5 Conclusion ................................. 91 5 Evidence Fusion Method to Decipher the Numeric Pins and Alphanumeric Pass- words ....................................... 93 5.1 Introduction ................................ 94 5.1.1 Baseline Method .......................... 98 5.2 Related Work ............................... 99 5.3 Proposed Method ............................. 106 5.3.1 Problem Definition ........................ 106 5.3.2 Dempster-Shafer Theory (DST) .................. 107 5.3.3 Confidence Assignment ...................... 108 5.4 Experiment Design ............................. 111 5.4.1 Data Collection Experiments ................... 111 5.4.2 Heuristics for the Assignment of Parameters ........... 112 5.4.3 Basic Probability Assignments .................. 115 5.4.4 Confidence Assignment: A Worked Out Example ......... 115 5.4.5 Decision Fusion Scheme ...................... 117 5.5 Performance Evaluation .......................... 117 5.5.1 Row and Column Inference .................... 120 5.5.2 Pin Inference ........................... 122 5.6 Conclusion ................................. 122 6 Attack on EEG based Authentication ...................... 123 6.1 Introduction ................................ 124 6.2 Attack Model ............................... 129 6.3 Experimental Setup ............................ 133 6.3.1 Data Collection Protocols ..................... 133 6.3.2 Data Pre-processing and Feature Extraction ............ 135 viii Page 6.3.3 Target Authentication System Prototype .............. 139 6.4 Correlation Analysis Details ........................ 140 6.5 Attack Results ............................... 142 6.5.1 Attack Performance for Each User ................. 144 6.5.2 Attack Performance for Decision Time .............. 144 6.5.3 Attack Performance for Time Interval between User Registration and Attack ............................... 147 6.5.4 Attack Performance with User’s Gender .............. 147 6.6 Related Work ............................... 148 6.7 List of Videos Used in our Study ...................... 150 6.8 Discrete Wavelet Transform ........................ 151 6.9 Statistical Significance of Error Rates - Comparative Analysis of the Attack Performance ................................ 153 6.10 Conclusion and Future Work ........................ 155 7 BodyTaps: A Gesture Based Authentication System ............... 156 7.1 Introduction ................................ 157 7.2 Related Work ............................... 161 7.3 Data Collection and Feature Analysis ................... 163 7.3.1 Data Collection .......................... 163 7.3.2 Data Preprocessing and Feature Analysis ............. 166 7.3.3 Feature Extraction ......................... 171 7.3.4 Feature Ranking and Selection .................. 175 7.4 Performance Evaluation .........................