Shepherd: Enabling Large-Scale Scanning of Websites After Social Single Sign-On
Total Page:16
File Type:pdf, Size:1020Kb
Open University of the Netherlands faculty of Management, Science & Technology Bachelor Computer Science Shepherd: Enabling Large-Scale Scanning of Websites after Social Single Sign-on Chair(wo)man: Authors: prof. dr. Tanja Vos Jelle Kalkman Supervisors: Alan Verresen dr. ir. Hugo Jonker Benjamin Krumnow, MSc. Presentation date: 02-08-2019 July 31, 2019 Course code: IB9906 Abstract Session security for web applications should keep users safe from session hijacking and ensure privacy and security in their online lives. A substantial part of their online lives are hidden behind a login field. To study how secure and private their online lives are, it is needed to do this from the same, authenticated, perspective. However, for a large-scale study, this would require us to automate authentication and account creation for a large number of web applications. This has proven to be a major challenge because web application can roll out countermea- sures against automated account creation, valid credentials should be used, the web is very heterogeneous, and several ethical concerns need to be addressed. We found that we could leverage Single Sign-On, such as signing in with Facebook or Google, to automate authentication for a large amount of websites. At least 6.3 % of the web applications in the daily Alexa top 1M lists offer the option to use Single Sign-On to authenticate and in 56 % of the cases this is sufficient to reach the authenticated state. We extended the Shepherd framework to increase the reach of research on session security by adding modules to automatically detect if a web application offers Social Single Sign-On and, if so, the ability to automatically authenticate for that web application. With these extensions Shepherd is able to detect Single Sign-On and managed to authen- ticate in almost all cases where no additional account creation handling is needed. Contents 1 Introduction 5 1.1 Aim, Scope, and Contributions of Study . .6 1.2 Context of Study . .6 1.3 Overview of Thesis . .6 2 Background 7 2.1 Web Session Management . .7 2.2 Web Session Hijacking . .8 2.3 Web Session Security . .9 2.4 Social Single Sign-On . 11 2.5 Web Crawling . 12 2.6 Browser Automation Tools . 14 2.7 Summary . 15 3 Related Work 16 3.1 SSO Security . 16 3.2 SSO Detection . 17 3.3 The Shepherd Project . 18 3.4 Summary . 18 4 The Shepherd Framework 19 4.1 Introduction . 19 4.2 Problem Description . 19 4.3 Shepherd's Architecture . 19 4.3.1 Apps and Modules . 20 4.4 Stable Web Pages . 20 4.5 Summary . 21 5 Automated Detection of Social Single Sign-On Authentication 22 5.1 Overview . 23 5.2 Prevalence of SSSO On The Web . 24 5.2.1 Problem Description . 24 5.2.2 Observations . 24 5.2.3 Conclusions . 26 5.3 Integration of SSSO Authentication . 26 5.3.1 Problem Description . 26 5.3.2 Observations . 26 5.3.3 Conclusions . 28 5.4 Detection of SSSO Authentication . 28 5.4.1 Problem Description . 28 5.4.2 Observations . 29 1 5.4.3 Conclusions . 31 5.5 Locations of SSSO Buttons . 31 5.5.1 Problem Description . 31 5.5.2 Observations . 31 5.5.3 Conclusions . 33 5.6 Recognizing SSSO Buttons . 34 5.6.1 Problem Description . 34 5.6.2 Observations . 34 5.6.3 Conclusions . 38 5.7 Clicking on SSSO Buttons . 39 5.7.1 Problem Description . 39 5.7.2 Observations . 39 5.7.3 Conclusions . 43 5.8 Finding URLs of Relevant Web Pages . 43 5.8.1 Problem Description . 43 5.8.2 Observations . 43 5.8.3 Conclusions . 44 5.9 Searching for SSSO Buttons On A Page . 45 5.9.1 Problem Description . 45 5.9.2 Observations . 45 5.9.3 Conclusions . 47 5.10 Searching for SSSO Buttons On A Website . 48 5.10.1 Problem Description . 48 5.10.2 Observations . 48 5.10.3 Conclusions . 50 5.11 Discussion and Final Conclusions . 51 5.11.1 Evaluation . 51 5.11.2 Future Work . 51 5.11.3 Conclusions . 51 6 Automated Authentication via Social Single Sign-On 52 6.1 Overview . 52 6.2 Commonalities and Differences in Authenticating to Different SSSO Identity Providers . 53 6.2.1 Problem Description . 53 6.2.2 Observations . 53 6.2.3 Conclusions . 57 6.3 Automating Social Single Sign-On . 58 6.3.1 The SSSO Login Module . 58 6.3.2 Testing the SSSO Login Module . 59 6.3.3 Conclusions . 60 6.4 Account Creation . 60 6.4.1 Problem Description . 60 6.4.2 Observations . 60 6.4.3 Registration Forms . 62 6.4.4 Conclusions . 65 6.5 Processing Forms with Formasaurus . 65 6.5.1 Formasaurus . 65 6.5.2 Shepherd's Account Creation Module . 65 6.5.3 Testing and Tuning the Account Creation . 66 6.5.4 Conclusions . 67 2 6.6 Verification of the Authenticated State . 68 6.6.1 Problem Description . 68 6.6.2 Observations . 68 6.6.3 Conclusions . 71 6.7 Automating the Verification . 71 6.7.1 Shepherds SSSO Verification Module . 71 6.7.2 Testing Shepherds Verification Module . 72 6.7.3 Conclusions . 72 6.8 Results and Discussion . 73 7 Conclusions and Future Work 74 Appendices 78 A SSSO Detection Experiments 79 A.1 Prevalence of SSSO On The Web . 79 A.1.1 Sampling . 79 A.1.2 Process . ..