UNIVERSITY OF CALIFORNIA, SAN DIEGO Understanding the Role of Outsourced Labor in Web Service Abuse A dissertation submitted in partial satisfaction of the requirements for the degree Doctor of Philosophy in Computer Science by Marti A. Motoyama Committee in charge: Professor Stefan Savage, Co-Chair Professor George Varghese, Co-Chair Professor Geoffrey M. Voelker, Co-Chair Professor Gert Lanckriet Professor Akos Ronas-Tas Professor Lawrence K. Saul 2011 Copyright Marti A. Motoyama, 2011 All rights reserved. The dissertation of Marti A. Motoyama is approved, and it is acceptable in quality and form for publication on microfilm and electronically: Co-Chair Co-Chair Co-Chair University of California, San Diego 2011 iii DEDICATION To my family, friends, and loved ones, who all made this journey a little less painful. iv EPIGRAPH Cool story, bro. —Christopher William Kanich A mind needs books as a sword needs a whetstone, if it is to keep its edge. —Tyrion Lannister I think it is the excitement that only a free man can feel, a free man starting a long journey whose conclusion in uncertain. —Red v TABLE OF CONTENTS Signature Page ................................... iii Dedication ...................................... iv Epigraph ....................................... v Table of Contents .................................. vi List of Figures .................................... ix List of Tables .................................... xii Acknowledgements . xiii Vita ......................................... xv Abstract of the Dissertation .............................xvii Chapter 1 Introduction ............................. 1 1.1 Contributions ......................... 3 1.2 Organization ......................... 4 Chapter 2 Background and Related Work ................... 6 2.1 Freelancing Sites ....................... 6 2.2 CAPTCHAs .......................... 9 2.3 Related Work ......................... 10 2.3.1 Outsourcing Abuse Tasks . 10 2.3.2 CAPTCHA Solving . 11 Chapter 3 The Role of Freelance Labor in Web Service Abuse . 13 3.1 Data Overview ........................ 15 3.1.1 Data Collection Methodology . 15 3.1.2 Data Summary .................... 15 3.1.3 Categorizing Jobs ................... 17 3.1.4 Posting Job Listings . 19 3.2 Case Studies .......................... 20 3.2.1 Accounts ....................... 20 3.2.2 OSN Linking ..................... 27 3.2.3 Spamming ...................... 33 3.2.4 Search Engine Optimization . 36 3.3 User Analysis ......................... 41 3.3.1 Country of Origin ................... 42 vi 3.3.2 Specialization ..................... 44 3.3.3 Reputation and Lifetime . 44 3.4 Discussion ........................... 45 3.5 Summary ........................... 46 Chapter 4 Understanding CAPTCHA-Solving Services in an Economic Con- text ................................. 47 4.1 Automated Software Solvers . 49 4.1.1 Empirical Case Studies . 49 4.1.2 Economics ...................... 52 4.2 Human Solver Services .................... 54 4.2.1 Opportunistic Solving . 54 4.2.2 Paid Solving ..................... 55 4.2.3 Economics ...................... 55 4.2.4 Active Measurement Issues . 57 4.3 Solver Service Quality .................... 59 4.3.1 Customer Account Creation . 59 4.3.2 Customer Interface . 60 4.3.3 Service Pricing .................... 61 4.3.4 Test Corpus ...................... 61 4.3.5 Verifying Solutions . 62 4.3.6 Quality of Service . 62 4.3.7 Value ......................... 68 4.3.8 Capacity ....................... 69 4.3.9 Load and Availability . 70 4.4 Workforce ........................... 72 4.4.1 Account Creation ................... 72 4.4.2 Worker Interface ................... 72 4.4.3 Worker Wages .................... 73 4.4.4 Geolocating Workers . 75 4.4.5 Adaptability ...................... 77 4.4.6 Targeted Sites ..................... 80 4.5 Summary ........................... 81 Chapter 5 Conclusion ............................. 85 5.1 Future Directions ....................... 86 5.1.1 Purchasing Via Retail Sites Versus Outsourcing . 86 5.1.2 Identification of Abuse Jobs at Scale . 87 5.1.3 Combating Human-Based CAPTCHA Solving . 88 5.2 Final Thoughts ........................ 88 Appendix A Interesting Job Posts and Bids from Freelancer.com . 90 A.1 Interesting Jobs ........................ 90 vii A.1.1 Legitimate ...................... 90 A.1.2 Accounts ....................... 90 A.1.3 SEO .......................... 91 A.1.4 Spamming ...................... 91 A.1.5 OSN Linking ..................... 92 A.1.6 Miscellaneous .................... 92 A.2 Interesting Bids ........................ 93 A.2.1 Accounts ....................... 93 A.2.2 SEO .......................... 93 A.2.3 Spamming ...................... 93 A.2.4 OSN Linking ..................... 94 A.2.5 Misc .......................... 94 Bibliography .................................... 95 viii LIST OF FIGURES Figure 2.1: Screenshot showing a job post on Freelancer.com. The numbers correspond to the following fields: –project budget, –employer username and rating, –project description, –employer selected keywords. ............................... 7 Figure 2.2: Screenshot demonstrating the various fields for the worker bids on Freelancer.com. The numbers correspond to the following fields: –worker username, –worker bid, typically not reflective of de- sired salary, –worker rating, –worker message to employer de- tailing skills, additional messages are sent via private message. 8 Figure 2.3: Examples of CAPTCHAs from various Internet properties. 9 Figure 3.1: Growth in Freelancer activity over time. Numbers in parentheses in the legend denote the largest activity in a month. 16 Figure 3.2: Median monthly prices offered by buyers for 1,000 CAPTCHA solves (top) and the monthly volume of CAPTCHA solving posts (bottom), both as functions of time. The solid vertical price bars show 25% to 75% price quartiles. 24 Figure 3.3: Sites targeted in account registration jobs. 26 Figure 3.4: Demand for account registration jobs over time. The dashed vertical lines indicate approximate dates when Craigslist introduced phone verification for erotic services ads (March 2008) and other services (May 2008). ............................. 27 Figure 3.5: Number of job postings for social networking links. 29 Figure 3.6: Social link growth rate. The first day a social link appears is counted as Day 1. ............................... 31 Figure 3.7: The number of user accounts common to each pair of workers hired to create social links. Labeled solid lines indicate at least 100 user accounts (out of 1,000 requested) in common, dashed lines indicate at least 10 but fewer than 100 user accounts in common. Work per- formed by MY1, PK4, and BD2 was done in April, while the remaining jobs were done roughly a month later. 32 Figure 3.8: Median number of friends vs. median number of page social links for the sets of users linked to our websites. 32 Figure 3.9: Median monthly prices offered by buyers for each Craigslist ad posted (top), and the monthly number of posts (bottom), both as a function of time. The solid vertical price bars show 25% to 75% price quartiles. The dashed vertical lines indicate approximate dates when Craigslist introduced phone verification for erotic services ads (March 2008) and other services (May 2008). The three bids re- ceived in response to our solicitation are indicated with a triangle on the right edge. ........................... 35 ix Figure 3.10: Median monthly prices offered by buyers for each article posted (top) and monthly average number of buyer posts per day (bottom), as a function of time. Vertical price bars show 25% to 75% price quartiles. ............................... 37 Figure 3.11: Average price buyers offered for backlinks on pages with a given PageRank (PR). Higher PRs correspond to more popular (and valu- able) pages. The number above the bar corresponds to the number of jobs requesting backlinks of that PR. 40 Figure 3.12: Distributions of countries for buyers and bidders. 42 Figure 3.13: Top five countries of buyers posting abusive jobs. 43 Figure 3.14: Top five countries of workers bidding on abusive jobs. 43 Figure 3.15: How the various elements of the market fit together . 45 Figure 4.1: Examples of CAPTCHAs downloaded directly from reCaptcha at dif- ferent time periods. .......................... 51 Figure 4.2: CAPTCHA-solving market workflow: GYC Automator tries to register a Gmail account and is challenged with a Google CAPT- CHA. GYC uses the DeCaptcher plug-in to solve the CAPTCHA at $2 per 1,000. DeCaptcher queues the CAPTCHA for a worker on the affiliated PixProfit back end. PixProfit selects a worker and pays at $1/1,000. Worker enters a solution to PixProfit, which returns it to the plug-in. GYC then enters the solution for the CAPTCHA to Gmail to register the account. 54 Figure 4.3: Median error rate and response time (in seconds) for all services. Services are ranked top-to-bottom in order of increasing error rate. 63 Figure 4.4: Error rate and median response time for each combination of ser- vice and CAPTCHA type. The area of each circle upper table is pro- portional to the error rate (among solved CAPTCHAs). In the lower table, circle area is proportional to the response time minus ten sec- onds (for increased contrast); negative values are denoted by un- shaded circles. Numeric values corresponding to the values in the leftmost and rightmost columns are shown on the side. Thus, the error rate of BypassCaptcha on Youku CAPTCHAs is 66%, and for BeatCaptchas on PayPal 4%. The median