How Good are Humans at Solving CAPTCHAs? A Large Scale Evaluation Elie Bursztein, Steven Bethard, Celine Fabry, John Lab Security Computer Stanford Mitchell, Dan Jurafsky, E. Bursztein, S. Bethard, C. Fabry, J. Mitchell, D. Jurafsky How Good are Humans at Solving CAPTCHAs? A Large Scale Evaluation http://ly.tl/p11 ? E. Bursztein, S. Bethard, C. Fabry, J. Mitchell, D. Jurafsky How Good are Humans at Solving CAPTCHAs? A Large Scale Evaluation http://ly.tl/p11 ? users E. Bursztein, S. Bethard, C. Fabry, J. Mitchell, D. Jurafsky How Good are Humans at Solving CAPTCHAs? A Large Scale Evaluation http://ly.tl/p11 ? bots users E. Bursztein, S. Bethard, C. Fabry, J. Mitchell, D. Jurafsky How Good are Humans at Solving CAPTCHAs? A Large Scale Evaluation http://ly.tl/p11 ? bots users CAPTCHA Completely Automated Public Turing test to tell Computers and Humans Apart E. Bursztein, S. Bethard, C. Fabry, J. Mitchell, D. Jurafsky How Good are Humans at Solving CAPTCHAs? A Large Scale Evaluation http://ly.tl/p11 E. Bursztein, S. Bethard, C. Fabry, J. Mitchell, D. Jurafsky How Good are Humans at Solving CAPTCHAs? A Large Scale Evaluation http://ly.tl/p11 E. Bursztein, S. Bethard, C. Fabry, J. Mitchell, D. Jurafsky How Good are Humans at Solving CAPTCHAs? A Large Scale Evaluation http://ly.tl/p11 E. Bursztein, S. Bethard, C. Fabry, J. Mitchell, D. Jurafsky How Good are Humans at Solving CAPTCHAs? A Large Scale Evaluation http://ly.tl/p11 E. Bursztein, S. Bethard, C. Fabry, J. Mitchell, D. Jurafsky How Good are Humans at Solving CAPTCHAs? A Large Scale Evaluation http://ly.tl/p11 E. Bursztein, S. Bethard, C. Fabry, J. Mitchell, D. Jurafsky How Good are Humans at Solving CAPTCHAs? A Large Scale Evaluation http://ly.tl/p11 E. Bursztein, S. Bethard, C. Fabry, J. Mitchell, D. Jurafsky How Good are Humans at Solving CAPTCHAs? A Large Scale Evaluation http://ly.tl/p11 E. Bursztein, S. Bethard, C. Fabry, J. Mitchell, D. Jurafsky How Good are Humans at Solving CAPTCHAs? A Large Scale Evaluation http://ly.tl/p11 E. Bursztein, S. Bethard, C. Fabry, J. Mitchell, D. Jurafsky How Good are Humans at Solving CAPTCHAs? A Large Scale Evaluation http://ly.tl/p11 E. Bursztein, S. Bethard, C. Fabry, J. Mitchell, D. Jurafsky How Good are Humans at Solving CAPTCHAs? A Large Scale Evaluation http://ly.tl/p11 E. Bursztein, S. Bethard, C. Fabry, J. Mitchell, D. Jurafsky How Good are Humans at Solving CAPTCHAs? A Large Scale Evaluation http://ly.tl/p11 E. Bursztein, S. Bethard, C. Fabry, J. Mitchell, D. Jurafsky How Good are Humans at Solving CAPTCHAs? A Large Scale Evaluation http://ly.tl/p11 E. Bursztein, S. Bethard, C. Fabry, J. Mitchell, D. Jurafsky How Good are Humans at Solving CAPTCHAs? A Large Scale Evaluation http://ly.tl/p11 E. Bursztein, S. Bethard, C. Fabry, J. Mitchell, D. Jurafsky How Good are Humans at Solving CAPTCHAs? A Large Scale Evaluation http://ly.tl/p11 E. Bursztein, S. Bethard, C. Fabry, J. Mitchell, D. Jurafsky How Good are Humans at Solving CAPTCHAs? A Large Scale Evaluation http://ly.tl/p11 E. Bursztein, S. Bethard, C. Fabry, J. Mitchell, D. Jurafsky How Good are Humans at Solving CAPTCHAs? A Large Scale Evaluation http://ly.tl/p11 E. Bursztein, S. Bethard, C. Fabry, J. Mitchell, D. Jurafsky How Good are Humans at Solving CAPTCHAs? A Large Scale Evaluation http://ly.tl/p11 E. Bursztein, S. Bethard, C. Fabry, J. Mitchell, D. Jurafsky How Good are Humans at Solving CAPTCHAs? A Large Scale Evaluation http://ly.tl/p11 E. Bursztein, S. Bethard, C. Fabry, J. Mitchell, D. Jurafsky How Good are Humans at Solving CAPTCHAs? A Large Scale Evaluation http://ly.tl/p11 E. Bursztein, S. Bethard, C. Fabry, J. Mitchell, D. Jurafsky How Good are Humans at Solving CAPTCHAs? A Large Scale Evaluation http://ly.tl/p11 E. Bursztein, S. Bethard, C. Fabry, J. Mitchell, D. Jurafsky How Good are Humans at Solving CAPTCHAs? A Large Scale Evaluation http://ly.tl/p11 E. Bursztein, S. Bethard, C. Fabry, J. Mitchell, D. Jurafsky How Good are Humans at Solving CAPTCHAs? A Large Scale Evaluation http://ly.tl/p11 E. Bursztein, S. Bethard, C. Fabry, J. Mitchell, D. Jurafsky How Good are Humans at Solving CAPTCHAs? A Large Scale Evaluation http://ly.tl/p11 93% E. Bursztein, S. Bethard, C. Fabry, J. Mitchell, D. Jurafsky How Good are Humans at Solving CAPTCHAs? A Large Scale Evaluation http://ly.tl/p11 93% 86% E. Bursztein, S. Bethard, C. Fabry, J. Mitchell, D. Jurafsky How Good are Humans at Solving CAPTCHAs? A Large Scale Evaluation http://ly.tl/p11 93% 86% 70% E. Bursztein, S. Bethard, C. Fabry, J. Mitchell, D. Jurafsky How Good are Humans at Solving CAPTCHAs? A Large Scale Evaluation http://ly.tl/p11 E. Bursztein, S. Bethard, C. Fabry, J. Mitchell, D. Jurafsky How Good are Humans at Solving CAPTCHAs? A Large Scale Evaluation http://ly.tl/p11 E. Bursztein, S. Bethard, C. Fabry, J. Mitchell, D. Jurafsky How Good are Humans at Solving CAPTCHAs? A Large Scale Evaluation http://ly.tl/p11 E. Bursztein, S. Bethard, C. Fabry, J. Mitchell, D. Jurafsky How Good are Humans at Solving CAPTCHAs? A Large Scale Evaluation http://ly.tl/p11 E. Bursztein, S. Bethard, C. Fabry, J. Mitchell, D. Jurafsky How Good are Humans at Solving CAPTCHAs? A Large Scale Evaluation http://ly.tl/p11 E. Bursztein, S. Bethard, C. Fabry, J. Mitchell, D. Jurafsky How Good are Humans at Solving CAPTCHAs? A Large Scale Evaluation http://ly.tl/p11 7.3 sec E. Bursztein, S. Bethard, C. Fabry, J. Mitchell, D. Jurafsky How Good are Humans at Solving CAPTCHAs? A Large Scale Evaluation http://ly.tl/p11 8.2 sec 7.3 sec E. Bursztein, S. Bethard, C. Fabry, J. Mitchell, D. Jurafsky How Good are Humans at Solving CAPTCHAs? A Large Scale Evaluation http://ly.tl/p11 8.2 sec 7.3 sec 9.3 sec E. Bursztein, S. Bethard, C. Fabry, J. Mitchell, D. Jurafsky How Good are Humans at Solving CAPTCHAs? A Large Scale Evaluation http://ly.tl/p11 8.2 sec 10.6 sec 7.3 sec 9.3 sec E. Bursztein, S. Bethard, C. Fabry, J. Mitchell, D. Jurafsky How Good are Humans at Solving CAPTCHAs? A Large Scale Evaluation http://ly.tl/p11 Outline • Study methodology • Population demography • Captcha measures E. Bursztein, S. Bethard, C. Fabry, J. Mitchell, D. Jurafsky How Good are Humans at Solving CAPTCHAs? A Large Scale Evaluation http://ly.tl/p11 Accuracy E. Bursztein, S. Bethard, C. Fabry, J. Mitchell, D. Jurafsky How Good are Humans at Solving CAPTCHAs? A Large Scale Evaluation http://ly.tl/p11 Accuracy Solving time E. Bursztein, S. Bethard, C. Fabry, J. Mitchell, D. Jurafsky How Good are Humans at Solving CAPTCHAs? A Large Scale Evaluation http://ly.tl/p11 The supply chain Websites Paper E. Bursztein, S. Bethard, C. Fabry, J. Mitchell, D. Jurafsky How Good are Humans at Solving CAPTCHAs? A Large Scale Evaluation http://ly.tl/p11 The supply chain Websites Paper Scraping E. Bursztein, S. Bethard, C. Fabry, J. Mitchell, D. Jurafsky How Good are Humans at Solving CAPTCHAs? A Large Scale Evaluation http://ly.tl/p11 The supply chain Websites Paper Scraping Solving E. Bursztein, S. Bethard, C. Fabry, J. Mitchell, D. Jurafsky How Good are Humans at Solving CAPTCHAs? A Large Scale Evaluation http://ly.tl/p11 The supply chain Websites Paper Scraping Solving Data Mining E. Bursztein, S. Bethard, C. Fabry, J. Mitchell, D. Jurafsky How Good are Humans at Solving CAPTCHAs? A Large Scale Evaluation http://ly.tl/p11 Scraping • Alexa top 50 • 23 scheme • 10 000 captcha samples • Custom scraper • Cookies • Javascript events • Ip rate limiting • User agent E. Bursztein, S. Bethard, C. Fabry, J. Mitchell, D. Jurafsky How Good are Humans at Solving CAPTCHAs? A Large Scale Evaluation http://ly.tl/p11 Scraping • Alexa top 50 • 23 scheme • 10 000 captcha samples • Custom scraper • Cookies • Javascript events • Ip rate limiting • User agent E. Bursztein, S. Bethard, C. Fabry, J. Mitchell, D. Jurafsky How Good are Humans at Solving CAPTCHAs? A Large Scale Evaluation http://ly.tl/p11 E. Bursztein, S. Bethard, C. Fabry, J. Mitchell, D. Jurafsky How Good are Humans at Solving CAPTCHAs? A Large Scale Evaluation http://ly.tl/p11 Authorize E. Bursztein, S. Bethard, C. Fabry, J. Mitchell, D. Jurafsky How Good are Humans at Solving CAPTCHAs? A Large Scale Evaluation http://ly.tl/p11 Authorize Baidu E. Bursztein, S. Bethard, C. Fabry, J. Mitchell, D. Jurafsky How Good are Humans at Solving CAPTCHAs? A Large Scale Evaluation http://ly.tl/p11 Authorize Baidu captcha.net E. Bursztein, S. Bethard, C. Fabry, J. Mitchell, D. Jurafsky How Good are Humans at Solving CAPTCHAs? A Large Scale Evaluation http://ly.tl/p11 Authorize Baidu captcha.net eBay E. Bursztein, S. Bethard, C. Fabry, J. Mitchell, D. Jurafsky How Good are Humans at Solving CAPTCHAs? A Large Scale Evaluation http://ly.tl/p11 Authorize Baidu captcha.net eBay Digg E. Bursztein, S. Bethard, C. Fabry, J. Mitchell, D. Jurafsky How Good are Humans at Solving CAPTCHAs? A Large Scale Evaluation http://ly.tl/p11 Authorize Baidu captcha.net eBay Digg Google E. Bursztein, S. Bethard, C. Fabry, J. Mitchell, D. Jurafsky How Good are Humans at Solving CAPTCHAs? A Large Scale Evaluation http://ly.tl/p11 Authorize Baidu captcha.net eBay Digg Google Blizzard E. Bursztein, S. Bethard, C. Fabry, J. Mitchell, D. Jurafsky How Good are Humans at Solving CAPTCHAs? A Large Scale Evaluation http://ly.tl/p11 Authorize Yahoo Baidu captcha.net eBay Digg Google Blizzard E. Bursztein, S.