The art of breaking and designing captchas Elie Bursztein Session ID: HT02-402 Insert presenter logo here onSession slide master. SeeClassification: hidden xxxxxxxxxxxx slide 4 for direcons Insert presenter logo here on slide master. See hidden slide 4 for direcons 2 Insert presenter logo here on slide master. See hidden slide 4 for direcons 2 Insert presenter logo here on slide master. See hidden slide 4 for direcons 2 Insert presenter logo here on slide master. See hidden slide 4 for direcons 2 3 Elie Bursztein (@elie) https://elie.net 3 Elie Bursztein (@elie) https://elie.net 3 Elie Bursztein (@elie) https://elie.net 3 Elie Bursztein (@elie) https://elie.net 3 Elie Bursztein (@elie) https://elie.net World Most-Popular Captchas ! [Reddit] [CNN] [Megaupload] [eBay] [Baidu] [Recaptcha] [Authorize] [Captcha.net] [Skyrock] [Digg] [NIH] [Google] [Slashdot] [Wikipedia] [Blizzard] Elie Bursztein (@elie) https://elie.net 4 World Most-Popular Captchas ! [Reddit] [CNN] [Megaupload] [eBay] [Baidu] [Recaptcha] [Authorize] [Captcha.net] [Skyrock] [Digg] [NIH] [Google] [Slashdot] [Wikipedia] [Blizzard] Elie Bursztein (@elie) https://elie.net 4 Captcha Design Goal Hard for computer Hard for human Elie Bursztein (@elie) https://elie.net 5 Captcha Design Goal Hard for computer Human Hard for human Elie Bursztein (@elie) https://elie.net 5 Captcha Design Goal AI ? Hard for computer Human Hard for human Elie Bursztein (@elie) https://elie.net 5 Captcha Design Goal AI ? sweet spot Hard for computer Human Hard for human Elie Bursztein (@elie) https://elie.net 5 Focus of this talk ! xw How to break and design CAPTCHAs Elie Bursztein (@elie) https://elie.net 6 Based on the breaking 21 of the most popular schemes and designing the new Wikipedia captcha Elie Bursztein (@elie) https://elie.net 7 Outline Elie Bursztein (@elie) https://elie.net 8 Outline ! How to break text captcha Elie Bursztein (@elie) https://elie.net 8 Outline ! How to break text captcha ! How to make captchas easier for human Elie Bursztein (@elie) https://elie.net 8 Outline ! How to break text captcha ! How to make captchas easier for human ! How to break audio captcha Elie Bursztein (@elie) https://elie.net 8 Outline ! How to break text captcha ! How to make captchas easier for human ! How to break audio captcha ! How to break video captcha Elie Bursztein (@elie) https://elie.net 8 Evaluation metrics Accuracy Elie Bursztein (@elie) https://elie.net 9 Evaluation metrics Accuracy Solving time Elie Bursztein (@elie) https://elie.net 9 Evaluation metrics Accuracy Solving time Learnability Elie Bursztein (@elie) https://elie.net 9 How to Break Text-Captchas Insert presenter logo here on slide master. See hidden slide 4 for direcons 10 Think Lego Elie Bursztein (@elie) https://elie.net 11 7 3 3 11 How to break a captcha: example Elie Bursztein (@elie) https://elie.net 12 7 3 3 11 Pre-processing: background removal Elie Bursztein (@elie) https://elie.net 12 7 3 3 11 Pre-processing: background removal Elie Bursztein (@elie) https://elie.net 12 7 3 3 11 Pre-processing: captcha binarization Elie Bursztein (@elie) https://elie.net 12 7 3 3 11 Pre-processing: captcha binarization Elie Bursztein (@elie) https://elie.net 12 7 3 3 11 Pre-processing: Line detection Elie Bursztein (@elie) https://elie.net 12 7 3 3 11 Pre-processing: Line detection Elie Bursztein (@elie) https://elie.net 12 7 3 3 11 Pre-processing: Line removal Elie Bursztein (@elie) https://elie.net 12 7 3 3 11 Pre-processing: Line removal Elie Bursztein (@elie) https://elie.net 12 7 3 3 11 Segmentation: clustering algorithm Elie Bursztein (@elie) https://elie.net 12 7 3 3 1 Segmentation: clustering algorithm Elie Bursztein (@elie) https://elie.net 12 7 3 3 1 Segmentation: cluster separation Elie Bursztein (@elie) https://elie.net 12 3 7 13 Segmentation: cluster separation Elie Bursztein (@elie) https://elie.net 12 3 7 13 Post-segmentation: inverting rotation Elie Bursztein (@elie) https://elie.net 12 3 7 13 Post-segmentation: inverting rotation Elie Bursztein (@elie) https://elie.net 12 3 7 13 Recognition: Elie Bursztein (@elie) https://elie.net 12 Recognition: 3 7 1 3 Elie Bursztein (@elie) https://elie.net 12 Breaker 5 Stages Pipeline Slashdot captcha 13 Elie Bursztein (@elie) https://elie.net Breaker 5 Stages Pipeline Preprocessing 13 Elie Bursztein (@elie) https://elie.net Breaker 5 Stages Pipeline Preprocessing 13 Elie Bursztein (@elie) https://elie.net Breaker 5 Stages Pipeline Preprocessing Segmentation 13 Elie Bursztein (@elie) https://elie.net Breaker 5 Stages Pipeline Preprocessing Segmentation 13 Elie Bursztein (@elie) https://elie.net Breaker 5 Stages Pipeline Preprocessing Segmentation Post- segmentation 13 Elie Bursztein (@elie) https://elie.net Breaker 5 Stages Pipeline Preprocessing Segmentation Post- segmentation 13 Elie Bursztein (@elie) https://elie.net Breaker 5 Stages Pipeline Preprocessing Segmentation Post- segmentation Recognition 13 Elie Bursztein (@elie) https://elie.net Breaker 5 Stages Pipeline Preprocessing Segmentation Post- segmentation Recognition f a e t e s t 13 Elie Bursztein (@elie) https://elie.net Breaker 5 Stages Pipeline Preprocessing Segmentation Post- segmentation Recognition f a e t e s t Post-recognition 13 Elie Bursztein (@elie) https://elie.net Breaker 5 Stages Pipeline Preprocessing Segmentation Post- segmentation Recognition f a e t e s t Post-recognition f a s t e s t 13 Elie Bursztein (@elie) https://elie.net From the image to the matrix representation 14 From the image to the matrix representation 14 From the image to the matrix representation 14 From the image to the matrix representation 14 From the image to the matrix representation 14 From the matrix representation to the vector representation 15 From the matrix representation to the vector representation 15 From the matrix representation to the vector representation 15 L1 L2 From the matrix representation to the vector representation 15 L1 L2 L3 From the matrix representation to the vector representation 15 vectorL1 L2 L3 L4 L5 L6 From the matrix representation to the vector representation 15 Known vectors Distance A A B vector B C C From the vector representation to the segment value (classification) 16 Known vectors Distance A A B vector B C C From the vector representation to the segment value (classification) 16 Known vectors Distance A 42 A B vector B C C From the vector representation to the segment value (classification) 16 Known vectors Distance A 42 A B vector B C C From the vector representation to the segment value (classification) 16 Known vectors Distance A 42 A 40 B vector B C C From the vector representation to the segment value (classification) 16 Known vectors Distance A 42 A 40 B vector B C C From the vector representation to the segment value (classification) 16 Known vectors Distance A 42 A 40 B 32 vector B C C From the vector representation to the segment value (classification) 16 Known vectors Distance A 42 A 40 B 32 vector B 70 C 12 C 18 From the vector representation to the segment value (classification) 16 Known vectors Distance A 42 A 40 B 32 vector B 70 C 12 C 18 From the vector representation to the segment value (classification) 16 Breaker efficiency Solver accuracy = Coverage * Precision^length Coverage: Segmentation rate Precision: Recognition rate Elie Bursztein (@elie) https://elie.net 17 Anti-recognition techniques Elie Bursztein (@elie) http://elie.im 18 Anti-recognition techniques Blurring Elie Bursztein (@elie) http://elie.im 18 Anti-recognition techniques Blurring Distortion Elie Bursztein (@elie) http://elie.im 18 Anti-recognition techniques Blurring Distortion Rotation Elie Bursztein (@elie) http://elie.im 18 Anti-recognition techniques Blurring Distortion Rotation Fonts Elie Bursztein (@elie) http://elie.im 18 Anti-recognition techniques Blurring Distortion Rotation Fonts Charsets Elie Bursztein (@elie) http://elie.im 18 Anti-recognition techniques Blurring Distortion Rotation Fonts Charsets 0123456789 Elie Bursztein (@elie) http://elie.im 18 SVM learning rate 100% 90% 80% 70% 60% 50% 09 40% %success AZ09 30% azAZ09 Distortion 20% 3 fonts 10% 5 fonts Angles 0% 10 20 50 100 200 500 Trainning set size Elie Bursztein (@elie) https://elie.net 19 KNN learning rate 100% 90% 80% 70% 60% 50% 09 40% % success AZ09 30% azAZ09 Distortion 20% 3 fonts 10% 5 fonts Angles 0% 10 20 50 100 200 500 Trainning set size Elie Bursztein (@elie) https://elie.net 20 Anti-recognition taxonomy Elie Bursztein (@elie) http://elie.im 21 Anti-recognition taxonomy Background Confusion Elie Bursztein (@elie) http://elie.im 21 Background confusion Anti-recognition taxonomy Background Confusion Elie Bursztein (@elie) http://elie.im 21 Background confusion Anti-recognition taxonomy Background Confusion Elie Bursztein (@elie) http://elie.im 21 Background confusion Anti-recognition taxonomy Background Confusion Elie Bursztein (@elie) http://elie.im 21 Background confusion Anti-recognition taxonomy Background Confusion Lines Elie Bursztein (@elie) http://elie.im 21 Background confusion Anti-recognition taxonomy Background Confusion Lines Elie Bursztein (@elie) http://elie.im 21 Background confusion Anti-recognition taxonomy Background Confusion Lines Elie Bursztein (@elie) http://elie.im 21 Background confusion Anti-recognition taxonomy Background Confusion Lines Elie Bursztein (@elie) http://elie.im 21 Background confusion Anti-recognition taxonomy Background Confusion Lines Collapsing Elie Bursztein (@elie) http://elie.im 21 Background confusion Anti-recognition taxonomy Background Confusion Lines Collapsing