(12) United States Patent (10) Patent No.: US 9.408,076 B2 Chen Et Al
Total Page:16
File Type:pdf, Size:1020Kb
USOO9408076B2 (12) United States Patent (10) Patent No.: US 9.408,076 B2 Chen et al. (45) Date of Patent: Aug. 2, 2016 (54) SENSOR-ASSISTED BIOMETRIC (56) References Cited AUTHENTICATION FOR SMARTPHONES U.S. PATENT DOCUMENTS (71) Applicant: The Regents of the University of California, Oakland, CA (US) 8,542,879 B1* 9/2013 Nechyba ............ GO6K9/00228 382,103 (72) Inventors: Shaxun Chen, Menlo Park, CA (US); 8,805,110 B2 * 8/2014 Rhoads ......................... 382/255 Amit Pande, Davis, CA (US); Prasant 8,913,004 B1* 12/2014 Bozarth ............. GO6K9/OO604 Mohapatra, Davis, CA (US) 345,156 20080118152 A1* 5/2008 Thorn ................... G06F 1,3218 (73) Assignee: The Regents of the University of 382,190 California, Oakland, CA (US) 2014/0025973 A1* 1/2014 Schillings ................ H04Q 9/00 T13,323 (*) Notice: Subject to any disclaimer, the term of this 2015/00 19440 A1* 1/2015 Yang ...................... G06Q 20/36 patent is extended or adjusted under 35 705/65 U.S.C. 154(b) by 0 days. 2015,0055821 A1* 2, 2015 Fotland ................ GO6K 9,3241 382,103 (21) Appl. No.: 14/711,631 * cited by examiner (22) Filed: May 13, 2015 Primary Examiner — Khai MNguyen (65) Prior Publication Data (74) Attorney, Agent, or Firm — Park, Vaughan, Fleming & US 2015/0334567 A1 Nov. 19, 2015 Dowler LLP, Laxman Sahasrabuddhe (57) ABSTRACT Related U.S. Application Data During a technique for authenticating a user using a portable (60) Provisional application No. 61/993,017, filed on May electronic device (such aS a smartphone) while preventing 14, 2014. Successful 2D media attacks and virtual-camera attacks, the portable electronic device provides an instruction to a user to (51) Int. Cl. move the portable electronic device relative to the user's face. H04M I/66 (2006.01) Then, using an imaging sensor in the portable electronic HO4WI2/06 (2009.01) device, the portable electronic device acquires, a series of G06K 9/00 (2006.01) images while the portable electronic device is at different (52) U.S. Cl. positions or orientations relative to the user's face, where the CPC ........... H04 W 12/06 (2013.01); G06K 9/00248 series of images include at least two images relative to a (2013.01); G06K 9/00288 (2013.01); G06K symmetry axis of the user's face (such as the midline). More s 9/00302 (201301) over, the portable electronic device determines a change in a (58) Field of Classification Search sign of a nose angle in at least the two images. Next, the CPC HO4W 12/06 GO6K9/OO3O2; GO6K portable electronic device authenticates the user based on at - - - - - - - - - - - - - - 9FOO248: G06K 9f0O288 least one of the series of images using a facial-recognition USPC ....................... 455/411,418,419,420,550.1 technique. See application file for complete search history. 20 Claims, 6 Drawing Sheets - 200 StART ROWAN INSTRUCTION 210 NCREASE ARIGHTNESS OF A ISPLAY (OPTIONAL) 212 ACUREASERIES OF IMAS 214 DETERMINEACHANGENAFACA FATURE OR CRENTATION 216 ASSOCATE MOTION INFORMATION WITHA CHANGE INACSITION OF AN OBJECT (OPTIONAL) 218 AUTHENTCATAUSER 220 U.S. Patent Aug. 2, 2016 Sheet 1 of 6 US 9.408,076 B2 U.S. Patent Aug. 2, 2016 Sheet 2 of 6 US 9.408,076 B2 - 200 PROVIDE AN INSTRUCTION 210 INCREASE ABRIGHTNESS OF A DISPLAY (OPTIONAL) 212 ACOUIRE A SERIES OF IMAGES 214 DETERMINE A CHANGE IN AFACIAL FEATURE OR ORIENTATION 216 ASSOCATE MOTION INFORMATION WITH A CHANGE INA POSITION OF AN OBJECT (OPTIONAL) 218 AUTHENTICATE AUSER 220 END FIG. 2 U.S. Patent Aug. 2, 2016 Sheet 3 of 6 US 9.408,076 B2 MOTION N SMARTPHONE FIG. 3 U.S. Patent Aug. 2, 2016 Sheet 4 of 6 US 9.408,076 B2 IMAGE IMAGE U.S. Patent Aug. 2, 2016 Sheet 5 of 6 US 9.408,076 B2 2 O G 3. O CD l r- o s CD L 1. Z - 1 l 1 I h s | HC I CO I n N | N I U.S. Patent Aug. 2, 2016 Sheet 6 of 6 US 9.408,076 B2 ELECTRONIC - DEVICE 600 MEMORY SUBSYSTEM NETWORKING SUBSYSTEM 612 614 OPERATING ANTENNA(S) SYSTEM 624 PROGRAM CONTROL INTERFACE MODULE LOGIC CIRCUIT 622 616 618 SENSOR PROCESSING DISPLAY SUBSYSTEM SUBSYSTEM SUBSYSTEM 630 610 626 FIG. 6 US 9,408,076 B2 1. 2 SENSOR-ASSISTED BOMETRIC (Note that photo attacks and video attacks are together AUTHENTCATION FOR SMARTPHONES referred to as 2D media attacks.) In another more sophisti cated facial-recognition technique, a 3D facial recognition CROSS-REFERENCE TO RELATED technique requires users to turn their heads towards four APPLICATION directions according to a sequence of arrows shown on a screen. In this way, an authentication program is able to This application claims priority under 35 U.S.C. S 119(e) to differentiate a real 3D face from a flat photo. However, the U.S. Provisional Application Ser. No. 61/993,017, entitled entire authentication process takes approximately 30 sec “Sensor-Assisted Facial Recognition: An Enhanced Biomet onds, which is too long and compromises ease of use (which ric Authentication System for Smartphones.” by Shaxun 10 is one of the important advantages of biometric authentica Chen, Amit Pande and Prasant Mohapatra, filed on May 14, tion). 2014, the contents of which are herein incorporated by refer Furthermore, the availability of virtual cameras has also CCC. limited the use of facial recognition. A virtual camera is a type of software that adds a layer between a real physical camera BACKGROUND 15 and an operating system. This type of software adds dynamic effects to the images from the physical camera, making the 1. Field video look more beautiful and live chat more interesting. The described embodiments relate to techniques for However, virtual cameras have now become sufficiently pow authenticating an individual. In particular, the described erful that they can not only modify the face, hair and back embodiments relate to techniques for authenticating an indi grounds, but also stream a pre-recorded video, making the vidual using facial images and/or sensor data from sensors in operating system believe it is captured by the physical camera an electronic device to prevent spoofing attempts. in real time. Consequently, in spite of their original purpose, 2. Related Art virtual camera software can seriously threaten the security of Portable electronic devices, such as Smartphones and tab facial recognition-based authentication. lets, are increasingly popular computing platforms. More 25 over, the ubiquity and enhanced functionality of portable SUMMARY electronic devices has resulted is an ever-increasing Suite of applications for use with portable electronic devices, includ The described embodiments relate to a portable electronic ing mobile-payment applications. Consequently, securing device. This portable electronic device includes: an imaging valuable information on portable electronic devices is 30 sensor (such as a front-facing camera) that acquires one or increasingly important. more images; a processor, and memory that stores a program A variety of security techniques are used on portable elec module that is executed by the processor. During execution of tronic devices, including authentication. During an authenti the program module, the portable electronic device provides cation process, whetheran individual is who they say they are an instruction to a user to move the portable electronic device is confirmed. For example, a user can be authenticated using 35 relative to a face of the user. Then, the portable electronic a credential. Such as a username?password or a passcode. device acquires, using the imaging sensor, a series of images In principle, biometric authentication offers advantages while the portable electronic device is at different positions over credential-based authentication. In particular, biometric relative to the face of the user, where the series of images authentication is considered to be more secure, because it is include at least two images relative to a symmetry axis of the based on who the user is and biometric information is typi 40 face of the user. Moreover, the portable electronic device cally difficult to forge or spoof. In contrast, credential-based determines a change in a sign of a nose angle in at least the two authentication relies on what the user knows, which can be images. Next, the portable electronic device authenticates the lost or stolen and is more likely to result in identity theft. In user based on at least one of the series of images using a addition, biometric authentication is usually much easier to facial-recognition technique. use. For example, users do not need to remember a list of 45 Furthermore, the portable electronic device may include a passwords for various websites and applications. While a motion sensor (such as an accelerometer, a gyroscope, an portable electronic device or a browser can remember the inertial motion sensor, a compass and/or a magnetometer) username and password for users, this often introduces addi that measures motion information. Prior to the authenticating, tional security threats, such as allowing other to access user the portable electronic device may associate the motion infor accounts ifa user leaves their portable electronic device unat 50 mation with a change in a position of an object in the series of tended. Because of these advantages, biometric authentica images. For example, the change in the position may be based tion (e.g., via fingerprint sensors) are widely used on portable on differences in pixels in the series of images. Note that the electronic devices. motion information may include vibrations and rotations of While the availability of high-resolution front-facing cam the portable electronic device. In some embodiments, the eras in many portable electronic devices can allow facial 55 associating involves correlating the motion information and recognition based on high quality images of a user's face to be the change in the position of the object.