Introduction to the Special Section on Real-World Face Recognition
Total Page:16
File Type:pdf, Size:1020Kb
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. 33, NO. 10, OCTOBER 2011 1921 Introduction to the Special Section on Real-World Face Recognition Gang Hua, Senior Member, IEEE,Ming-HsuanYang,Senior Member, IEEE, Erik Learned-Miller, Member, IEEE,YiMa,Senior Member, IEEE, Matthew Turk, Senior Member, IEEE,DavidJ.Kriegman,Senior Member, IEEE,and Thomas S. Huang, Fellow, IEEE Ç 1INTRODUCTION N recent years, there has been an emerging demand for on many real-world face recognition tasks (e.g., recognition Irobust face recognition algorithms that are able to deal of people known to the human viewer)? How can we learn with real-world face images. This is largely due to two a good model of a face from a small number of examples? factors. First, consumers have shown an increasing desire to How can we achieve the level of robustness exhibited by annotate or tag their digital photos to facilitate organization, human face recognition? These questions and new applica- access, and online sharing of their personal albums. tions for the technology promise to keep this area active for Industrial responses to this consumer desire can be the forseeable future. exemplifed by successful commercial face recognition Standard face recognition systems often start with a set systems included in applications and web sites such as of labeled gallery faces. When a new probe image is provided, Google Picasa, Windows Live Photo Gallery, Apple iPhoto, it is matched against the gallery faces to be recognized as a face.com, PolarRose, etc. Second, the growing applications known face or rejected. In a well-controlled setting, face in public security also call for robust face recognition images can be carefully captured for both gallery and probe technologies that can identify individuals from surveillance faces. In a moderately controlled setting, we may have cameras in uncontrolled situations. quality control over either the gallery faces or the probe In consumer digital imaging, face recognition must faces, but not both. In an uncontrolled setting, we lose contend with uncontrolled lighting, large pose variations, control of both. a range of facial expressions, make-up, changes in facial Face recognition in well-controlled settings has been hair, eyewear, weight gain, aging, and partial occlusions. extensively studied and is relatively mature. Earlier face Similarly, in scenarios such as visual surveillance, videos recognition methods often directly appled pattern recogni- are often acquired in uncontrolled situations or from tion and machine learning techniques on informative face moving cameras. These factors have been the focus of face features and are only effective when the probe and gallery recognition research for decades, but they are still not well images are frontal. More recently, to achieve higher resolved. recognition performance, many works have started to In addition to these market forces, face recognition also consider more precise geometric, shape, lighting, and represents an ongoing set of key scientific challenges. How reflectance models of faces. Notwithstanding the successes can we match or exceed the performance of humans on face of these techniques, there remains much room for improve- ment of face recognition in real-world scenarios since, . G. Hua is with the IBM T.J. Watson Research Center, Hawthorne, NY overall, much less attention has been paid to these less 10532. E-mail: [email protected]. controlled settings. M.-H. Yang is with the Department of Electrical Engineering and Two general misconceptions are that face recognition is a Computer Science, University of California, Merced, CA 95344. E-mail: [email protected]. solved problem and, on the other hand, that the uncon- . E. Learned-Miller is with the Computer Science Department, University of trolled scenarios are too difficult to address in practice. Massachusetts, Amherst, MA 01003. E-mail: [email protected]. Neither is true in our opinion. A primary purpose of this . Y. Ma is with Microsoft Research Asia, Beijing, China, and is on leave from special section is to deliver the following message to the the Electrical and Computer Engineering Department, University of Illinois at Urbana-Champaign, Urbana, IL 61801. community: Although significant progress has been made E-mail: [email protected]. in the last few decades, there remain plenty of challenges . M. Turk is with the Computer Science Department, University of and opportunities ahead. California, Santa Barbara, CA 93106. E-mail: [email protected]. D.J. Kriegman is with the Computer Science and Engineering Department, In the consumer digital imaging domain, practical face University of California, San Diego, La Jolla, CA 93093. annotation systems are emerging based on existing face rec- E-mail: [email protected]. ognition technologies. This brings human factors in the . T.S. Huang is with the Electrical and Computer Engineering Department, University of Illinois at Urbana-Champaign, Urbana, IL 61801. assistanted annotation system since good user interface (UI) E-mail: [email protected]. and user experience (UX) design are essential in order to For information on obtaining reprints of this article, please send e-mail to: compensate for possible failures of the face recognition [email protected]. algorithm. Last but not least, in many of these applications 0162-8828/11/$26.00 ß 2011 IEEE Published by the IEEE Computer Society 1922 IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. 33, NO. 10, OCTOBER 2011 there may be rich additional contextual information and the watch-list. The method attacks a real-world face meta-data that one can leverage to improve face recognition. recognition problem with an interesting and solid approach. Amultidisciplinaryexplorationmayberequiredto 3D face recognition has been regarded as a natural deliver real working systems. solution to pose variation. In “Using Facial Symmetry to Handle Pose Variations in Real-World 3D Face Recogni- tion,” Passalis et al. propose using facial symmetry to 2REVIEW PROCESS handle pose variation in 3D face recognition, while in The motivations for organizing this special section were to “Unconstrained Pose Invariant Face Recognition Using 3D better address the challenges of face recognition in real- Generic Elastic Models,” Prabhu et al. propose a generic 3D world scenarios, to promote systematic research and elastic model for pose invariant face recognition. Both are evaluation of promising methods and systems, to provide plausible approaches for using 3D information to assist in a snapshot of where we are in this domain, and to stimulate face recognition under large pose variations. While histori- discussion about future directions. We solicited original cally 3D face recognition has been criticized for lack of real- contributions of research on all aspects of real-world face world 3D sensory cameras, this issue may be resolved in the recognition, including: future with inexpensive 3D sensors as evidenced by the PrimeSense sensor used in the Xbox Kinect from Microsoft. the design of robust face similarity features and The other three accepted papers all deal with face metrics, recognition in photos and images in the wild. In “Describable . robust face clustering and sorting algorithms, Visual Attributes for Face Verification and Image Search,” . novel user interaction models and face recognition algorithms for face tagging, Kumar et al. present a face verification algorithm based on a . novel applications of web face recognition, representation that uses a set of describable attributes. Their . novel computational paradigms for face recognition, classifier achieves very good results on two publicly available . challenges in large scale face recognition tasks, e.g., benchmarks, namely Labeled Faces in the Wild (LFW) and the on the Internet, Public Figures (PubFig) data set. Since their method was first . face recognition with contextual information, published at ICCV ’09, there has been a lot of work using . face recognition benchmarks and evaluation metho- attribute-based representations for various visual recognition dology for moderately controlled or uncontrolled problems beyond face recognition. envi-ronments, and In “Effective Unconstrained Face Recognition by Com- . video face recognition. bining Multiple Descriptors and Learned Background We received 42 original submissions, four of which were Statistics,” Wolf et al. propose an approach for face rejected without review; the other 38 papers entered the verification in the wild that combines multiple descriptors normal review process. Each paper was reviewed by three with learned statistics from background context. Its face reviewers who are experts in their respective topics. More verification accuracy ranked first on the LFW benchmark. In than 100 expert reviewers have been involved in the review “Scalable Face Image Retrieval with Identity-Based Quanti- process. zation and Multireference Reranking,” Wu et al. present a The papers were equally distributed among the guest scalable face image retrieval system using identity-based editors. A final decision for each paper was made by at least quantization to build a visual representation and multiple two guest editors assigned to it. To avoid conflict of interest, references for reranking. It builds a good foundation to no guest editor submitted any papers to this special section. tackle the problem of searching for face images over the internet image corpus. The face recognition research community has built