A Study on an Automatic System for Analyzing the Facial Beauty of Young Women Neha Sultan A Thesis in The Department of Computer Science and Software Engineering Presented in Partial Fulfillment of the Requirements For the Degree of Master of Computer Science at Concordia University Montreal, Quebec, Canada January 2014 © 2014 Neha Sultan CONCORDIA UNIVERSITY School of Graduate Studies This is to certify that the thesis prepared By: Neha Sultan Entitled: A Study on an Automatic System for Analyzing the Facial Beauty of Young Women and submitted in partial fulfillment of the requirements for the degree of MASTER OF COMPUTER SCIENCE complies with the regulations of the University and meets the accepted standards with respect to originality and quality. Signed by the final Examining Committee: ________________________________ Chair Dr. Jayakumar Rajagopalan ________________________________ Examiner Dr. Tonis Kasvand ________________________________ Examiner Dr. Yuhong Yan ________________________________ Thesis Supervisor Dr. Ching Y. Suen Approved by ___________________________________________________ Chair of Department or Graduate Program Director ___________________________________________________ Dr. Christopher Trueman, Dean, Faculty of Engineering and Computer Science Date ____________________________________________________ ACKNOWLEDGEMENTS My supervisor, Dr. Ching Y. Suen has provided me with constant guidance and invaluable feedback. I would also like to extend my gratitude to the Centre for Pattern Recognition and Machine Intelligence (CENPARMI) at Concordia University for providing the perfect environment for enabling innovation and excellence in research. I also thank the Natural Sciences and Engineering Research Council for the support associated with this research. In addition, the Faculty of Engineering and Computer Science at Concordia University has been a great source of invaluable knowledge and support for which I am grateful. I would like to thank my family for always being there for me and for all their support. DEDICATION This thesis is dedicated to my parents, who encourage and support me constantly and due to their guidance I have been able to achieve my goals. ABSTRACT A Study on an Automatic System for Analyzing the Facial Beauty of Young Women Neha Sultan Beauty is one of the foremost ideas that define human personality. However, only recently has the concept of beauty been scientifically analyzed. This has mostly been due to extensive research done in the area of face recognition and image processing on identification and classification of human features as contributing to facial beauty. Current research aims at precisely and conclusively understanding how humans classify a given individual's face as beautiful. Due to the lack of published theoretical standards and ground truths for human facial beauty, this is often an ambiguous process. Current methods of analysis and classification of human facial beauty rely mainly on the geometric aspects of human facial beauty. The classifiers used in current research include the k-nearest neighbor algorithm, ridge regression, and basic principal component analysis. In this research, various approaches related to the comprehension and analysis of human beauty are presented and the use of these theories is outlined. Each set of theories is translated into a feature model that is tested for classification. Selecting the best set of features that result in the most accurate model for the representation of the human face is a key challenge. This research introduces the combined use of three main groups of features for classification of female facial beauty, to be used with classification through support vector machines. The classifier utilized is Support Vector Machine (SVM) and the accuracy obtained through this classifier is 86%. Current research in the field has iii produced algorithms with percentages of accuracy that are in the range of 75-85%. The approach used is one of analysis of the central tenets of beauty, the successive application of image processing techniques, and finally the usage of relevant machine learning methods to build an effective system for the automatic assessment of facial beauty. The ground truths used for verifying results are derived from ratings extracted from surveys conducted. The proposed methodology involves a novel algorithm for the representation of facial beauty, which combines the use of geometric, textural, and shape based features for the analysis of facial beauty. This algorithm initially develops an overall landmark model of the entire human face. A significant advantage of this methodology is the accurate model of the human face which synthesizes the geometric, textural and shape-related aspects of the face. The landmark model is then used for extracting critical characteristics which are then used in a feature vector for training using machine learning. The features extracted help to represent facial characteristics in three major areas. Geometric features help to represent the symmetrical properties and ratio-based properties of landmarks on the face. Textural features extracted help capture information related to skin texture and composition. Finally, face shape and outline features help to categorize the overall shape of a given face, which helps to represent the given female face shape and outline for further analysis of any deviations from the basic face shapes. These features are then used in a classifier to appropriately categorize each image. The database used for the source of images contains images of female subjects from a variety of backgrounds and levels of attractiveness. iv TABLE OF CONTENTS LIST OF FIGURES .................................................................................................................. XI LIST OF TABLES .......................................................................................................... xiv CHAPTER 1 INTRODUCTION ............................................................................................ 1 1.1 Related Fields of Research ......................................................................................... 2 1.1.1 Facial Expressions and Emotions .......................................................................... 3 1.1.2 Composite Faces .................................................................................................... 5 1.1.3 Gender Classification ............................................................................................. 8 1.1.4 Facial Beauty Analysis ......................................................................................... 10 1.2 Problem Description ................................................................................................. 13 1.3 Main Contributions .................................................................................................. 14 1.4 Applications of Automatic Beauty Analysis ............................................................ 16 CHAPTER 2 STANDARDS AND THEORIES ASSOCIATED WITH FACIAL BEAUTY ............................................................................ 17 2.1 Beauty Throughout the Ages .................................................................................... 17 2.1.1 Pythagoras's Contributions .................................................................................. 18 2.1.2 The Golden Ratio ................................................................................................. 19 2.1.3 Plato ..................................................................................................................... 21 2.1.4 Aristotle ................................................................................................................ 22 2.2 Modern Philosophy ................................................................................................... 22 2.3 Evolution and Beauty ............................................................................................... 23 2.4 Objectivity of Facial Beauty ..................................................................................... 24 v 2.5 Theories Related to Facial Beauty ........................................................................... 26 2.5.1 Averageness .......................................................................................................... 26 2.5.2 Symmetry ............................................................................................................. 27 2.5.3 Skin Texture ......................................................................................................... 27 2.5.4 Geometric Facial Features ................................................................................... 28 2.5.5 Golden Ratio ........................................................................................................ 28 2.5.6 Facial Thirds ........................................................................................................ 29 CHAPTER 3 DATA ANALYSIS AND COLLECTION .................................................. 30 3.1 Databases Utilized in this Research ......................................................................... 33 3.1.1 AR Face Database ................................................................................................ 34 3.1.2 Psychological Image Collection at Stirling ......................................................... 36 3.1.2.1 Stirling Faces Database ................................................................................