Graduate Theses, Dissertations, and Problem Reports 2018 Machine Learning Approaches to Human Body Shape Analysis Marco Piccirilli Follow this and additional works at: https://researchrepository.wvu.edu/etd Recommended Citation Piccirilli, Marco, "Machine Learning Approaches to Human Body Shape Analysis" (2018). Graduate Theses, Dissertations, and Problem Reports. 6417. https://researchrepository.wvu.edu/etd/6417 This Dissertation is protected by copyright and/or related rights. It has been brought to you by the The Research Repository @ WVU with permission from the rights-holder(s). You are free to use this Dissertation in any way that is permitted by the copyright and related rights legislation that applies to your use. For other uses you must obtain permission from the rights-holder(s) directly, unless additional rights are indicated by a Creative Commons license in the record and/ or on the work itself. This Dissertation has been accepted for inclusion in WVU Graduate Theses, Dissertations, and Problem Reports collection by an authorized administrator of The Research Repository @ WVU. For more information, please contact [email protected]. MACHINE LEARNING APPROACHES TO HUMAN BODY SHAPE ANALYSIS Marco Piccirilli Thesis submitted to the Benjamin M. Statler College of Engineering and Mineral Resources at West Virginia University in partial fulfillment of the requirements for the degree of Doctor of Philosophy in Electrical Engineering Donald Adjeroh, Ph.D., Committee Chairperson Gianfranco Doretto, Ph.D., Co-Chair Arun Ross, Ph.D. Bojan Cukic, Ph.D. Natalia Schmid, Ph.D. Peter Giacobbi, Ph.D. Xin Li, Ph.D. Lane Department of Computer Science and Electrical Engineering Morgantown, West Virginia 2018 Keywords:Computer Vision, Machine Learning, Computational Geometry, Soft Biometrics, Human Body Copyright c 2018 Marco Piccirilli ABSTRACT Machine Learning Approaches to Human Body Shape Analysis Marco Piccirilli Soft biometrics, biomedical sciences, and many other fields of study pay particular attention to the study of the geometric description of the human body, and its variations. Although multiple contributions, the interest is particularly high given the non-rigid nature of the human body, capable of assuming different poses, and numerous shapes due to variable body composition. Unfortunately, a well-known costly requirement in data-driven machine learning, and particu- larly in human-based analysis, is the availability of data, in the form of geometric information (body measurements) with related vision information (natural images, 3D mesh, etc.). We in- troduce a computer graphics framework able to generate thousands of synthetic human body meshes, representing a population of individuals with stratified information: gender, Body Fat Percentage (BFP), anthropometric measurements, and pose. This contribution permits an ex- tensive analysis of different bodies in different poses, avoiding the demanding, and expensive acquisition process. We design a virtual environment able to take advantage of the generated bodies, to infer the body surface area (BSA) from a single view. The framework permits to simulate the acquisition process of newly introduced RGB-D devices disentangling different noise components (sensor noise, optical distortion, body part occlusions). Common geometric descriptors in soft biometric, as well as in biomedical sciences, are based on body measure- ments. Unfortunately, as we prove, these descriptors are not pose invariant, constraining the usability in controlled scenarios. We introduce a differential geometry approach assuming body pose variations as isometric transformations of the body surface, and body composition changes covariant to the body surface area. This setting permits the use of the Laplace-Beltrami opera- tor on the 2D body manifold, describing the body with a compact, efficient, and pose invariant representation. We design a neural network architecture able to infer important body seman- tics from spectral descriptors, closing the gap between abstract spectral features, and traditional measurement-based indices. Studying the manifold of body shapes, we propose an innovative generative adversarial model able to learn the body shapes. The method permits to generate new bodies with unseen geometries as a walk on the latent space, constituting a significant advantage over traditional generative methods. “Studere studere....., post mortem quid valere?” (cit. Mautilio) “Memento Audere Semper!” (cit. D’Annunzio) iii Acknowledgments I would like to express my immense gratitude to my parents for their continuous support, and the motivation to look always ahead, with passion and determination. My sincere thanks go to my advisors for their patience and the long years of research. Finally, I thank my fellow labmates for the long discussions and the great camaraderie. iv Contents Abstract ii Acknowledgments iv List of Figures xi List of Tables xvi 1 Introduction 1 1.1 Human Body Shape Analysis: A Vision-Driven Approach . 1 1.2 Human Body Shape Analysis: A Soft-Biometrics Viewpoint . 2 1.2.1 Related Work in Soft-Biometrics . 4 1.3 Human Body Shape Analysis: A Medical Science Viewpoint . 5 1.3.1 Body Mass Index . 6 1.3.2 Body Surface Area . 7 1.3.3 Body Fat Percentage (BFP) . 8 1.4 Human Body Shape Analysis, A New Approach . 10 2 VirtualBody: A Virtual Dataset for Body Shape Analysis 14 2.1 Introduction . 14 v 2.2 Shape Semantics . 15 2.3 Related Work . 17 2.3.1 Datasets . 17 2.3.2 Models . 19 2.3.3 Methods . 21 2.4 3D Body Model and Virtual Body Framework . 23 2.4.1 Generation of Virtual (Synthetic) Humans . 24 2.5 Results: Virtual dataset . 27 3 Whole Body Surface Area Estimation 31 3.1 Introduction . 31 3.1.1 WBSA: Measurements and Estimation . 32 3.1.2 The Problem . 35 3.1.3 Virtual Environment . 36 3.2 Methods . 37 3.2.1 Dataset . 38 3.2.2 Virtual Camera . 38 3.2.3 Whole Body Surface Area from a Single View . 42 3.2.4 WBSA Prediction . 44 3.3 Results . 48 3.3.1 WBSA Prediction . 48 3.3.2 Linear Regression Analysis . 49 3.3.3 Impact of Azimuth and Elevation on Computed WBSA . 50 3.3.4 Regression with Stature . 52 3.3.5 Regression with Grouping . 52 vi 3.4 Discussion . 53 3.4.1 Frontal VBSA Vs Rear VBSA . 53 3.4.2 Non-Linearity in the WBSA-VBSA Relationship . 54 3.4.3 Evaluating WBSA Measurements . 55 3.4.4 Reconstruction . 55 4 3D Body Shape Analysis 65 4.1 Shape Analysis in Computer Vision . 65 4.2 Spectral Analysis . 68 4.2.1 Generic 3D Shape Retrieval techniques . 74 4.3 Human Body Shape: A Spectral Geometry Approach . 76 4.3.1 Challenges in non-rigid shape analysis and Spectral Analysis. 76 4.4 WBSA and the Spectrum . 78 4.4.1 Weyl’s Law on the asymptotic behavior of the eigenvalues. 78 4.4.2 LB Spectra of Subdomains . 80 4.4.3 Extension to Body Parts . 82 4.4.4 Weyl proof for the 2D rectangular interval case . 83 4.5 Body Fat Percentage using Spectral Analysis . 84 4.5.1 Problem Definition . 85 4.5.2 Proposed method . 86 4.5.3 Interaction between BFP and Body Weight. 87 4.5.4 Bag of Features Approach . 95 4.6 Results . 99 4.6.1 Dataset Preparation . 99 4.6.2 siHKS Features . 99 vii 4.6.3 Training . 100 4.6.4 Performance . 101 4.7 Conclusion and Future Work . 102 5 Pose Invariant Soft Biometrics 104 5.1 Background and Literature Review . 107 5.1.1 Anthropometric Features From the Body . 108 5.1.2 Anthropometric Datasets . 111 5.1.3 Main Contributions . 112 5.2 Variability of Anthropometric Measurements under Pose Transformations . 113 5.3 Spectral Geometry Approach to Soft Biometrics . 118 5.3.1 Spectral and Anthropometric Matching . 121 5.3.2 Soft Biometrics from Spectral Features . 122 5.4 Results . 125 5.4.1 Datasets . 126 5.4.2 Anthropometric Measurements – Impact of Pose . 128 5.4.3 Spectral Features for Soft Biometrics . 133 5.4.4 Predicting Semantic Features . 135 5.5 Conclusion . 138 6 Exploring the Human Body Manifold 140 6.1 Representation Learning: The Manifold Hypothesis . 141 6.2 Human Body Manifold Learning . 143 6.2.1 Human Body Manifold . 143 6.3 A Generative Model Approach for Human Body Semantics . 145 6.3.1 Generative Models . 146 viii 6.3.2 The DCGAN Architecture . 151 6.4 Method: Creating New Body Shapes . 152 6.4.1 Latent Space Z . 153 6.4.2 Evaluation Network . 157 6.5 Results . 158 6.6 Conclusion . 164 7 Conclusion and Future work 166 7.1 Conclusion . 166 7.2 Future Work . 168 7.2.1 Spectral Geometry/3D based Geometric Processing . 169 7.2.2 2D Computer Vision . 169 Bibliography 170 A Multi-views Body Fat Percentage 1 A.1 Introduction . 1 A.2 Problem Definition . 3 A.3 Dataset . 5 A.4 A Renderer for the VirtualBody Dataset . 5 A.4.1 Rendering . 6 A.4.2 Data Augmentation and Jittering . 7 A.5 Network Models . ..