TALLINN UNIVERSITY OF TECHNOLOGY SCHOOL OF ENGINEERING Department of Electrical power Engineering and Mechatronics FACE IDENTIFICATION USING CAPSULE NETWORK WITH SMALL DATA SET NÄOTUVASTUS VÄIKESE ANDMEHULGAGA KAPSELVÕRGU ABIL MASTER THESIS Üliõpilane: Ahmed Amin Osman Üliõpilaskood: 165596 MAHM Juhendaja: Saleh Ragheb Saleh Alsaleh, Eng, Tallinn 2020 AUTHOR’S DECLARATION Hereby I declare, that I have written this thesis independently. No academic degree has been applied for based on this material. All works, major viewpoints and data of the other authors used in this thesis have been referenced. “.......” .................... 20….. Author: .............................. /signature / Thesis is in accordance with terms and requirements “.......” .................... 20…. Supervisor: …......................... /signature/ Accepted for defence “.......”....................20… . Chairman of theses defence commission: ................................................. /name and signature/ 2 Non-exclusive Licence for Publication and Reproduction of GraduationTthesis¹ I, Ahmed Amin Osman (12/20/1992 ) hereby 1. grant Tallinn University of Technology (TalTech) a non-exclusive license for my thesis FACE IDENTIFICATION USING CAPSULE NETWORK WITH SMALL DATA-SET supervised by Prof.Mart Tamre and Eng, Saleh Ragheb Saleh Alsaleh ____________________________________________________________, 1.1 reproduced for the purposes of preservation and electronic publication, incl. to be entered in the digital collection of TalTech library until the expiry of the term of copyright; 1.2 published via the web of TalTech, incl. to be entered in the digital collection of TalTech library until the expiry of the term of copyright. 1.3 I am aware that the author also retains the rights specified in clause 1 of this license. 2. I confirm that granting the non-exclusive license does not infringe third persons' intellectual property rights, the rights arising from the Personal Data Protection Act or rights arising from other legislation. ¹ Non-exclusive Licence for Publication and Reproduction of Graduation Thesis is not valid during the validity period of restriction on access, except the university`s right to reproduce the thesis only for preservation purposes. ______________ (signature) ______________ (date) 3 Department of Electrical Power Engineering and Mechatronics FACE IDENTIFICATION USING CAPSULE NETWORK WITH SMALL DATA-SET Student: Ahmed Amin Osman, 165596 Study programme, Mechatronics Supervisor(s): Eng, Saleh Ragheb Saleh Alsaleh, +372 5875 8076 (in English) Face Identification Using Capsule Network With Small Data-Set (in Estonian) NÄOTUVASTUS VÄIKESE ANDMEHULGAGA KAPSELVÕRGU ABIL Thesis main objectives: 1. Exploring the potential of Capsule Network on Face Identification 2. Comparative analysis against proven models and networks Thesis tasks and time schedule: No Task description Deadline 1. Literature Review 11/25/2019 2. Building the Capsule model 11/27/2019 3. Building The CNNs Model and Face-Net Model 11/30/2019 4. Writing 1/2/2020 Language: English Deadline for submission of thesis: 1/3/2020 Student: Ahmed Amin Osman .......……........ “........”………….....................2020 /signature/ Supervisor: Eng, Saleh Ragheb Saleh Alsaleh…“.................”.........................2020 /signature/ Head of the study programme: Prof.Mart Tamre ……… “.......”.............2020 /signature/ Terms of thesis closed defence and/or restricted access conditions to be formulated on the reverse side 4 CONTENTS PREFACE .........................................................................................................................7 List of abbreviations and symbols ......................................................................................8 INTRODUCTION .............................................................................................................9 Problem Statement ........................................................................................................... 11 1 LITERATURE REVIEW ......................................................................................... 12 1.1 Human vision..................................................................................................... 12 1.2 Computer vision ................................................................................................ 12 1.3 Face detection .................................................................................................... 13 1.3.1Robust real-time object detection ............................................................................. 13 2 DATA-SET AND LIBRARIES AND HARDWARE ............................................... 15 2.1 Platforms and libraries ....................................................................................... 16 2.1.1Hardware and Software ............................................................................................ 16 2.2 Data-set pre-processing for CAPS-Net ............................................................... 17 3 CAPS-NET .............................................................................................................. 18 3.1 Capsule neural network ...................................................................................... 18 3.1.1Applications ............................................................................................................. 18 3.2 Theory ............................................................................................................... 19 3.3 The intuition behind CapsNet ............................................................................. 21 3.3.1Primary capsules ...................................................................................................... 21 3.3.2Higher layer capsules ............................................................................................... 23 3.3.3Caps-face ................................................................................................................. 25 3.3.4Activation for CNNs layer ........................................................................................ 26 3.3.5ReLU activation function ......................................................................................... 27 3.3.6Loading data set ....................................................................................................... 28 3.4 Primary capsule layer ......................................................................................... 29 3.5 Face capsule ...................................................................................................... 31 3.6 Routing by agreement ........................................................................................ 31 3.7 Results ............................................................................................................... 32 3.7.1Routing iteration (1) ................................................................................................. 32 3.7.2Routing iteration (2) ................................................................................................. 34 3.8 Experimenting with larger data-set ..................................................................... 34 4 CONVOLUTIONAL NEURAL NETWORK ........................................................... 38 5 4.1 Theory ............................................................................................................... 38 4.2 Architecture ....................................................................................................... 40 4.3 Results ............................................................................................................... 41 5 FACE-NET .............................................................................................................. 42 5.1 Theory ............................................................................................................... 42 5.1.1Triplet loss ............................................................................................................... 42 5.1.2Triplet selection........................................................................................................ 43 5.2 Architecture ....................................................................................................... 43 5.2.1Support vector machines .......................................................................................... 43 5.2.2Face vector calculation ............................................................................................. 44 5.3 Results ............................................................................................................... 45 6 ANALYSIS .............................................................................................................. 47 SUMMARY .................................................................................................................... 49 LIST OF REFERENCES ................................................................................................. 53 APPENDICES ................................................................................................................. 56 6 PREFACE The Face identification using Capsul networks work was done at the Robotics Control Laboratory of the mechatronics and autonomous systems centre at the school of engineering at Tallinn university of technology in Tallinn, Estonia. The research had two academic supervisors, Researcher Saleh Alsaleh