Modeling Prosopagnosia: Computational Theory and Experimental Investigations of a Deficit in Face Recognition Von der Fakult¨atf¨urMathematik und Informatik der Universit¨atLeipzig angenommene DISSERTATION zur Erlangung des akademischen Grades DOCTOR RERUM NATURALIUM (Dr.rer.nat.) im Fachgebiet Informatik vorgelegt von Diplommathematiker Rainer Stollhoff geboren am 24.3.1980 in Stuttgart Die Annahme der Dissertation haben empfohlen: 1. Professor Dr. J¨urgenJost (MPIMN Leipzig) 2. Professor Dr. Christoph von der Malsburg (FIAS Frankfurt) Die Verleihung des akademischen Grades erfolgt mit Bestehen der Verteidigung am 23.02.2010 mit dem Gesamtpr¨adikat magna cum laude. Contents 1. Introduction1 1.1. Prosopagnosia..................................3 1.1.1. Characteristic Behavioral Differences in Face Recognition.....4 1.1.2. Congenital Predispositions......................5 1.1.3. A Deficit in the Processing of Facial Identity............5 1.2. Computer Vision Systems and Computational Models of High-level Vision7 1.3. Outline.....................................9 2. Computational Theory of Classification and Identification 11 2.1. Observation Process.............................. 13 2.1.1. Physical Description of Objects.................... 13 2.1.2. Sensory Encoding of Observations.................. 15 2.2. Identification and Classification Tasks.................... 17 2.2.1. Bayesian Inference........................... 18 2.3. Information Processing and Compression................... 21 2.3.1. Curse and Blessings of Dimensionality................ 25 2.3.2. Example: Dimensionality of Sensory Features............ 27 2.4. Object Transformations............................ 36 2.4.1. Transformation Invariant Features.................. 37 2.4.2. Alignment................................ 38 2.4.3. Noise.................................. 40 2.5. Discussion and Application to Human Face Recognition.......... 41 2.5.1. Summary................................ 41 2.5.2. Differences between Face and Object Recognition.......... 43 2.5.3. Experimental Investigations of Face Recognition.......... 44 3. Modeling Facial Encoding in Congenital Prosopagnosia 47 3.1. Feedforward Network Representations.................... 49 3.1.1. Unsupervised Representation..................... 50 3.1.2. Supervised Classification........................ 52 3.2. Sparseness Constraints in Face Encoding................... 54 3.2.1. Materials and Methods........................ 56 3.2.2. Results................................. 59 3.3. Prosopagnosic Processing of Sparse Representations............ 63 3.3.1. Materials and Methods........................ 64 3.3.2. Results................................. 70 II Contents 3.4. Discussion.................................... 74 3.4.1. Summary................................ 74 3.4.2. Converging Methods Approach.................... 76 3.4.3. Encoding in a Hierarchically Structured Network.......... 76 4. Detailed Investigation of Face Recognition Deficits in Congenital Prosopag- nosia 79 4.1. Method & Materials.............................. 83 4.1.1. Participants............................... 83 4.1.2. Screening Questionnaire........................ 85 4.1.3. Object Recognition Tests....................... 85 4.1.4. Bielefeld Famous Face Test...................... 85 4.1.5. Object and Face Recognition Test Battery.............. 86 4.1.6. Longterm Memory Test........................ 90 4.1.7. Fixed Presentation Time Test..................... 91 4.1.8. Presentation.............................. 94 4.1.9. Statistical Analysis........................... 94 4.2. Results...................................... 97 4.2.1. Object Recognition Tests....................... 97 4.2.2. Bielefeld Famous Face Test...................... 97 4.2.3. Object and Face Recognition Test Battery.............. 98 4.2.4. Longterm Memory Test........................ 103 4.2.5. Fixed Presentation Time Test..................... 104 4.2.6. Variability of Performance across Tests................ 106 4.3. Discussion.................................... 111 4.3.1. Summary................................ 111 4.3.2. Heterogeneity in Congenital Prosopagnosia............. 112 5. Conclusion 115 5.1. Summary and Discussion of the Results................... 115 5.1.1. Individuation Agnosia......................... 116 5.1.2. Face Processing in the Williams Syndrome............. 117 5.1.3. Classification of Congenital Prosopagnosia into Subtypes..... 118 5.1.4. Constraints on the Development of a Neural Face Processing System119 5.2. Implications for Computational Vision.................... 120 5.3. Outlook..................................... 121 A. Face Recognition and Prosopagnosia 123 A.1. Face Recognition................................ 124 A.1.1. Models of Face Recognition...................... 125 A.1.2. Stimulus Transformations....................... 133 A.1.3. A Distributed Neural System of Face Recognition.......... 134 A.1.4. Development of Face Recognition................... 135 A.1.5. Are Faces Special?........................... 137 Contents III A.2. Acquired Prosopagnosia............................ 138 A.2.1. Neuroanatomy of Acquired Prosopagnosia.............. 139 A.2.2. Diagnostic Tests............................ 141 A.2.3. Models of Acquired Prosopagnosia.................. 141 A.3. Congenital Prosopagnosia........................... 142 A.3.1. Manifestation.............................. 143 B. Mathematical Derivations 147 B.1. Compression Lemma.............................. 147 B.2. Mututal Information in Binary Identification Tasks............. 148 B.2.1. Noiseless Case, iid........................... 148 B.2.2. Noiseless Case, best and worse.................... 149 B.3. Mutual Information in Binary Classification Tasks............. 151 B.3.1. Minimizing Description Length.................... 151 B.3.2. Noiseless Case, non-overlapping.................... 151 C. Eye-distance under Rotation in Depth 153 D. Model Based Comparisons using Generalized Linear Mixed Models 157 E. Screening Questionnaire 161 1. Introduction Face recognition1 is crucial to a wide range of social behavior, and suddenly loosing all or part of one's face recognition abilities is confusing at the least. In a classical study, Bodamer(1947) reports of a patient who lost the ability to identify persons by looking at their face after having suffered a closed head injury. The deficit occurred not only in the identification of persons encountered after the injury (e.g. personnel at the hospital) but also in the identification of members of the family and the self-identification in a mirror. The patient reported being blind to the specific information contained in faces \No, the face doesn't tell me anything, faces don't tell me anything at all, they all look the same to me."2 In this and other cases of acquired prosopagnosia, colloquially also referred to as \face- blindness", the cause of the deficit is easily identifiable: A closed head injury and the resulting damage inflicted upon specific cortical ares. However, in recent times more and more cases of prosopagnosia have been reported where the deficit was not acquired due to an accident, but presumably present from birth, i.e. congenital3 (see e.g. McConachie, 1976; Ariel and Sadeh, 1996; Barton et al., 2001; Kress and Daum, 2003a; Behrmann and Avidan, 2005; Kennerknecht et al., 2006; Gr¨uteret al., 2007). Recent research on this congenital form of prosopagnosia has provided an estimate of the prevalence of this deficit at 2.5% (Kennerknecht et al., 2006) and persons concerned include celebrities like Jane Goodall or Oliver Sacks (Gr¨uter, 2007). In congenital prosopagnosia, the deficit can be equally profound as in the acquired form and equally selective such that only facial identification is impaired while all other aspects of face and object recognition remain intact (Duchaine, 2006). The study of prosopagnosia is contributing greatly to our understanding of human face recognition. Establishing correspondences between the location of damaged regions and the exact nature of the deficits in acquired prosopagnosia has contributed to the 1In the following the term face recognition will always be used in the general sense of extracting information from face images. If the exact type of information extracted during the process of face recognition is important it will be specified explicitly (e.g. facial identification or gender recognition) whenever this is not clear from the context. 2Nein, das Gesicht sagt mir nichts, Gesichter sagen mir ¨uberhaupt nichts, sie sehen alle f¨urmich gleich aus. [own translation] 3In this work, congenital prosopagnosia will be used to demarcate the difference between acquired prosopagnosia, characterized by a shift from functional to non-functional face processing, and non- acquired prosopagnosia where processing never reached a functional stage in the first place. Other terms for the latter non-acquired form of prosopagnosia that have been used in the literature include developmental prosopagnosia and hereditary prosopagnosia. 2 Introduction localization of subsystems of face processing (see e.g. Damasio et al., 1990). For example, it has been shown that, under normal conditions of maturation and development of the brain, faces and objects are processed in different parts of the inferotemporal cortex (Kanwisher et al., 1997; Gauthier et al., 2000a; Hoffman and Haxby, 2000), which are possibly integrated into distinct cortical systems for face vs. object recognition (Haxby et al., 2000). The spatial separation of cortical systems