View metadata, citation and similar papers at core.ac.uk brought to you by CORE provided by Okayama University Scientific Achievement Repository Engineering Industrial & Management Engineering fields Okayama University Year 2004 Robust face recognition by combining projection-based image correction and decomposed eigenface Takeshi Shakunaga Fumihiko Sakaue Okayama University Okayama University Kazuma Shigenari Okayama University This paper is posted at eScholarship@OUDIR : Okayama University Digital Information Repository. http://escholarship.lib.okayama-u.ac.jp/industrial engineering/24 Robust Face Recognition by Combining Projection-Based Image Correction and Decomposed Eigenface Takeshi Shakunaga Fumihiko Sakaue Kazuma Shigenari Department of Information Technology, Faculty of Engineering Okayama University Okayama-shi, Okayama 700-8530, Japan E-mail: {shaku,sakaue}@chino.it.okayama-u.ac.jp Abstract lected in the registration stage, the EF can be constructed by Principal Component Analysis (PCA). However, an EF for This paper presents a robust face recognition method which an individual cannot be stably constructed when an insuf- can work even when an insufficient number of images are ficient number of sample images are available or when the registered for each person. The method is composed of im- sample images have been taken under very similar light- age correction and image decomposition, both of which are ing conditions. In these situations, realizing illumination- specified in the normalized image space (NIS). The image insensitive identification requires some refinements of the correction[1, 2] is realized by iterative projections of an eigenface method. In our previous paper [3], we analyzed image to an eigenspace in NIS. It works well for natural the eigenface approach and proposed concepts of virtual images having various kinds of noise, including shadows, eigenfaces and the decomposed eigenface for the forego- reflections, and occlusions. We have proposed decompo- ing purpose. In this paper, we combine these concepts with sition of an eigenface into two orthogonal eigenspaces[3], a projection-based image correction method [2] in order to and have shown that the decomposition is effective for re- refine the face recognition methodology. alizing robust face recognition under various lighting con- ditions. This paper shows that the decomposed eigenface 2. Normalized Image Space and Nor- method can be refined by projection-based image correc- tion. malized Eigenspace 2.1. Normalized Image Space 1. Introduction In object recognition, eigenspaces are often constructed in A human face changes in appearance with lighting condi- a least-squares sense, faithful to the original images [4, 6]. tions, and difficulty is encountered in controlling lighting Eigenspaces are also effective for image-based rendering conditions in natural environments where face images are under changing lighting conditions [7, 8]. They can also be taken. These facts suggest that robust face recognition re- constructed from original images, by use of the photomet- quires construction of a face recognition algorithm that is ric SVD (Singular Value Decomposition) algorithm [9, 10]. insensitive to lighting conditions. Meanwhile, appearance- These methods are commonly discussed in the original im- based face recognition can be resolved into the eigenface age space. method [4], which in many cases is identical with the sub- Although the original image space is effective for some space method [5, 6]. Eigenfaces are widely used for both purposes, it often fails to work when illumination falls out personal identification and detection of (unknown) faces in of range. In such a case, we empirically utilize some types an image. When intended for detection of faces in an image, of image normalization. eigenfaces are constructed from many persons and when In this paper, normalization is based on L1-norm. Let an intended for personal identification, each eigenface should N-dimensional vector X denote an image whose elements be constructed from face images of the individual. In the are all non-negative, and let 1 denote an N-dimensional present paper we focus on the second purpose, and as used vector whose elements are all 1. The normalized image x of herein an eigenface (EF) always means an eigenspace con- an original image X is defined as x = X/(1T X). After the structed from face images of the individual for the purpose normalization, x is normalized in the sense that 1T x =1. of personal identification. If many face images can be col- By this normalization, any nonzero image X(= 0) is Proceedings of the Sixth IEEE International Conference on Automatic Face and Gesture Recognition (FGR’04) 0-7695-2122-3/04 $ 20.00 © 2004 IEEE mapped to a point in the Normalized Image Space (NIS). Let n and y denote the normalized images of image N The NIS is closed to any averaging operation, and the real and Y = X + N, where X is a signal, N is a noise and Y is variance is encoded up to a scale factor. an input image. Let us define y =(1− α)x + αn, where α = 1T N/1T Y. Then the projection, y∗, and the residual, 2.2. Normalized Eigenspace y, are respectively represented by When an image class is given, a k-dimensional eigenspace ∗ ∗ T y =(1− α)x + αΦk (n − x) (3) is constructed in NIS by the conventional PCA from the mean vector x and covariance matrix Σ T y = αn = α(n − x) − αΦkΦk (n − x). (4) K K 1 1 T x = xj Σ= (xj − x)(xj − x) , In Eq. (4), the first term indicates the existence of n it- K and K j=1 j=1 self. The second term indicates that the noise affects the T whole image with weight −αΦkΦk .Evenifn is very where K is the number of images in the class. localized, the effect spreads to the whole image. Since Let Λ denote a diagonal matrix in which diagonal terms T ΦkΦk is positive semidefinite, the second term yields a are eigenvalues of Σ in descending order, and Φ a matrix counteraction to the noise. A negative reaction is generated in which the i-th column is the i-th eigenvector of Σ. Then T from a positive noise, whereas a positive reaction is gener- PCA implies Λ=Φ ΣΦ. Using a submatrix Φk of Φ, ated from a negative noise. Refer to [1] on the estimations which consists of k principal eigenvectors, the projection, ∗ ∗ of α and x as well as a detection of noise region in image x ,ofx onto the eigenspace and the residual, x , of the correction. projection are given by x∗ T x − x , =Φk ( ) (1) 3.2. Noise Detection by Relative Residual ∗ x = x − x − Φkx . (2) Let us define a relative residual ri for the i-th pixel of x by k In our problems, is a small number because human faces T ei x are almost Lambertian. ri = , eT x x∗ Let us call the k-dimensional eigenspace the Normalized i ( +Φk ) Eigenspace (NES). We also use another notation, x, Φk, x e which explicitly specify x and Φk. where is an absolute residual given by Eq.(2), and i is a unit vector which consists of 1 in the i-th element and 0 in the other elements. We use the relative residual instead of 2.3. Canonical Space T absolute residual for the i-th pixel, ei x , because we would In this paper, a face space is defined as a space composed like to suppress noise not in the absolute scale but in the rel- from a set of frontal faces, which includes images of nu- ative scale. For example, low noise in a dark area should be merous persons taken under a wide variety of lighting con- suppressed when the relative residual is sufficiently large. ditions. To simplify the problem, we assume that good Noise detection is basically performed by |ri|. However, segmentation is readily accomplished as shown in Fig. 10. Eq. (4) suggests that when a considerable amount of noise Eigenspace analysis on the face space reduces the dimen- is included, the zero level of the relative residual may shift sion of the face space, with little loss of representability in response to the amount of noise and Φk. In order to com- [6, 4]. pensate the possible shift, we use |ri − r| instead of |ri|, Let xc, Φc denote a NES constructed over a canonical where r is the median of the whole ri. We don’t use the av- set. We call this the canonical space (CS). In our experi- erage, because we would like to neglect the direct noise fac- ments, a 45d CS is constructed from a canonical image set tors in Eq. (4). Consequently, noise can be detected when that consists of face images of 50 persons taken under 24 |ri − r|≥rθ, where rθ is a threshold. lighting conditions. 3. Noise Detection and Image Correc- 3.3. Image Correction by Projection The noise correction algorithm can be created on the basis tion of the noise detection, as follows: When |ri − r|≥rθ, the i x − α eT x x∗ 3.1. Effect of Noise in NIS -th pixel of should be replaced to (1 ) i ( +Φk ), where x∗ and α are provided simultaneously by calculat- Let us analyze the effect of noise on the projection onto ing a partial projection[1] when the partial region excludes NES x, Φk and the residual. Suppose that an object view noise regions. The image correction makes an intensity is completely encoded to the NES by value consistent with the projection. For example, shad- ∗ T x =Φk (x − x). ows and reflection regions are corrected when the NES is 2 Proceedings of the Sixth IEEE International Conference on Automatic Face and Gesture Recognition (FGR’04) 0-7695-2122-3/04 $ 20.00 © 2004 IEEE Figure 1: Example of eigenplane for Lambertian surface: Figure 3: Individual Eigenface: Average and 3 principal Average and 2 principal bases.