University of Tennessee, Knoxville TRACE: Tennessee Research and Creative Exchange Doctoral Dissertations Graduate School 12-2008 Multispectral Imaging For Face Recognition Over Varying Illumination Hong Chang University of Tennessee - Knoxville Follow this and additional works at: https://trace.tennessee.edu/utk_graddiss Part of the Electrical and Computer Engineering Commons Recommended Citation Chang, Hong, "Multispectral Imaging For Face Recognition Over Varying Illumination. " PhD diss., University of Tennessee, 2008. https://trace.tennessee.edu/utk_graddiss/480 This Dissertation is brought to you for free and open access by the Graduate School at TRACE: Tennessee Research and Creative Exchange. It has been accepted for inclusion in Doctoral Dissertations by an authorized administrator of TRACE: Tennessee Research and Creative Exchange. For more information, please contact [email protected]. To the Graduate Council: I am submitting herewith a dissertation written by Hong Chang entitled "Multispectral Imaging For Face Recognition Over Varying Illumination." I have examined the final electronic copy of this dissertation for form and content and recommend that it be accepted in partial fulfillment of the requirements for the degree of Doctor of Philosophy, with a major in Electrical Engineering. Mongi A. Abidi, Major Professor We have read this dissertation and recommend its acceptance: Paul B. Crilly, Seddik M. Djouadi, Andreas Koschan, Hamparsum Bozdogan Accepted for the Council: Carolyn R. Hodges Vice Provost and Dean of the Graduate School (Original signatures are on file with official studentecor r ds.) To the Graduate Council: I am submitting herewith a dissertation written by Hong Chang entitled “Multispectral imaging for face recognition over varying illumination.” I have examined the final electronic copy of this dissertation for form and content and recommend that it be accepted in partial fulfillment of the requirements for the degree of Doctor of Philosophy, with a major in Electrical Engineering. Mongi A. Abidi, Major Professor We have read this thesis and recommend its acceptance: Paul B. Crilly _________________________ Seddik M. Djouadi __________________________ Andreas Koschan __________________________ Hamparsum Bozdogan __________________________ Acceptance for the Council: Carolyn R. Hodges ______________________________ Vice Provost and Dean of the Graduate School (Original signatures are on file with official student records) Multispectral Imaging For Face Recognition Over Varying Illumination A Dissertation Presented for the Doctor of Philosophy Degree The University of Tennessee, Knoxville Hong Chang December 2008 Abstract This dissertation addresses the advantage of using multispectral narrow-band images over conventional broad-band images for improved face recognition under varying illumination. To verify the effectiveness of multispectral images for improving face recognition performance, three sequential procedures are taken into action: multispectral face image acquisition, image fusion for multispectral and spectral band selection to remove information redundancy. Several efficient image fusion algorithms are proposed and conducted on spectral narrow-band face images in comparison to conventional images. Physics-based weighted fusion and illumination adjustment fusion make good use of spectral information in multispectral imaging process. The results demonstrate that fused narrow-band images outperform the conventional broad-band images under varying illuminations. In the case where multispectral images are acquired over severe changes in daylight, the fused images outperform conventional broad-band images by up to 78%. The success of fusing multispectral images lies in the fact that multispectral images can separate the illumination information from the reflectance of objects which is impossible for conventional broad-band images. To reduce the information redundancy among multispectral images and simplify the imaging system, distance-based band selection is proposed where a quantitative evaluation metric is defined to evaluate and differentiate the performance of multispectral narrow-band images. This method is proved to be exceptionally robust to parameter changes. Furthermore, complexity-guided distance-based band selection is proposed using model selection criterion for an automatic selection. The performance of selected bands outperforms the conventional images by up to 15%. From the significant performance improvement via distance-based band selection and complexity-guided distance-based band selection, we prove that specific facial information carried in certain narrow-band spectral images can enhance face recognition performance compared to broad-band images. In addition, both algorithms are proved to be independent to recognition engines. Significant performance improvement is achieved by proposed image fusion and band selection algorithms under varying illumination including outdoor daylight conditions. Our proposed imaging system and image processing algorithms lead to a new avenue of automatic face recognition system towards a better recognition performance than the conventional peer system over varying illuminations. i Publications 1. H. Chang, Y. Yao, A. Koschan, B. Abidi, and M. Abidi, “Improving face recognition via narrow-band spectral range selection using Jeffrey divergence,” accepted IEEE Trans. on Information Forensics and Security 2. H. Chang, A. Koschan, B. Abidi, and M. Abidi, “Fusing visible continuous spectral images for face recognition under indoor and outdoor illuminations,” accepted Machine Vision and Applications, 2008. 3. H. Chang, Y. Yao, A. Koschan, B. Abidi, and M. Abidi, “Spectral range selection for improving face recognition under varying illuminations,” IEEE proc. ICIP 2008, pp. 2756-2759. 4. H. Chang, C.-H. Won, S.G. Kong, A. Koschan, and M. Abidi, "Multispectral visible and infrared imaging for face recognition,” Proc. IEEE Conference on Computer Vision and Pattern Recognition CVPR 2008, Fifth IEEE International Workshop on Object Tracking and Classification Beyond the Visible Spectrum (OTCBVS), 2008. 5. H. Chang, A. Koschan, B. Abidi, and M. Abidi, “Physics-based fusion of multispectral data for improved face recognition,” IEEE proc. ICPR2006, vol. III, 2006, pp. 1083-1086. 6. H. Chang, H. Harishwaran, M. Yi, A. Koschan, B. Abidi, and M. Abidi, “An indoor and outdoor, Multimodal, Multispectral and Multi-illuminant database for face recognition,” IEEE proc. CVPR2006, workshop on multi-model biometrics, 2006, pp. 54-61. 7. H. Chang, M. Yi, H. Harishwaran, A. Koschan, B. Abidi, and M. Abidi, “Multispectral face data fusion for indoor, outdoor authentication,” Biometrics Symposium, 2006. 8. H. Chang and M. Howlader, “Fractional symbol differential detection for DS-CDMA over Rayleigh fading channels,” IEEE proc. Wireless Communication and Networking Conference, vol.1, 2004, pp. 543-547. 9. M. Howlader and H. Chang, “Performance of chip-level differential detection for DS-CDMA in multipath fading channels,” IEEE proc. Military Communications Conference, vol. 2, 2003, pp.1065-1070. 10. H. Chang and J. Dong, “Research in software radio architecture,” Telecommunication Technology, vol. 3, pp. 17-19, 2001. ii Table of contents 1 Introduction..................................................................................................... 1 1.1 Motivation..........................................................................................................2 1.2 State of the art ....................................................................................................5 1.3 Contributions......................................................................................................6 1.4 Document organization......................................................................................8 2 Related work.................................................................................................. 10 2.1 Challenges to face recognition.........................................................................10 2.1.1 Face recognition.................................................................................11 2.1.2 Illumination variation.........................................................................12 2.1.3 Face databases....................................................................................14 2.2 Multispectral imaging ......................................................................................15 2.2.1 Light sources......................................................................................17 2.2.2 Human skin reflectance......................................................................18 2.2.3 Multispectral imaging using rotating wheels.....................................19 2.2.4 Multispectral imaging using electronically tunable filters.................20 2.3 Multispectral image fusion ..............................................................................21 2.3.1 Multispectral band selection ..............................................................22 2.3.2 Multispectral data fusion....................................................................24 3 Multispectral imaging system ...................................................................... 26 3.1 System setups...................................................................................................27 3.1.1 Hardware components .......................................................................27