Use of Photometric Stereo for the Accurate Modelling of Three
Total Page:16
File Type:pdf, Size:1020Kb
Use of Photometric Stereo for the Accurate Modelling of Three-Dimensional Skin Microrelief Ali Sohaib A thesis submitted in partial fulfilment of the requirements of the University of the West of England, Bristol for the degree of Doctor of Philosophy Faculty of the Environment and Technology University of the West of England, Bristol October 2013 Abstract The largest organ of human body - ”skin” is a multilayered organ with complex reflectance properties that not only vary with the direction of illumination but also with the wavelength of light. The complex Three Dimensional (3D) structure and optical properties of human skin makes it very difficult for techniques like Pho- tometric stereo to accurately recover its 3D shape. One problem in particular concerns the presence of interreflections at concave regions of the skin surface topography. Common features such as wrinkles, moles, lesions, burns and sur- gical scars can appear as elevated skin area or as indentations in the skin sur- face, and usually have a different colour when compared to the surrounding skin. These differences in colour and the degree of concavity determine the amount of interreflection present and hence significantly affect the overall recovery of the 3D topography of the skin. This thesis explores the use of varying incident light wavelength in the visible spectrum to improve the recovered topography of human skin. New algorithms were developed and implemented to minimise the effects of interreflections and an accuracy assessment of using wavelengths in the visible spectrum was carried out for Caucasian, Asian and African American skin types. The results demonstrate that white light is not ideal for imaging skin relief and also illustrate the differences in recovered skin topography due to a non-diffuse Bidi- rectional Reflectance Distribution Function (BRDF) for each colour illumination used. In addition to the complication caused by interreflections, the translucency of skin allows light to scatter within the skin. This confutes the assumption asso- ciated with photometric stereo that - light incident on a particular patch of the skin is reflected at the same point. This issue is taken care of by incorporating polarisation information with photometric stereo to improve the 3D skin shape recovery. The experiments conducted show improvement in the recovered skin ii topography when specular reflected light was used instead of the multiply sub- surface scattered diffuse light emanating from within the skin. The results also show an increase in polarised reflected light as the albedo of the skin decreases with wavelength in the visible spectrum, while darker skin tend to reflect more polarised light than lighter skin at similar wavelengths in the visible spectrum. These characteristics of human skin have not been investigated before and are important additions to the current state of the art in skin optics. iii Acknowledgements This research project would not have been possible without the support of many people. Foremost, I would like to express my gratitude to my principal supervisor Professor Melvyn Smith for his support and thoughtful guidance during my Ph.D study. In addition, I would like to acknowledge my supervisory team Dr Lyndon Smith, Dr Abdul Farooq and Mr Robb Warr for their invaluable assistance in the completion of this thesis and for providing constructive feedback on my research throughout. I would also like to express my gartitude to Dr Jeffrey Bamber and Nigel Bush at the Institute of Cancer Research, Sutton, UK for allowing me to use the PRI- MOS device. Furthermore, I would like to take this opportunity to thank all my colleagues at the Centre for Machine Vision, UWE, especially Laurence Broad- bent for his help with the CAD drawings of the numerous parts made during my PhD and Jahanzeb Ahmed for his help with the PovRay software. I would also like to thank Christopher Hart for the rapid prototyping of various parts used in the experiments. I am also thankful to all the participants who agreed to take part in this re- search study. Finally, I would like to express my deepest gratitude to my family: Mum, Dad, my brother and sister and my wife for their endless patience, support and encour- agement when it was most required. iv This work is dedicated to my family, especially Mum and Dad. v Contents 1 Introduction 1 1.1 Original Contributions . 3 1.2 Applications of Research . 5 1.3 Thesis Outline . 7 2 Literature survey 9 2.1 Review of Wrinkle Grading Scales . 9 2.2 Skin Anatomy . 10 2.3 Optical Properties of Skin . 14 2.4 Wrinkle Grading . 17 2.4.1 Photographic Scales . 18 2.4.2 Conditions for Analysis . 21 2.4.3 Wrinkle Grading Systems Discussion . 21 2.5 Review of 3D Imaging Techniques . 23 2.5.1 Replica Technique . 23 2.5.2 Mechanical Microscopy . 24 2.5.3 Laser Triangulation . 25 2.5.4 Fringe Projection . 26 2.5.5 Light Transmission . 26 2.5.6 Photometric Stereo . 27 2.6 Simulation Based Analysis of Skin Topography . 30 vi 2.7 Parameters Commonly Used to Characterise Surfaces . 30 2.8 Discussion . 33 3 Photometric Stereo and Reflectance Models 35 3.1 Reflectance Models . 38 3.1.1 Lambertian Model . 39 3.1.2 Phong and Blinn-Phong Model . 40 3.1.3 Oren Nayer Model . 42 3.1.4 Torrance-Sparrow Model . 43 3.1.5 Ward Model . 46 3.2 Discussion . 47 4 Interreflection Modelling of Skin 49 4.1 Literature Survey . 51 4.2 Skin Optical Characteristics . 55 4.3 Multispectral Reflectance of Skin . 57 4.4 BRDF of Skin as a Function of Wavelength . 58 4.5 Materials and Methods . 59 4.5.1 Photometric Stereo Setup and Acquisition . 59 4.5.2 Camera Calibration . 61 4.6 Segmentation Based Method . 64 4.6.1 Skin Replica and Ground Truth . 64 4.6.2 Error Analysis . 65 4.6.3 Algorithm for Improved Depth Estimate . 67 4.6.4 Skin Roughness Analysis . 70 4.7 Gradient Spread Method . 75 4.7.1 Skin Replica and Ground Truth . 75 4.7.2 Error Analysis . 76 4.7.3 Surface Normal Mapping in Gradient Space . 80 vii 4.7.4 Validation . 84 4.8 Discussion . 85 4.9 Conclusion . 87 5 Shape from Polarisation 90 5.1 Literature Survey . 92 5.2 Polarisation Based Separation of Diffuse and Specular components 94 5.3 Umov Effect . 96 5.3.1 Setup for Umov Effect . 97 5.4 Shape from Specular and Diffuse Reflection . 100 5.5 Synthetic Image Analysis . 101 5.6 Algorithm . 104 5.7 Real Surface Analysis . 105 5.8 Skin Microrelief Analysis . 107 5.9 Discussion . 110 6 Conclusions, Limitations and Future Work 112 6.1 Summary of Contributions . 112 6.2 Future Work . 116 Glossary 139 List of Symbols 143 Appendices 146 A Datasheets 146 A.1 LED Datasheet . 147 A.2 Camera Datasheet . 160 A.3 Camera Datasheet . 163 A.4 Triband Filter Datasheet . 166 viii B Mathematical Derivations 168 B.1 Cross Product for Normal Vector Calculation . 168 B.2 Constraint calculation for diffuse component . 170 ix List of Figures 2.1 (a) Cross-sectional view of skin showing epidermis, dermis and appendage structures. (b) Cellular structure of epidermis . 12 2.2 Interaction of light with skin. 15 3.1 A graphical representation of facet. 37 3.2 Arrangement of incident, viewing, halfway and reflection vector. 40 4.1 The underestimation of concave shape from photometric stereo. 49 4.2 Multispectral reflectance of different skin types (NCSU skin reflectance data [1]). 57 4.3 (a) Schematics of the photometric stereo based 3D capture sys- tem. (b) The 3D skin macro and microrelief from the back of the hand acquired using the device. 60 4.4 (a) CCD spectral response. (b) Triband filter response. (c) Spectral response of the LEDs . 62 4.5 Cross-sectional view of the wrinkled region obtained from each colour light and the proposed algorithm. 66 4.6 (a) original image. (b) Segmented wrinkle line using watershed algorithm. 68 4.7 Comparison of algorithm with ground truth. 70 4.8 (Graphical representation of cube size variation ((a) - (c)) used for the calculation of Fractal dimension for a patch of skin. 74 x 4.9 (a) Forehead wrinkle directly imaged using photometric stereo de- vice. (b) Corresponding replica. (c) 3D reconstruction of skin im- ages taken in vivo............................ 76 4.10 (a) 2D Profiles obtained for the wrinkled region from each colour light and the proposed method (The wrinkle lies between 6-8mm length). (b)-(f) RMS error in height from red, green, blue, white light and the algorithm respectively (The dotted rectangular region points out the location of wrinkle). 78 4.11 (a) Over and underestimation of slope from each (R, G, B) light at the wrinkled region (valley). The mid region data value rise repre- sents lowest point of valley of wrinkle where slope changes sign. (b) Increased scale to show differences in slope from one side of valley. 79 4.12 Gradient map obtained from red and green light (The center cluster of a lighter shading represents the group of gradients from red light). 80 4.13 Movement of gradients from red light towards the green. The gradi- ent set is calculated in the region bounded between quadratic and linear fits. 81 5.1 The setup uses a rotating polariser in front of the camera and linear polarisers in front of RGBW LEDs. 97 5.2 1st row: Degree of polarisation (DOP) for red light, green and blue light for a Caucasian (European) Subject. 2nd row: Degree of po- larisation for red light, green and blue light for a Asian (Chinese) Subject. (The Image intensity represents the degree of polarisa- tion of light in the range 0 to 1) .