Feature Extraction Using Dimensionality Reduction Techniques: Capturing the Human Perspective
Total Page:16
File Type:pdf, Size:1020Kb
Wright State University CORE Scholar Browse all Theses and Dissertations Theses and Dissertations 2015 Feature Extraction using Dimensionality Reduction Techniques: Capturing the Human Perspective Ashley B. Coleman Wright State University Follow this and additional works at: https://corescholar.libraries.wright.edu/etd_all Part of the Computer Engineering Commons, and the Computer Sciences Commons Repository Citation Coleman, Ashley B., "Feature Extraction using Dimensionality Reduction Techniques: Capturing the Human Perspective" (2015). Browse all Theses and Dissertations. 1630. https://corescholar.libraries.wright.edu/etd_all/1630 This Thesis is brought to you for free and open access by the Theses and Dissertations at CORE Scholar. It has been accepted for inclusion in Browse all Theses and Dissertations by an authorized administrator of CORE Scholar. For more information, please contact [email protected]. FEATURE EXTRACTION USING DIMENSIONALITY REDUCITON TECHNIQUES: CAPTURING THE HUMAN PERSPECTIVE A thesis submitted in partial fulfillment of the requirements for the degree of Master of Science By Ashley Coleman B.S., Wright State University, 2010 2015 Wright State University WRIGHT STATE UNIVERSITY GRADUATE SCHOOL December 11, 2015 I HEREBY RECOMMEND THAT THE THESIS PREPARED UNDER MY SUPERVISION BY Ashley Coleman ENTI TLED Feature Extraction using Dimensionality Reduction Techniques: Capturing the Human Perspective BE ACCEPTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF Master of Science. Pascal Hitzler, Ph.D. Thesis Advisor Mateen Rizki, Ph.D. Chair, Department of Computer Science Committee on Final Examination John Gallagher, Ph.D. Thesis Advisor Mateen Rizki, Ph.D. Chair, Department of Computer Science Robert E. W. Fyffe, Ph.D. Vice President for Research and Dean of the Graduate School Abstract Coleman, Ashley. M.S., Department of Computer Science and Engineering. Wright State University, 2015. Feature Extraction Using Dimensionality Reduction Techniques: Capturing the Human Perspective. The purpose of this paper is to determine if any of the four commonly used dimensionality reduction techniques are reliable at extracting the same features that humans perceive as distinguishable features. The four dimensionality reduction techniques that were used in this experiment were Principal Component Analysis (PCA), Multi-Dimensional Scaling (MDS), Isomap and Kernel Principal Component Analysis (KPCA). These four techniques were applied to a dataset of images that consist of five infrared military vehicles. Out of the four techniques three out of the five resulting dimensions of PCA matched a human feature. One out of five dimensions of MDS matched a human feature. Two out of five dimensions of Isomap matched a human feature. Lastly, none of the resulting dimensions of KPCA matched any of the features that humans listed. Therefore PCA was the most reliable technique for extracting the same features as humans when given a set number of images. iii Table of Contents Introduction .............................................................................................................. 1 Motivation ............................................................................................................... 1 Human Perspective ................................................................................................ 1 Computer Vision ………….......………………………………………………………… 2 Introduction to Dimensionality Reduction ……………………………………...…….. 3 Problem Statement .……………………………………………………………………. 4 Related Work ............................................................................................................ 4 Experiment ............................................................................................................... 5 Survey Creation ...................................................................................................... 5 Dataset …………………………………………………………………………………… 8 Principal Component Analysis (PCA) ………………………………………………… 9 Multi-dimensional Scaling (MDS) ……………………………………………………. 11 Kernel PCA …………………………………………………………………………….. 13 Isomap ………………………………………………………………………………….. 14 Experiment Outline ……………………………………………………………………. 16 Result …………………………………………………………………………………….. 19 Survey …………………………………………………………………………………. 19 PCA ……..………………………………………………………………………..……. 20 MDS …………………………...………………………………………………............. 24 Isomap ……………………………...………………………………………………….. 26 Kernel PCA ……………………………...……………………………………............. 28 Summary …………………………………………………………………...………….. 29 Conclusion ………………………………………………………………………………. 31 Bibliography …………………………………………………………………………….. 34 Appendix ………………………………………………………………………………… 36 Survey Results ………………………………………………………………………...36 Feature Extractions ………………………………………………………………...36 Feature Labeling ………………………………………………………………...… 37 iv Eigenvector Visualization ……………………………………………………………. 38 Plots ……………………………………………………………………………………. 39 PCA …………………………………………………………………………………. 39 MDS ……………………………………………………………………………….... 46 Isomap ……………………………………………………………………………… 54 Kernel PCA .……………………………………………………………….............. 61 Linear Imaging Results ……………………………………………………………... 69 PCA …………………………………………………………………………..……. 69 MDS ………………………………………………………………………............. 76 Isomap …………………………………………………………………………….. 84 Kernel PCA ………………………………………………………………............. 92 Extreme Testing Results …………………………………………………………… 99 PCA ………………………………………………………………………………... 99 MDS ………………………………………………………………………............. 104 Isomap …………………………………………………………………………….. 109 Kernel PCA ………………………………………………………………............. 114 v List of Figures Figure/Illustration Page 1. Example of Dimensionality Reduction ………………………………... 3 2. Feature Extraction Survey Question ………………………………….. 6 3. Feature Labeling Survey Question ……………………………………. 7 4. Example of Image in Dataset ………………………………………….. 8 5. Plot of Dimension 1 by 2 of PCA ………………………………………. 17 6. Last 25 Images of the Resulting PCA 2nd Dimension ……………….. 18 7. Window Color Extreme Test results for Isomap ……………………… 18 8. Visualization of Resulting 1st Dimension of PCA ……………………... 21 9. Visualization of Resulting 2nd Dimension of PCA ……………………... 21 10. Visualization of Resulting 3rd & 4th Dimensions of PCA …………….… 22 11. Visualization of Resulting 5th Dimension of PCA ……………………… 23 12. Visualization of Resulting 2nd Dimension of MDS …………………….. 24 13. Visualization of Resulting 3rd, 4th & 5th Dimensions of MDS ………..... 25 14. Visualization of Resulting 1st Dimension of Isomap …………………… 27 15. Visualization of Resulting 2nd Dimension of Isomap …………………... 27 16. Visualization of Resulting 1st & 2nd Dimension of KPCA ……………… 29 Appendix Figures Page A1. Eigenvectors Visualization of PCA Figures 4.1-4.5 …………………………………………………..... 38 A2. Plots for PCA vi Plots 1.1-1.15 ……………………………………………………… 39-46 A3. Plots for MDS Plots 2.1-2.15 ……………………………………………………… 46-53 A4. Plots for Isomap Plots 3.1-3.15 ……………………………………………………… 54-61 A5. Plots for KPCA Plots 4.1-4.15 ……………………………………………………… 61-68 A6. Linear Visualization of PCA Results Figures 5.1.1-5.5.3 ..……………………………………………..... 69-76 A7. Linear Visualization of MDS Results Figures 6.1.1-6.5.3 ..……………………………………………..... 76-84 A8. Linear Visualization of Isomap Results Figures 7.1.1-7.5.3 ..……………………………………………..... 84-92 A9. Linear Visualization of KPCA Results Figures 8.1.1-8.5.3 ..……………………………………………..... 92-99 vii List of Tables Tables Page 1. Summary of Results for PCA ………..………………………………... 24 2. Summary of Results for MDS ………...……………………………….. 26 3. Summary of Results for Isomap ....……………………………………. 28 4. Summary of Results for KPCA ….…………………………………….. 29 5. Summary of Results for All Techniques …...…………………………. 30 Appendix Figures Page A1. Feature Extraction Survey Results All ...………………………..…… 36 A2. Feature Extraction Survey Results …..………....…………………… 36 A3. Feature Labeling Survey Results Figures 3.1-3.5 …………………………………………………... 37 A4. Extreme Test for PCA ...………………………………………………... 99-104 A5. Extreme Test for MDS ………………………………………………….. 104-109 A6. Extreme Test for Isomap ……………………………………………….. 109-114 A7. Extreme Test for KPCA ..……………………………………………….. 114-118 viii Introduction Motivation In the field of computer vision, computer scientists have strived to make a computer’s vision similar to human vision using methods such as neural networks and dimensionality reduction techniques for image processing. In the case of dimensionality reduction the features that are extracted are generally unknown. However if these features were determined to be the same features that humans perceive when given a set of images it will provide a better understanding of dimensionality reduction techniques. This understanding could lead to advancements in the field of pattern recognition and tracking that uses these dimensionality reduction techniques as a way to analyze and/or process images. Human Perspective The human brain takes less than a fraction of a second for a human to recognize an object. The part of the brain which is responsible for object recognition is the visual cortex, located in the cerebral cortex. There are over 40 areas in the primary visual cortex, which could explain why the underlying mechanisms for perceiving objects is not well understood [8]. Despite the fact that this area isn’t well understood there are different theories on how the brain recognizes objects. The most popular theory is the Two Stream Hypothesis. This hypothesis states that there are two streams, the ventral and the dorsal streams located inside of the primary visual cortex. 1 In the beginning, the two streams