Design and Implementation of Image Processing and Compression Algorithms for a Miniature Embedded Eye Tracking System Pavel Morozkin
Total Page:16
File Type:pdf, Size:1020Kb
Design and implementation of image processing and compression algorithms for a miniature embedded eye tracking system Pavel Morozkin To cite this version: Pavel Morozkin. Design and implementation of image processing and compression algorithms for a miniature embedded eye tracking system. Signal and Image processing. Sorbonne Université, 2018. English. NNT : 2018SORUS435. tel-02953072 HAL Id: tel-02953072 https://tel.archives-ouvertes.fr/tel-02953072 Submitted on 29 Sep 2020 HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est archive for the deposit and dissemination of sci- destinée au dépôt et à la diffusion de documents entific research documents, whether they are pub- scientifiques de niveau recherche, publiés ou non, lished or not. The documents may come from émanant des établissements d’enseignement et de teaching and research institutions in France or recherche français ou étrangers, des laboratoires abroad, or from public or private research centers. publics ou privés. Sorbonne Université Institut supérieur d’électronique de Paris (ISEP) École doctorale : « Informatique, télécommunications & électronique de Paris » Design and Implementation of Image Processing and Compression Algorithms for a Miniature Embedded Eye Tracking System Par Pavel MOROZKIN Thèse de doctorat de Traitement du signal et de l’image Présentée et soutenue publiquement le 29 juin 2018 Devant le jury composé de : M. Ales PROCHAZKA Rapporteur M. François-Xavier COUDOUX Rapporteur M. Basarab MATEI Examinateur M. Habib MEHREZ Examinateur Mme Maria TROCAN Directrice de thèse M. Marc Winoc SWYNGHEDAUW Encadrant Abstract Human-Machine Interaction (HMI) progressively becomes a part of coming future. Being an example of HMI, embedded eye tracking systems allow user to interact with objects placed in a known environment by using natural eye movements. The EyeDee™ portable eye tracking solution (developed by SuriCog) is an example of an HMI- based product, which includes Weetsy™ portable wire/wireless system (including Weetsy™ frame and Weetsy™ board), π-Box™ remote smart sensor and PC-based processing unit running SuriDev eye/head tracking and gaze estimation software, delivering its result in real-time to a client’s application through SuriSDK (Software Development Kit). Due to wearable form factor developed eye-tracking system must conform to certain constraints, where the most important are low power consumption, low heat generation low electromagnetic radiation, low MIPS (Million Instructions per Second), as well as support wireless eye data transmission and be space efficient in general. Eye image acquisition, finding of the eye pupil ROI (Region Of Interest), compression of ROI and its wireless transmission in compressed form over a medium are very beginning steps of the entire eye tracking algorithm targeted on finding coordinates of human eye pupil. Therefore, it is necessary to reach the highest performance possible at each step in the entire chain. Applying of image compression allows to drastically reduce amount of data to be sent to a remotely located PC-based processing unit performing eye tracking. However, to reach maximal performance image compression algorithms must be executed at the same frequency as eye image acquisition frequency (100 Hz minimum). Such a high frequency makes a certain constraints on several compression system parameters as well as compressed eye images, where the major ones are size of compressed eye image, quality of decompressed eye image, time needed to perform compression/decompression, computational complexity, operating memory requirements and some others. In contrast with state-of-the-art general-purpose image compression systems, it is possible to construct an entire new eye tracking application-specific image processing and compression methods, approaches and algorithms, design and implementation of which are the goal of this thesis. ii Contents Abstract ................................................................................................................................................... ii Contents .................................................................................................................................................. iii List of Figures ........................................................................................................................................... v List of Tables ......................................................................................................................................... viii List of Abbreviations ............................................................................................................................... ix 1 Introduction ......................................................................................................................................... 1 1.1 Objective and Scope of the Research ........................................................................................... 2 1.2 Summary of the Contributions .................................................................................................... 2 1.3 Thesis Outline .............................................................................................................................. 4 1.4 Thesis Information ....................................................................................................................... 4 2 Theoretical Part .................................................................................................................................. 5 2.1 Introduction ................................................................................................................................. 5 2.2 Eye Tracking Specifics ................................................................................................................. 5 2.3 Image Nature and Terminology Specifics .................................................................................... 8 2.4 Image Compression Fundamentals .............................................................................................. 9 2.4.1 Using Advantages of Human Visual System ...................................................................... 11 2.4.2 Image Compression Building Blocks .................................................................................. 12 2.4.3 Image Compression Techniques (JPEG and JPEG 2000) ................................................. 13 2.4.4 Video Compression Techniques (H.264 and H.265) ........................................................... 14 2.4.5 Image Compression Recent Improvements......................................................................... 15 2.5 Machine Learning ...................................................................................................................... 17 2.6 Neural Network Fundamentals .................................................................................................. 19 2.6.1 Neural Network Basics ....................................................................................................... 19 2.6.2 Multilayer Perceptron and Convolutional Neural Network ............................................... 30 2.6.3 Some Related Work in ANNs/CNNs/DNNs ...................................................................... 32 2.7 Eye Tracking System Application Specifics ............................................................................... 33 2.8 Computational Complexity and Implementation Aspect .......................................................... 35 2.9 Conclusion .................................................................................................................................. 37 3 Methodology ...................................................................................................................................... 39 3.1 Introduction ............................................................................................................................... 39 3.2 SuriCog's Eye Tracking Application Specifics ........................................................................... 39 3.2.1 SuriCog's Eye Tracking Algorithm .................................................................................... 39 3.2.2 SuriCog's Application-Specific Image Compression ........................................................... 40 3.2.3 Finding of the Eye Image Compression Algorithm Requirements ..................................... 40 3.2.3.1 Finding Maximal Time of Eye Image Compression/Decompression .......................... 41 3.2.3.2 Finding Minimal Size of Compressed Eye Image........................................................ 42 3.2.3.3 Finding Minimal Quality of Decompressed Eye Image .............................................. 42 3.2.3.4 Considerations on Ability to Operate in Lossy Transmission Medium ...................... 43 3.3 Eye Image Compression Alternative Approaches ...................................................................... 44 3.3.1 3D Modeling of Pupil Surface ............................................................................................ 44 3.3.2 Dictionary Based Eye Image Compression ......................................................................... 46 3.3.3 Dictionary + DCT Based Eye Image Compression ........................................................... 48 3.3.4 Schemes with Shared Model ..............................................................................................