DEGREE PROJECT IN ELECTRICAL ENGINEERING, SECOND CYCLE, 30 CREDITS STOCKHOLM, SWEDEN 2020 Electronic System in Robotic Head with Human Eye Movements N. TOBIAS FORSÉN KTH ROYAL INSTITUTE OF TECHNOLOGY SCHOOL OF ELECTRICAL ENGINEERING AND COMPUTER SCIENCE Abstract Eyetracking applications are becoming more and more common. Applications such as monitoring a driver's focus on the road and behavior of customers in a store are just a few examples. For a long time, eye tracking has been expensive and too complex to make all kinds of applications. Thanks to the recent explosion in camera sensor technology it is possible to develop eye trackers cheaper and better. However, developing and testing these sensors requires advanced algorithms. These algorithms are then tested by a variety of people to confirm that they work. This thesis will try to see if it is possible to make robotic eyes that move like a human's eyes. This project includes a detailed process for developing a human- robotic head. Implemented movements on the robot are Saccader and smooth- pursuit with potential for Vergence movements and Vestibulo-ocular movements and microsaccades. The robot head could be used instead of a person for Eyetrackers. The report contains how the author developed the electronics to power the robotic human head. The electronics scheme is discussed and developed. The model of how the robot head and its eyes moves are explained and then implemented. The thesis also contains an explanation of the software used. With this project, it will make it easier to invent and develop good Eye trackers. Keywords Robotic Head; Humanoid eyes; Robotic saccade; Saccade movement; Sammanfattning Applicationer med Eyetracking börjar bli mer och mer vanliga. Applikationer som att övervaka en chaförs fokus på vägen samt beteende hos kunder i en butik är bara ett få exempel. Under lång tid har ögonspårning varit dyrt och för komplex för att göra alla möjliga applikationer. Tack vare den senaste tidens explosion i teknik i kamerasensor går det att utveckla ögonspårare billigare och bättre. Men att utveckla och testa dessa sensorer kräver avancerade algoritmer. Dessa algoritmer testas sedan av en mängd personer för att bekräfta att de fungerar. Denna avhandling kommer att försöka se om det är möjligt att göra robotögon som rör sig som en människas ögon. Detta projekt innehåller en detaljerad process för att utveckla ett mänskligt robot huvud. Implementerade rörelser på roboten är Saccader och smoothpursuit med potentioal för Vergence-rörelser och Vestibulo-okulära rörelser och microsaccades. Robothuvudet skulle kunna användas istället för en person för Eyetrackers. Rapporten innehåller hur författaren utvecklat elektroniken för att driva det robotiska mänskliga huvudet. Elektronik-schemat diskuteras och utvecklas. Modellen av hur robothuvudet och dess ögon förklaras och sedan implementeras. I avhandlingen finns även en förklaring av den programvara som används. Med detta projekt kommer det bli lättare att ta fram och utveckla bra Eye- trackers Nyckelord RobotHuvud; Humanoida ögon; Robot saccader; Saccade rörelse; Acknowledgment I had a great time working at Tobii on this robotic head. I would like to thank Jennifer Tannfelt Wu my coworker for great teamwork. And I would also like to thank my handler Magnus Smedberg in Tobii and all the other super nice and helpful people in Tobii. I also want to give a big thank you to my supervisor Zhonghai Lu for helping me with this report. It has been really helpful. Table of Contents 1 Introduction ................................................................................................................. 1 Background ..................................................................................................................... 1 Problem............................................................................................................................. 2 Purpose ............................................................................................................................. 2 Goal ..................................................................................................................................... 2 Benefits ............................................................................................................................. 2 Ethics and Sustainability ............................................................................................ 2 Methodology / Methods .............................................................................................. 3 2 Background theory .................................................................................................... 4 Anatomy of the eye ....................................................................................................... 4 Field of view .................................................................................................................... 5 Distance between the pupil of the eyes ................................................................. 5 Eye movement ................................................................................................................ 5 2.4.1 Saccades ..................................................................................................................................... 6 2.4.2 Smooth pursuit ........................................................................................................................ 6 2.4.3 Vergence movement ............................................................................................................ 7 2.4.4 Vestibulo-ocular movement ............................................................................................. 7 Eye trackers ..................................................................................................................... 7 3 Previous work in the field .................................................................................... 10 4 Implementation ....................................................................................................... 13 Motor selection ............................................................................................................ 13 Mechanical design ....................................................................................................... 18 Electronic design ......................................................................................................... 20 Controller ....................................................................................................................... 20 Implementing schematics ........................................................................................ 23 Programming of the system..................................................................................... 27 Model of system ........................................................................................................... 29 Movement planning in robot ................................................................................... 30 User interface................................................................................................................ 34 End implementation ................................................................................................... 36 5 Results of the Project ............................................................................................. 38 Final testing ................................................................................................................... 38 Test of saccades ............................................................................................................ 39 Test of smooth pursuit .............................................................................................. 40 Test running over a long period of time.............................................................. 42 Test of head movements ........................................................................................... 43 Test of two eyes ............................................................................................................ 43 Testing with Eye-Tracking ....................................................................................... 44 Sources of errors ......................................................................................................... 46 6 Conclusions ............................................................................................................... 47 The electronics reflection ........................................................................................ 47 Reproducible ................................................................................................................. 47 Control reflection ........................................................................................................ 48 Goals ................................................................................................................................. 48 i Future improvements ................................................................................................ 48 7 References .................................................................................................................... 1 ii 1 1 Introduction Humans have many ways of interacting with the world. We are inventing more and more different types of interactions with electronics. For a long time, we had sensors like the microphone to listen to humans