Controlling a 3D Printed Bionic Hand by Using Brain Waves by BOYAN LI, B.S.E.E. A THESIS Submitted to The Department of Engineering Colorado State University-Pueblo In partial fulfillment of the requirements for the degree of Master of Science CERTIFICATION OF ACCEPTANCE This thesis presented in partial fulfillment of the requirements for the degree of Master of Science has been accepted by the Program with emphasis in Mechatronics Engineering Colorado State University – Pueblo APPROVED: Dr. Nebojsa I. Jaksic, Ph.D., P.E., Committee Chair Date Dr. Jude L. DePalma, Ph.D., Committee Member Date Dr. Bahaa I. Kazem, Ansaf, Ph.D., Committee Member Date Master’s Candidate BOYAN LI, B.S.E.E. Date of Thesis Presentation August 15th, 2018 BOYAN LI for the degree of Master of Science in Engineering with emphasis in Mechatronics, presented on August 15th, 2018 at Colorado State University-Pueblo Title: Controlling a 3D Printed Bionic Hand by Using Brain Waves Abstract approved: Professor Nebojsa I. Jaksic, Ph.D., P.E., Advisor Abstract High quality bionic hands are expensive while low-cost bionic hands have many operational problems during the process of using them. Hence, this thesis aims to improve the stability of low-cost bionic hands. The Hackberry bionic hand has been selected for characterization and improvement. A new control system combining Electroencephalogram (EEG) with an Inertial Measurement Unit (IMU) sensor is proposed and implemented for the Hackberry bionic hand in this thesis. The purpose of the IMU sensor is to decide when the bionic hand should close based on the hand movements. The EEG-based headset (Mindwave Mobile) designed by Neurosky is used to communicate with human brains. Several validation tests have been conducted to verify the stability of the new control system. Conclusions and future work are also addressed. Table of Contents Abstract ........................................................................................................................................ III Table of Contents ......................................................................................................................... iii List of Figures ................................................................................................................................ v List of Tables ................................................................................................................................ ix Acknowledgements ....................................................................................................................... x Abbreviations ............................................................................................................................... xi Chapter I Introduction ............................................................................................................. 1 1.1 Motivation ........................................................................................................................ 1 1.2 Research Goals ................................................................................................................. 2 1.3 Structure of Thesis ........................................................................................................... 5 Chapter II Background and Literature Review .................................................................. 7 2.1 History of prosthetics ....................................................................................................... 7 2.2 Technologies .................................................................................................................. 11 2.2.1 Bionic Hands ........................................................................................................... 11 2.2.1 Brain-Computer Interface (BCI) ............................................................................. 15 2.2.2 Electroencephalogram............................................................................................. 18 2.3 Previous Research .......................................................................................................... 21 2.4 Conclusion ...................................................................................................................... 25 Chapter III Hardware ........................................................................................................... 26 3.1 The Methodology of Choosing Hardware ...................................................................... 26 3.2 MakerBot® ..................................................................................................................... 27 3.2.1 3D-Printing ............................................................................................................. 28 iii 3.2.2 MakerBot® Replicator 2 ......................................................................................... 29 3.3 Hackberry ....................................................................................................................... 31 3.4 Arduino........................................................................................................................... 34 3.5 Neurosky Mindwave Mobile .......................................................................................... 37 3.6 IMU Sensor: MPU6050 ................................................................................................. 40 3.7 Bluetooth Module: HC-05 .............................................................................................. 41 Chapter IV Bionic Hand Control System ............................................................................ 43 4.1 System Design ................................................................................................................ 43 4.2 Hardware Design ............................................................................................................ 48 4.3 Software Design ............................................................................................................. 52 Chapter V Testing of Bionic Hand ......................................................................................... 59 5.1 Test Design ..................................................................................................................... 59 5.2 Testing ............................................................................................................................ 63 5.2.1 Hardware/Software Verification ............................................................................. 63 5.2.2 Standardized Assessment Procedure ....................................................................... 74 5.2.3 Physical Application Tests ...................................................................................... 77 5.3 Conclusions .................................................................................................................. 102 Chapter VI Conclusions and Future Work ....................................................................... 105 References .................................................................................................................................. 108 Appendix A Bionic Hand Program ..................................................................................... 113 iv List of Figures Figure 1. An Artificial Toe Found in Egypt ....................................................................... 8 Figure 2. An Iron Hand of a German Mercenary.............................................................. 9 Figure 3. An Artificial Hand Designed by Thomas Openshaw ...................................... 10 Figure 4. Oscar Pistorius with Running Blades ............................................................... 11 Figure 5. Open Bionics ....................................................................................................... 13 Figure 6. Brain-Computer Interface [19] ......................................................................... 16 Figure 7. The American Electroencephalographic Society Standardized - the International 10-20 System [29] ................................................................................. 20 Figure 8. A 3D Printer - MakerBot Z18 ........................................................................... 28 Figure 9. MakerBot Replicator 2 ....................................................................................... 30 Figure 10. MakerBot 2 Extruder ....................................................................................... 31 Figure 11. Hackberry Bionic Hand [50] ........................................................................... 32 Figure 12. Arduino Nano Microcontroller ....................................................................... 35 Figure 13. Neurosky Mindwave Mobile Headset ............................................................. 38 Figure 14. High Attention Meter Value ............................................................................ 39 Figure 15. Low Attention Meter Value ............................................................................. 39 Figure 16. 3-axis Accelerometer/Gyro MPU6050 ............................................................ 41 Figure 17. Bluetooth Communication Board HC-05 ....................................................... 42 Figure 18. Bionic Hand Control System Flowchart ......................................................... 45 Figure 19. The New Control System Flowchart ..............................................................