SHANGHAI SERVICE ROBOT A Major Qualifying Project Report: Submitted to the Faculty of the WORCESTER POLYTECHNIC INSTITUTE In partial fulfillment of the requirements for the Degree of Bachelor of Science By: ___________________________________ ___________________________________ Jeremy Lowrey Timothy Sharood Date: May 27, 2013 In partnership with Shanghai University Partners: Weiyong Mao, Si Li Approved: ___________________________________ Professor Yiming (Kevin) Rong, Major Advisor, Mechanical Engineering Professor Sheng Bao, Co Advisor, SHU This report represents the work of one or more WPI undergraduate students submitted to the faculty as evidence of completion of a degree requirement. WPI routinely publishes these reports on its web site without editorial or peer review. Abstract A robotic base and control program capable of following a user was created that linked TwinCAT automation software, and Visual Studios C++. Microsoft’s Kinect sensor and Xbox Controller were used for communication between the robot and the user. The robot used skeletal gestures, speech recognition, and remote commands to accomplish tasks. 2 Table of Contents SHANGHAI SERVICE ROBOT ......................................................................................................................... 1 Abstract ....................................................................................................................................................... 2 Table of Figures ........................................................................................................................................... 5 Acknowledgments ....................................................................................................................................... 6 Chapter 1: Introduction to the Project Team .............................................................................................. 7 Chapter 2: Introduction to the Project ........................................................................................................ 9 Project Goal ......................................................................................................................................... 9 Chapter 3: Background ResearcH .............................................................................................................. 10 Industry Robots ..................................................................................................................................... 10 Hands Free Control MetHods (Kinect) ................................................................................................... 11 Alternate Control Methods ................................................................................................................... 15 ToucH Screens & Remote Desktop .................................................................................................... 15 Wireless Controllers .......................................................................................................................... 16 Microsoft XBOX Controller ................................................................................................................ 17 Control Hardware .................................................................................................................................. 17 Programmable Logic Controllers ....................................................................................................... 17 Development Software .......................................................................................................................... 18 Integrated Development Environment (IDE) ......................................................................................... 18 Microsoft Visual Studio ...................................................................................................................... 19 General Purpose Programming Languages ........................................................................................ 19 Service Oriented ArcHitectures .............................................................................................................. 20 Microsoft Robotic Development Studio ............................................................................................ 21 Robotic Operating System ................................................................................................................. 22 Elderly Assist Robot ............................................................................................................................... 22 Chapter 4: Methodology ........................................................................................................................... 24 Task Specification .................................................................................................................................. 24 Base Evaluation and Redesign ............................................................................................................... 24 Hardware Replacement & Testing ..................................................................................................... 25 Base Design ........................................................................................................................................ 26 Control Software ................................................................................................................................ 27 3 Program Development .......................................................................................................................... 28 User Interface .................................................................................................................................... 31 Microsoft Controller .......................................................................................................................... 33 Skeleton and gesture based control .................................................................................................. 34 Voice control ...................................................................................................................................... 35 Chapter 5: Results and Discussion ............................................................................................................. 37 Conclusion: ................................................................................................................................................ 40 Weekly Progress: ............................................................................................................................... 40 Works Cited ............................................................................................................................................... 42 Appendix A ................................................................................................................................................ 43 VB Examples ...................................................................................................................................... 43 Appendix B ................................................................................................................................................. 44 Gamepad ........................................................................................................................................... 44 Controller drive modes ...................................................................................................................... 45 4 Table of Figures Figure 1: Maserati Plant Figure 2: Swisslog Robotic Platform ........................................................... 11 Figure 3: Microsoft Kinect sensor components ......................................................................................... 12 Figure 4: MonocHrome deptH Image ......................................................................................................... 13 Figure 5: Kinect coordinate system ........................................................................................................... 14 Figure 6: Skeletal tracking .......................................................................................................................... 15 Figure 7: EigHt CHannel Wireless Relay and Remote ................................................................................. 16 Figure 8: Microsoft Xbox 360 Controller Guide ......................................................................................... 17 Figure 9: MRDS VPL Example manifest ...................................................................................................... 22 Figure 10: FlowcHart of Program Structure ............................................................................................... 30 Figure 11: VB .dll TwinCAT function .......................................................................................................... 31 Figure 12: Program start menu (GUI) ........................................................................................................ 32 Figure 13: Drive form (GUI) ....................................................................................................................... 33 Figure 14: Controller Form (GUI) ............................................................................................................... 34 Figure 15: Motor Output, Log file .............................................................................................................