Mobile Teleoperation of a Mobile Robot
Total Page:16
File Type:pdf, Size:1020Kb
Lappeenranta University Of Technology Faculty of Technology Management Department of Information Technology Master’s Thesis Ehab Aboudaya Mobile Teleoperation of a Mobile Robot Examiners: Professor Ville Kyrki Professor Heikki Kälviäinen Supervisor: Professor Ville Kyrki ABSTRACT Lappeenranta University of Technology Faculty of Technology Management Department of Information Technology Ehab Aboudaya Mobile Teleoperation of a Mobile Robot Thesis for the Degree of Master of Science in Technology 2010 109 pages, 67 figures and 8 tables. Examiners: Professor Ville Kyrki Professor Heikki Kälviäinen Keywords: Mobile Teleoperation, Mobile Devices, Nokia N770, Maemo 2.2, ARIA, Graphical User Interface. This thesis describes the design and implementation of a graphical application on a mobile device to teleoperate a mobile robot. The department of information tech- nology in Lappeenranta University conducts research in robotics, and the main mo- tivation was to extend the available teleoperation applications for mobile devices. The challenge was to port existing robot software library onto an embedded de- vice platform, then develop a suitable user interface application that provides suf- ficient functionality to perform teleoperation tasks over a wireless communication network. This thesis involved investigating previous teleoperation applications and conducted similar experiments to test and evaluate the designed application for functional ac- tivity and measure performance on a mobile device which have been identified and achieved. The implemented solution offered good results for navigation purposes particularly for teleoperating a visible robot and suggests solutions for exploration when no environment map for the operator is present. PREFACE It has been a long experience preparing this project, working and studying does not always meet but I thank Allah, then all my family back home in Libya for their support and my wife Gamira for living this experience with me. I wish to thank my supervisor for his patience and guidance, Mr Janne Laaksonen for his lab help and time spent testing this project with me. I also would like to warmly thank Ms Susanna Koponen for her assistance and rescue in official matters. I hope this material can be useful for someone in the near future. Tampere, September 1st, 2010 Ehab Aboudaya 4 Contents 1 INTRODUCTION 15 1.1 Background . 15 1.2 Objective and Scope . 15 1.3 Structure of the thesis . 16 2 FUNDAMENTAL CONCEPTS 17 2.1 Mobile Robots . 17 2.2 Teleoperation . 20 2.3 Mobile Teleoperation . 21 2.4 Mobile Devices . 22 2.5 Data Communication . 25 2.6 Graphical User Interfaces . 28 3 RELATED WORK 31 3.1 Stationary Teleoperation . 33 3.1.1 Desktop Client Interfaces . 34 3.1.2 Web-Based Interfaces . 38 3.2 Mobile Teleoperation . 45 3.3 Conclusions . 57 4 REQUIREMENTS AND SPECIFICATION 61 5 4.1 Overall Description . 61 4.2 Functional Requirements . 61 4.3 Hardware . 62 4.4 Communication . 63 4.5 Software . 63 4.6 Teleoperating Environment . 63 4.7 Dependencies and Scope . 64 4.8 Design and Implementation Constraints . 66 4.9 Use Cases . 67 4.9.1 Connection Settings . 69 4.9.2 Connect To ARIA Server . 70 4.9.3 Disconnect from ARIA Server . 71 4.9.4 Command . 72 4.9.5 Stop . 74 4.9.6 Get Image . 75 4.9.7 Move . 76 4.9.8 Get Robot Status . 77 4.9.9 Get Environment Map . 78 4.10 Non-Functional Requirements . 79 4.10.1 Performance and Quality Requirements . 79 6 4.10.2 Safety Requirements . 80 5 IMPLEMENTATION 81 5.1 UI Wire Frames . 81 5.2 Deployment Diagram . 85 5.3 Architectural Design . 86 5.4 Class Descriptions . 87 5.5 Sequence Diagrams . 88 5.6 Teleoperation Command Buttons . 91 5.7 Development Tools . 91 5.8 Interface Screen Shots . 93 5.8.1 RTMU Startup and Idle State . 93 5.8.2 RTMU Menu And Settings Dialog . 93 5.8.3 Teleoperation With Environment Map . 94 5.8.4 Teleoperation With No Map . 94 5.8.5 RTMU Hidden Toolbar . 95 5.8.6 RTMU Ratio Increase / Decrease Indicator . 95 5.8.7 RTMU Robot Image Feedback . 96 5.9 Installation and Help system . 96 6 TESTS AND RESULTS 97 6.1 Functional Testing . 97 7 6.2 Teleoperation Testing . 97 6.2.1 Navigation Test . 97 6.2.2 Navigation and Exploration Test . 97 6.2.3 Robot and Test Arena Images . 99 6.3 Operator Feedback . 100 6.4 Discussion . 101 7 CONCLUSIONS AND FUTURE DEVELOPMENT 103 REFERENCES 104 8 List of Figures 1 Pioneer 3-DX Mobile Robot . 17 2 Typical ARIA application structure . 19 3 Human operator for mobile robots . 20 4 Teleoperation system architecture sample . 21 5 Remote driving with PdaDriver . 22 6 Maemo N770, 2005 Nokia . 24 7 Maemo platform Key Components . 24 8 Minimum WLAN setup for intended teleoperation communication. 26 9 Sample of WLAN connectivity configuration on the N770. 27 10 MSC for client/server communication . 27 11 Different dialog GUI but same functionality . 28 12 MaemoPad GUI application . 29 13 MVC Pattern . 29 14 Novel UI for 3D map Exploration . 31 15 Virtual joystick gestures . 32 16 Fixed Location Operator Teleoperation . 33 17 RobotUI Desktop client UI . 34 18 RobotUI, GUI Architecture . 35 19 The Advantage of Mobility - Desktop GUI design . 36 9 20 KhepOnTheWeb Web Interface . 38 21 WebDriver system architecture . 39 22 WebDriver user interface . 40 23 WebDriver map sample . 40 24 WebDriver ImageManager . 41 25 SWATS - Web Services Teleoperation . 42 26 Java Based Teleoperation System . 43 27 Java Based Teleoperation Web UI . 43 28 Finger gesture interface . 45 29 PdaDriver: . 46 30 PdaDriver System Architecture . 46 31 PdaDriver Screen modes . 48 32 PdaDriver Vision Screen . 49 33 PdaDriver Sensor Modes . 49 34 PdaDriver Combined Screen Mode . 50 35 The Advantage of Mobility - PDA Interface . 52 36 The Advantage of Mobility - Simulated map teleoperation . 53 37 The Advantage of Mobility - Visibility of Robot . 54 38 The Advantage of Mobility - PDA and Desktop covered area . 55 39 The Advantage of Mobility -Mean and Standard Deviation Visibil- ity Times . 55 10 40 Teleoperation Common Tasks and UI elements. 58 41 RTMU overall system entities. 61 42 P3DX with laptop . 62 43 Teleoperating Environment Map . 64 44 Screen shot of MobileSim robot simulator. 65 45 Operator manages robot hardware and ARIA server software. 67 46 RTMU application use cases. 68 47 RTMU Command Use Case Sequence Diagram. 73 48 Hildon Application Views . 82 49 Single Toolbar, 360 or 420 pixels . 82 50 RTMU Menu Items. 83 51 RTMU Settings Dialog. 83 52 RTMU Auxiliarly Popup’s . 84 53 RTMU Deployment Diagram. 85 54 RTMU Class View. 86 55 Connect SD . 88 56 Move or Stop Command . 89 57 Request Robot Image . 90 58 RTMU Startup and Idle State . 93 59 Menu And Settings Dialog . 93 11 60 Connected to ARIA Server with map . 94 61 Connected to ARIA server with no map . 94 62 RTMU connected to ARIA Server with map and hidden Toolbar. 95 63 RTMU connected to ARIA Server and velocity banner visible. 95 64 RTMU connected to ARIA Server and Requested Image. 96 65 Teleoperation Test Arena . ..