2017 2nd International Conference on Artificial Intelligence: Techniques and Applications (AITA 2017) ISBN: 978-1-60595-491-2

Hybrid Control for Low Cost Mobile Service Robot

Jia-xing ZHAI, Si-rui LIANG, Jiang-yong ZHU, Pei-pei XU, Xu LIU and Guo-qing WANG School of Construction Machinery, University of Chang'an, Xi an, 710064, Shaanxi, China

Keywords: Mobile service robot, Hybrid control system, Android, Monitoring system, Multi-core heterogeneous controller.

Abstract. The mobile service robot controller is a complex domain, not easily amenable to meet the need of being cheaper and easy-to-use trend. It involves such the disparate aspects as motion control, internet communication, multimedia interactivity and remote monitoring. It is a challenge to integrate so different kinds of capabilities together. This paper describes our attempts to combine those capabilities together, and reduce the cost and real-time response delay. Based on the management-oriented framework, we present a hybrid controller system which consists of the multi-core heterogeneous controller, an android device and a PC monitoring server. The android device communicates with the heterogeneous controller through the USB OTG mode. The Android device acts as the host part in charge of network communication, multimedia interactivity, video stream transfer, remote monitoring, and internal sensor acquisition. The heterogeneous controller is a slave end in charge of the motion control and motion sensor data acquisition. The PC server implements the display and storage of real-time video and other motion status data. At last, we provide the preliminary experimental results and believe this system is a novel approach to opening possibilities of making the service robot controller more powerful.

Introduction The control system of current mobile robot has to implement many functions, such as motion control, network communication, multimedia UI interaction, the device remote monitored control and so on. These requirements lead to high cost and complex structure. Therefore, it is not applicable to medium and low cost level products. It cannot meet the market demand. They are the main obstacles to the further mobile robot intelligent development. In order to meet the needs of more and more sensor and monitoring systems in mobile robot, since the early 60's, the research of mobile robot has started abroad. Ke Zhi Wang et al. proposed the Substation inspection robot that adopted the HitachiH8S of Pioneer3-AT as the control system, and put forward two modes of control. The faster processing speed improved the productivity, accuracy and flexibility of the substation inspection robot[1]. Ilmi Mohd Ariffina et al. proposed a new humanoid-operated 4-wheeled mobile platform. It adopted the Aldebaran RoboticsTM of NAO as the controller and obstacle-avoidance navigation system, and Arduino board are integrated. Therefore, the robot has all-direction, 24-hour standby and automatic navigation, which avoids obstacles effectively. However, it costs 80000 yuan, not applicable to low-cost products[2]. The Robot Operating System (ROS) adopted ROS control system and was on the Pioneer 3-DX platform. It can realize all-direction navigation indoor even go through windows and climb ramp for wheelchair and so on[3]. Although the new functions of the robot controller mentioned above are gradually improved, it basically meets the requirements of modern intelligent robots. But due to high cost and complex structure of the robot, these robots cannot be used on a massive scale. In this paper, the control system is a hybrid controller scheme with the Android equipment as the host controller and the multi-core heterogeneous micro-controller as auxiliary controller. The host controller can not only be used as a sensor module to replace various external sensors, but also can realize the functions of multimedia UI interaction, data transmission and processing and other functions. The host controller and the auxiliary controller are responsible for motion control and external sensors detection. The host controller and the server of monitoring system transmits data through WiFi and WiFi-Direct. All those can not only reduce the production cost, simplify the robot structure, but also achieve precise operation of the robot, improve the real-time performance and fast response ability. This kind of robot will meet the market demand and be widely popularized.

System Description The hybrid controller system adopts the Android equipment as the host controller, and multi-core heterogeneous microcontroller as the auxiliary controller. The auxiliary controller is responsible for motion control and external sensors detection. The host controller can be used as a sensor integrated module to providing multiple sensor data for the hybrid control system. At the same time, it can also be used as a powerful logical operation module of the system. The auxiliary controller transmits real-time motion data and external sensor data to the host controller through OTG, and the host controller will deal with these data according to its internal control logic. The host controller can also upload these data to the monitoring system via WiFi and Wifi-Direct, and the data will be processed by the monitoring system. Then the commands are sent to the master controller. Therefore, a complete hybrid control system is completed, which can effectively realize the precise and harmonious control of mobile robots. The overall design scheme of hybrid control system is shown in Figure 1:

Figure 1. The Total design scheme of hybrid control system.

Motion Control and Signal Acquisition System (Hard ware)

Management-oriented Multi-core Heterogeneous Controller The controller adopts a management-oriented multi-core heterogeneous controller with complete intellectual property rights. Comparing with other single controller, the greatest advantage of the controller is the ability to assign tasks for micro-controllers based on their different performance. Therefore, a number of microprocessors are responsible for different hardware modules and processing specific tasks, which can effectively solve the problem of lack of reliability, real-time and other issues, and reduce the cost of the controller. The architecture diagram of management- oriented multi-core heterogeneous controller is shown in Figure 2:

Figure 2. Management-oriented multi-core heterogeneous controller.

Hardware design The management-oriented multi-core heterogeneous controller includes the power module, the control module, the digital I / O module, the sensor module, the synchronous debugging module and the synchronous bus debugging module, the hardware structure and the parts of schematic diagram are shown in Figure 3, Figure 4.

Figure 3. Management-oriented multi-core Figure 4. The partial diagram of hardware design. heterogeneous hardware design.

Function Design of Android Mobile Device

Start-up Interface of Android Application Start-up interface is usually produced by the specified time of thread delay, and then carries out the jump between Start-up interface and main interface. Two interfaces need to be created. One is Splash Activity used to make the start-up interface, and the other is Main Activity served as the main interface after jumping out of Splash Activity. GIF animation can enhance the dynamic visibility of the product, but Android itself does not contain the GIF animation control. Therefore, we use the open source project GIF View to display the GIF animation. Host Controller Design The host controller's Android device is designed as two different client modes: WiFi-based client mode and WiFi Direct-based client mode. The host controller not only needs to perform real-time communication with the server, but also needs to receive monitoring data from the auxiliary controller side and send control commands back to the auxiliary controller side. The real-time communication between host controller and the auxiliary controller is achieved through OTG, and the host controller can also act as a sensor module and provides sensor data. The Usage of Built-in Sensors in Host Controller. The built-in sensors on the host controller have the advantage of having more convenient programming control and faster data responsiveness than the external sensors controlled by the co-processor. The use of sensors in the device are mainly to obtain built-in sensor instance of Android devices by calling the Sensor Manager class. Monitoring Equipment Status Data by Host Controller. The host controller receives the real- time motion data and the external sensor data and the built-in sensor data of the mobile robot from the co-controller, and can process the real-time data according to the control logic inside the host controller. On the one hand, the real-time data can be processed according to the internal control logic of the host controller. On the other hand, the real-time data can be uploaded to the server via Wifi or Wifi Direct and processed by the monitoring system server. Therefore, the monitoring system server can achieve precise motion control of the mobile robot by analyzing the dynamic sensor values.

Server Monitoring Software Design

General Design of Server Considering whether the scene environment is wifi coverage, the Server is divided into two types .they are PC server based on wifi communication and Android Server based on wifi-direct communication. These two kinds of servers need real-time communication with Client of monitoring system with the help of Android development environment. From this Figure, we can see the monitoring system requirements of Client. On one hand, we need to develop a set of Wifi Client application programs based on PC Server. On the other hand, we need to develop a set of Wifi-Direct Client application programs based on Android Server. application program based on Android Server. The design of PC-based Monitoring Software

Monitoring Software Design of PC Server in Wifi environment. The PC Server receives the data of videos and the control commands which are transmitted from Android Client through Wifi. The data transmission technology between PC Server and Android device is Socket technology. The Interface of open PC server is shown in the Figure 7. When the connection is successful, the PC Server will automatically jump to the real-time video monitoring page while you double-click the corresponding client. The real-time video monitoring page can display the recording video sent by client. Real-time video monitoring imagery in PC server is shown in Figure 8.

Figure 7. Interface of open PC server. Figure 8. Real-time video monitoring imagery in PC server. Monitoring Software Design of Android Client in Wifi environment. The Android Client records surroundings of the hybrid controller device in real time by Android phone's camera. And the Android Client sends the data stream to WPF Server by Socket technology. The WPF Server can receive the data of image by the function called on Preview Frame which is included in the Camera. The Android Server of the monitoring system listens to the corresponding port. The server sends the data flow to the corresponding image display control when the corresponding port detected the data stream. Thus, the data of camera videos can be transmitted to the Android Server. To obtain real time values of host controller , android server can acquire data which can be displayed in the form of value or graph through the sensor manager class. Real-time video monitoring imagery is shown in the Figure 9. Value of Acceleration sensor in Android is shown in the Figure 10, graph of Acceleration sensor in Android is shown in the Figure 11.

Figure 9. Real-time video monitoring Figure 10. Value of Acceleration sensor Figure 11. Graph of Acceleration imagery in Android mobile device. in Android mobile device. Android mobile device.

Design of Android-based Monitoring Software Using the Wifi-Direct technology, the hybrid control system achieves the real-time communication between the system Server and the Client without network covered. The devices which has hardware support can connect with each other directly without intermediate access points by Wifi- Direct technology. Android 4.0 and after versions support the API of Wifi-Direct. Using API, the developer can implement communication between these device.

Conclusion Aiming at the problems such as high cost and poor real-time performance of mobile service robots, this paper presents a low-cost mobile service robot hybrid controller project. The main features are as follows: 1) This hybrid control project which implements the Android device as the host controller and the management-oriented multi-core heterogeneous micro-controllers as the co-controller achieves a "harmonious" control logic between the controller and co-controller. Designed some software for OTG, SPI, Socket and Wifi Direct communication. And it can effectively cut the cost of the mobile service robots. 2) Designed a multi-core heterogeneous controller including power supply module, control module, digital I/O module, sensor module and synchronous debugging module. And designed relevant soft wares which are downloaded to board. The test results are stable and accurate. Those results indicate the programs are correct and feasible. 3) Designed the two kinds of monitoring system, they are PC server based on wifi communication and Android Server based on wifi-direct communication. Monitoring server Implements many functions such as starting module, sensor data display module, communication module, control module and so on.

Acknowledgment This work was supported by Industrial Key-research of Shanxi Province, China (2016GY-003) and the Fundamental Research Funds for the Central Universities of China (No. 310825171004).

References [1] Kezhi Wang, S. Liang, H. Bi, X. Xian, "Implementation of a robot inspection system for substation equipment based on pioneer 3-at[J]," ICIC Express Letters, vol.01, pp. 221-226, 2011. [2] Hanafiah Yussof, Muhammad Fahmi Miskon, Ilmi Mohd Ariffin, Abdul Rahman Omar, "Sensor Based Mobile Navigation Using Humanoid Robot Nao[J]," Procedia Computer Science, vol. 76, pp. 474-479, 2015. [3] Safdar Zaman, Wolfgang Slany, Gerald Steinbauer, "ROS-based mapping, localization and autonomous navigation using a Pioneer 3-DX robot and their relevant issues[C]," Riyadh Saudi arabia: Saudi International Electronics, Communications and Photonics Conference 2011, SIECPC 2011, 2011. [4] Stern C, Adelt P, Schmitz M, "An Infrastructure for FlexibleRuntime. Reconfigurable Multi- microcontroller Systems[C]", Australian Comp Soc (ACS), Brisbane: 21st IFIP World Computer Congress (WCC), 2010. [5] Wang Guo-qing, Guo Miao, Li Zhe, Zhang Xiangwei, Yin Zhouyang, Mu Dong, "Study on Multi-core Heterogeneous Programmable Controller System of Construction Machine[J]," Journal of Chang'an University (Natural Science Edition), vol. 06, pp. 141-144,2015. [6] Mahendra O, Syamsi D, et al. Design of a Dual-microcontroller Scheme to Overcome The Freeze Problem for a Smart Data Logger[C]. Bandung Indonesia: 2nd International Conference on Information and Communication Technology (ICoICT), 2014. [7] Jimenez-Gonzalez A, Martinez-de Dios JR, Bernabe AD,"A Remote Testbed for Ubiquitous Robotics Experimentation[J]," Revista Iberoamericana de Automatica E Informatica Industrial, vol. 01, pp. 68-79, 2014. [8] Vyas S, Gupte A, Gill C D, "Hardware architectural support for control systems and sensor processing[J],"ACM Transactions on Embedded Computing Systems (TECS), vol.02,pp.16- 19,2013. [9] Scheper KYW, Tijmons S, de Visser CC," Behavior Trees for Evolutionary Robotics[J]," Artificial Life, vol. 01, pp. 23-48, 2016. [10] Abou Haidar G, Achkar R, Abou Dayya R," Remote Generator Control Using Android Application[C]," Taipei:Asia Modelling Symposium 2014 8th International Conference Mathematical Modelling Computer Simulation, 2014. [11] Bergesio Luca, Bernardos Ana M, Marquinez Inigo, "PERSEO: A system to Personalize smart Environments through Smartphones, sEnsors and media Objects[J],"Journal of Ambient Intelligence and Smart Environments, vol. 06, pp. 777-804, 2015. [12] Saquib SMT, Hameed S, Hameed S, "Wireless Control of Miniaturized Mobile Vehicle for Indoor Surveillance[C]," Indus Univ: 1st International Conference on Sensing for Industry, Control, Communications, and Security Technologies (ICSICCST), 2013. [13] Zhang JB, Bao KY, Liu X, "The Intelligent Home System Controlled by SMS[C]," Beijing: International Conference on Advanced Computer Science and Electronics Information (ICACSEI),2013. [14] Daud P, Heriana O, Rahman AN, "Concept Development of Integrated Monitoring System Model to Support Activities Monitoring in the Border Region[C]," Kuala Lumpur MALAYSIA: International Conference on Smart Sensors and Applications, 2015. [15] Teng KH, Lam ZY, Wong SK, "Dimmable WiFi-Connected LED Driver with Android Based Remote Control[C]," Kuching MALAYSIA: IEEE Symposium on Wireless Technology and Applications (ISWTA), 2013.