A Proposed Hardware and Software Architecture for a Robotic System Gutemberg S

1 A Proposed Hardware and Software Architecture for a Robotic System Gutemberg S. Santiago and Adelardo A. D. Medeiros

Abstract—To manage the procedure of capturing, processing, database maps, computational models response, status situa- and delivering all sensory information acquired by a complex tions achieved, and it needs to delivery some action to mobile robotic system, a robot architecture is needed. This class of robots devices of the robotic system. The software architecture man- deals with concurrent embedded real-time performance and intelligent algorithms at same time. The development of a generic ages the relationship among these decision algorithms. The robotic architecture to complex robotics systems has mainly two hardware architecture manages the sensors capture process challengers: the complex level of the robotic systems; and the and their configuration and the delivery process of decision hardware diversity that robots are built. To deal with these information to the actuators movement. problems a distributed hardware and software architecture for Due to hard sensory information processing, a complex robotic systems was developed using communication standards. This architecture aims to manage different robots and provide robotic system needs a distributed computational processing. a distributed communication system which allows the robots These systems may have many embedded computers to pro- communicate among themselves and with a control base station. cess the information. The subsystems that need hard real- This proposed architecture is based on CAN Network and ROS time processing use a separated embedded computer, since Framework using Orocos RTT Framework, and it is being they consume all the CPU power processing and need a implemented on a mobile robot at initial stage of building. real-time operational system. Other subsystems hold only a Index Terms—Robot Architecture, Complex Robotic Systems, system monitoring tasks and use a single computer with no Embedded Systems, CAN Network, ROS Framework, Orocos restrictions. RTT Framework. Robotic systems generally communicates with a base station or with other robots. A robotic architecture has to handle with I. INTRODUCTION this distributed robotic system configuration. A base station can define the tasks of the robot, environment restrictions, ROBOTIC systems are mechatronic devices capable of control and monitoring. A robot team is a robotic system R performing autonomous tasks. These systems combines configuration which robots communicate with each other and mechanics, electronics, computing and control engineering joint efforts to perform a task. with a certain degree of autonomy, based on some heuristic The general control system for a robotic system with a or artificial intelligence, to make possible the generation of a certain degree of autonomy and complexity is required to reliable and versatile robotic system. meet some behavior specifications and design requirements: Complex robotic systems are a class of robots which deals reactivity to the environment; intelligent behavior; resolving of with concurrent embedded real-time performance and intelli- multiple goals; global reasoning; reliability; and adaptability gent algorithms at the same time. The complexity level on [1] [2]. the development of these robots is based on the hard real- The development of a new architecture has mainly two time performance needed, on the slow intelligent algorithms challenges: the complex level of the robotic systems and the processing, and on the data exchange among software and hardware diversity that robots are built. hadware processes at robot. The complexity level of the robot development has grown. The degree of autonomy of these systems is directly related The architecture needs to be capable to deal with many to the amount of acquired sensor information. The robotic characteristics compatible to the more powerful computer systems take decisions based on sensory information. The (nowadays embedded computers), the more compact electronic more sensory information is processed, the more the robot circuits and new kinds of processed data sensors. Sensor like is capable of performing tasks. If this sensory information is lasers and cameras produce rich information after the data obtained from different kinds of sensors and have a certain processing. degree of redundancy, all these information can be fused to Considering the robot architecture point of view, robots can obtain a more accurate system performance. be classified as having a basic mechatronic system or having To manage the procedure of capturing, processing, and a complex mechatronic system [3] [4]. A basic mechatronic delivering all sensory information acquired, hardware and system is generally characterized for one arm robotic system software architecture are used. The decision algorithms pro- or a basic mobile robot. Generally these robots have no time cessing needs to be fed from sensory information, robot restrictions. A complex mechatronic system is usually characterized for a mix of basic mechatronics systems. Generally The authors are with the Department of Computing and Automation Engineering, Universidade Federal do Rio Grande do Norte, Natal, RN, Brazil these robots have strong time restrictions (Figures 1 and 2). (e-mail: [email protected]; [email protected]). Some robotic systems present special architecture capabil- 2

hardware and software architecture for autonomous complex robotic systems.

II. ARCHITECTURE CONCEPTS The main objective is to develop a distributed hardware and software architecture for robotic systems. This architecture aims to manage any kind of robot: underwater, surface, terrestrial, aerial and spatial. This architecture will provide a distributed communication system which will permit the robots to communicate among themselves and with a control base station. The specific objectives are: • to ensure a set of characteristics that the architecture of Fig. 1. Autonomous Complex Fig. 2. Autonomous Complex all the built robots will provide. For example, the real- Robotic System: ASIMO Robot from Robotic System: Justin Robot from n Honda (Picture from website [5]). DLR (Picture from website [6]). time communication bandwidth guarantee for a number of dispositives plugged into the robot; • to manage the interface between the real-time require- ities. An example is a fast reaction that a visual system is ments of the manipulation tasks of the robot and the non- capable: to shoot a cellphone and hold it back again. This realtime computational intelligent process of the decision is demonstration of skillful manipulation using a high-speed algorithms; robot system (Figure 3) [7]. • to standardize the communication interfaces among the robot dispositives and to standardize the communication interfaces among the main parts of the robot. • to develop an architecture capable of integrating and validating new technologies, such as different kinds of actuators and sensors. The methodology used to develop this new architecture was divided into two parts: the hardware part including the sensors, Fig. 3. Regrasping task (from left to right). Ishikawa Komuro Lab’s high-speed robot hand performing impressive acts of dexterity and skillful actuators and real-time network management; and the software manipulation. The system consists of visual and tactile sensors at a rate of part including the distributed manager base on the hybrid 1kHz and a high-speed hand-arm manipulator. Pictures from [7]. Ishikawa deliberative-reactive paradigm. Komuro Laboratory, Department of Creative Informatics, University of Tokyo. The hardware development methodology is to concentrate different types of sensors and actuators in a real-time industrial The lack of standards in the robots development makes network. The sensors and actuators are plugged into the each building different from each other. Robots have different industrial network through an interface board based on a physical characteristics and different hardware architectures. micro-controller. All dispositives will provide self-information In addition redundant infrastructure for robotic platforms is based on a sensor intelligent standard, such that when a developed with no interoperability among these platforms. No dispositive is plugged in the network, it negotiates with the framework is developed to capture new technologies. bus administrator manager and became available if previously Currently there is a relevant discurssion among the scientists approved. The external communication will be provided by of robots architecture to develop a hardware and software an Ethernet network and will permit the communication with robot standardization, once each robot is built with its own different types of robots. The vision system will be based on hardware and software. With the financial support of the specific image hardware processing. European Commission and the Robot Standards and Reference Architectures Consortium, the RAS Standing Committee on Standardization Activities has established a study group in III. ARCHITECTURE COMPONENTS 2007 and has organized a series of open workshops and The hardware elements used to compose the proposed expert meetings to address the questions of measurability, architecture are the IEEE 1451 protocol for smart sensors comparability, interoperability, and reusability of architectural which interfaces the robot sensors and actuators to the CAN concepts and components in robotics [8]. network; the Control Area Network (CAN) employed as the This paper is structured in the following parts: Section main internal bus of the robot; the Xenomai used as the real- Architecture Concepts lists the objectives to be reached and time operational system of the robot; the SocketCAN that is describes the methodology used to implement the architecture the way the CAN is accessed as a network card; the publish- concepts; Section Architecture Components presents the used subscribe that manages the messages in the CAN network; the hardware and software elements that compose the proposed ar- API used for real-time publish-subscribe over ethernet; and chitecture; and finally Section Proposed Architecture presents the Field-Programmable Gate Array (FPGA) used to process a methodology and the tools used to develop a distributed camera image in hardware. 3

A. Control Area Network (CAN) bus is done analogue to the use of the Internet Protocol CAN is a multi-master broadcast serial bus standard for con- via Sockets. Fundamental components of SocketCAN are the necting electronic control units (ECUs). The devices that are network device drivers for different CAN controllers and the connected by a CAN network are typically sensors, actuators, implementation of the CAN protocol family. and other control devices. These devices are not connected directly to the bus, but through a host processor and a CAN E. ROS Framework controller. The robot software is base on the ROS (Robot Operating Each node is able to send and receive messages, but not System), an meta-operating system for robots. It provides the simultaneously. A message consists primarily of an ID - services like hardware abstraction, low-level device control, usually chosen to identify the message-type or sender - and and message-passing between processes. up to eight data bytes. It is transmitted serially onto the bus. ROS is composed by the operating system commands that This signal pattern is encoded in NRZ and is sensed by all manage the packages, and a suite of user contributed packages nodes. (organized into sets called stacks) that implement functionality Bit rates up to 1 Mbit/s are possible at network lengths such as simultaneous localization and mapping, planning, below 40 m. Decreasing the bit rate allows longer network perception, simulation etc. distances (e.g., 500 m at 125 kbit/s). The CAN data link layer protocol is standardized in ISO F. Orocos 11898-1 (2003). This standard describes mainly the data link Orocos [11] is the acronym of the Open Robot Control layer - composed of the logical link control (LLC) sublayer Software project. The project’s aim is to develop a general- and the media access control (MAC) sublayer - and some purpose, free software, and modular framework for robot and aspects of the physical layer of the OSI reference model. All machine control. The Orocos project supports 4 C++ libraries: the other protocol layers are the network designer’s choice. the Real-Time Toolkit, the Kinematics and Dynamics Library, the Bayesian Filtering Library and the Orocos Component B. Smart Transducer Interface - IEEE 1451 Library. A smart sensor is a device that handles its own acquisition The Orocos Real-Time Toolkit (RTT) is not an application and conversion of data into a calibrated result in the units of in itself, but it provides the infrastructure and the functional- the physical attribute being measured. A smart sensor would ities to build robotics applications in C++. The emphasis is do all that internally and simply provide a physical entity on real-time, on-line interactive and component based appli- number as data. Smart sensors do not make judgments on cations. the data collected unless that data goes out of range for the sensor. G. Publish-Subscribe IEEE 1451 (Standard for a Smart Transducer Interface for Publish-Subscribe is an asynchronous messaging paradigm Sensors and Actuators) is a set of Smart transducer interface which senders (publishers) of messages are not programmed standards developed by the IEEE Instrumentation and Mea- to send their messages to specific receivers (subscribers). surement Society’s Sensor Technology Technical Committee Rather, published messages are characterized into classes, that describe a set of open, common, network-independent without knowledge of what (if any) subscribers there may communication interfaces for connecting transducers (sensors be. Subscribers express interest in one or more classes, and or actuators) to microprocessors, instrumentation systems, and only receive messages that are of interest, without knowledge control/field networks [9]. of what (if any) publishers there are. This decoupling of publishers and subscribers can allow for greater scalability and C. Xenomai a more dynamic network topology. Xenomai is a real-time development framework cooperat- H. Real-Time Publish-Subscribe over Ethernet ing with the Linux kernel, in order to provide a pervasive, interface-agnostic, hard real-time support to user-space appli- The real-time publish-subscribe over ethernet is possible cations, seamlessly integrated into the GNU/Linux environ- using Linux operational system through the use of the RTnet ment [10]. and the ORTE. The Xenomai drivers has the RTDM (Real-Time Driver RTnet is an Open Soure hard real-time network protocol Model) which composes the following modules: CAN De- stack for Xenomai and RTAI (real-time Linux extensions) vices, Real-time IPC Devices, Serial Devices, and Testing [12]. It makes use of standard Ethernet hardware and supports Devices. several popular NIC chip sets, including Gigabit Ethernet. Moreover, Ethernet-over-1394 support is available based on the RT-FireWire protocol stack. D. SocketCAN The ORTE (OCERA Real-Time Ethernet) is open source The SocketCAN concept extends the Berkeley sockets API implementation of RTPS communication protocol defined by in Linux by introducing a new protocol family PF CAN Real Time Innovations. RTPS is new application layer protocol that coexists with other protocol families like PF INET for targeted to real-time communication area, which is built on top the Internet Protocol. The communication with the CAN of standard UDP stack [13]. 4

I. Field-Programmable Gate Array (FPGA) The vision system can be handled by an embedded computer or by a specific vision system hardware. The embedded com- A Field-Programmable Gate Array (FPGA) is an integrated puter configuration permits the communication with the CAN circuit designed to be configured by the customer or designer bus and the wireless ethernet bus, and then sends processed after manufacturing-hence ”field-programmable”. FPGAs con- image data through these networks. The processed image data tain programmable logic components called ”logic blocks”, is managed as an intelligent sensor, as well as the other sensors and a hierarchy of reconfigurable interconnects that allow of the robot. The image data also can be managed as an the blocks to be ”wired together”-somewhat like a one-chip intelligent sensor, then it is possible to send the image data programmable breadboard. Logic blocks can be configured to through these networks too. perform complex combinational functions, or merely simple The use of a specific hardware holds the need of a faster logic gates like AND and XOR. In most FPGAs, the logic image processing. Some image processing algorithms can be blocks also include memory elements, which may be simple implemented in hardware. A proposed hardware that can be flip-flops or more complete blocks of memory. used to process images through an easier visual processing algorithms is a FPGA. The difficulty of this approach is the IV. PROPOSED ARCHITECTURE needing to develop the interfaces with the FPGA. To turn the communication with the robot more reliable, it The proposed architecture is based on a CAN network and is desirable to have several ways of doing it. This hardware a Wireless Ethernet Network (Figura 4). In the CAN network architecture permit the inclusion of any new communication are plugged the sensors and the actuators of the robot. The system through the CAN interface. Wireless Ethernet network is used to communicate with other The base station communicates with the robotic system robots and a system base station. These two networks are thought the real-time wireless ethernet bus. The base station configured to work on real-time constraints. can control and monitor all the robot movements. It is possible record data from a robot task for later analysis, and change the programming of an internal component based on a micro- controller.

A. Sensors Interfacing A micro-controller makes the interface between the sensors and the CAN bus (Figure 5). Micro-controllers are a single central processing unit that offers interface to all kind of interfaces. The micro-controller can be programmed, providing Fig. 4. General view of the proposed architecture the the interface of embedded algorithms such PI, PD, PID, or fuzzy controllers. The use of the micro-controller permits the interface of Connected to the CAN network and to the wireless network either a custom or new device with with a new protocol there are embedded computers. These computers (many of (Figure 6). It is possible to build a custom own electronic them or only one) are the central processing unit of the robotic motor controller board and interface with the CAN bus. It is system. They form a distributed information processing center easy to communicate with any computer using a PCI-CAN since these computers are also connected among them through interface board. a real-time ethernet. They receive the sensor and actuators information from CAN network, process the information using a distributed processing architecture and communicate with the other robots either receiving the processed information, or requesting information to process. The system process running on embedded computer can be real-time or non-real-time. If a software module is real-time needed, it can be implemented as a real-time module, to use the real-time advantages. If the module of software is non- real-time, it can be implemented as a non-real-time module. The software architecture manages the relationship between the real-time and non-real-time process of software. Fig. 5. An example of electronic communication interface circuit between RS232 and CAN bus (Picture adapted from [14]). Each sensor and actuator has its own interface and its own communication protocol. The different interfaces and the communication are standardized using an interface board based on Sensors have different communication interfaces and proto- a micro-controller. The micro-controller which interfaces with cols and do not have any auto-descriptive information. Sensors the sensor, interacts with the sensor protocol, manages the generally use RS-232, RS-422, RS-485, I2C, SPI, and USB sensor using a standardized protocol of intelligent sensor, and interfaces. To standardize the sensor interface and also the interfaces with the CAN network. CAN message communication with any sensor, a protocol can 5

C. Real-Time Processing The architecture is based on Linux Operational System patch by Xenomai. The kernel patch is the version 2.6.25, which the socket CAN is already incorporated in kernel. The user programming mode interacts with Xenomai through the Orocos RTT. The wireless ethernet network is managed by RTnet, the real-time network protocol stack for Xenomai. This network stack uses the UDP transfer layer to guarantee the real- time constraint. To interact with the wireless network using a publish-subscribe system the ORTE is used over a real-time Fig. 6. An example of electronic communication interface circuit between a custom device and a CAN bus (Picture adapted from [14]). ethernet network.

D. Intelligent Processing be programmed into the micro-controllers. The protocol chose ROS framework is used as a main tool to develop cognitive was the IEEE 1451 for intelligent sensors. programming for the robotic system. ROS uses the publish- IEEE 1451 is a standard in development that can make subscribe message protocol to interact with all software pro- all complying sensors, instruments and systems work together cess running in the robot. with relative ease. There are some compelling reasons to use Using the ROS packages is possible to interface with some a plug and play standard for smart sensors: there is a huge of the main robotics avaliable frameworks and libraries. It is variety of sensors from a number of different manufacturers; possible to use the framework from the The Player Project the customization of different sensors in a data acquisition (Player/Stage/Gazebo)[17] to manage or simulate a robot; to system can be a very complex task; the IEEE 1451 systems use the OpenCV (Open Source Computer Vision) library [18] will greatly simplify the development and installation of smart to process some images; or to use the Orocos to apply some sensor systems; and the use of the IEEE 1451-6 is a standard data filtering. in cars. ROS uses the Orocos RTT package to interacte with real- A relevant problem that came from the robot architecture time processes. building is the lack of bandwidth information from sensors. Then it is not possible to calculate the exact load that the E. Robot Vision System architecture has to manage. To solve this problem, a commu- The need to make robots each time more capable of interact- nication standard has to interface the robot data manager and ing with the environment where they are has motivated the use the sensors. The IEEE 1451 systems resolve this problem since of visual information. The images taken by cameras attached an auto-descriptive and configuration process is available for to robots are rich in information that once well extracted, they the sensors based on this standard. can provide to the robot the ability to perform tasks that other types of sensors would not provide. The visual information can provide data to the robot to B. Internal Data Communications solve problems in order to enable it to perform various tasks. Classical robotics problems such as localization, navigation, The use of CAN network is justified because it allows the object recognition, interface with humans can be solved with implementation of a distributed communication system. Then the aid of a vision system. In the case of location and is possible to build architectures of several computers that can robots navigation, several scientific studies have approached access any device from the robot. This possibility allows the the problem and have proposed different solutions. implementation of obtaining data from any robot sensor by any The processing of visual information is a procedure of high computer at any time. The USB bus does not allows distributed computational cost. The larger the image, the greater the com- communication since it use the master-slave message protocol. putational cost. The processing of visual information generally The CAN network is noise robust, can be configured to be interferes with other processes running on the computer that deterministic, and can reach bit rates to 1 Mbit/s at network processes the image. lengths below 40m. It also can be used in a robot like a truck The implementation of image processing can be performed size. The sensor network can be physical distributed through in hardware in order to significantly reduce an image pro- the body of the robot with no problems. cessing time. This processing can be done in an FPGA. The The CAN network permits the configuration of a real-time processing algorithms will be modeled in the language of VDL system message [15] [16]. The interface with the real-time description hardware and implemented in a FPGA platform. CAN network can be done through the SocketCAN drivers Simulations can be performed using the software platform and API. The SocketCAN permits the programmer work with Quartus II 8.0 Altera Corporation [19]. the CAN network as an ethernet CAN. Then many computer The integration between hardware and the camera that process can access the network concurrently. obtains images is a difficulty in implementing systems with 6 images. A solution to this problem is to create an interface [16] U. Nunes, J. A. Fonseca, L. Almeida, R. Araujo,´ and R. Maia, “Using between the camera and the FPGA. This interface will allow Distributed Systems in Real-Time Control of Autonomous Vehicles,” Robotica, vol. 21, no. 3, pp. 271–281, May 2003. the procedure to acquire an image from the camera, to process [17] T. H. Collett, B. A. MacDonald, and B. P. Gerkey, “Player 2.0: Toward a it and to provide processing data in hardware. Practical Robot Programming Framework,” in Australasian Conference on Robotics and Automation (ACRA), Sydney, Australia, December 2005. V. CONCLUSION [18] OpenCV, “OpenCV,” January 2011. [Online]. Available: http://opencv.willowgarage.com/wiki/ This paper presented the concepts, methodology and tools [19] Altera, “Altera corporation,” 2010. [Online]. Available: used to develop a distributed hardware and software archi- http://www.altera.com tecture for autonomous complex robotic systems. The architecture was described in a general view focusing on the relationship among main elements. The hardware architecture based on a CAN and ethernet network was described showing the relationship between these two networks. The software architecture is based on ROS Framework using Orocos RTT Framework with the hybrid deliberative-reactive paradigm to conciliate the reactive behavior of the sensors-actuators and the deliberative activities required to accomplish more complex tasks.

REFERENCES

[1] A. A. D. Medeiros, “Controleˆ D’execution´ pour Robots Mobiles Au- tonomes: Architecture, Specification´ et Validation (Execution Control for Autonomous Mobile Robots: Architecture, Specification and Vali- dation),” Ph.D. dissertation, Universite de Toulouse III (Paul Sabatier), Toulouse, France, February 1997. [2] A. A. D. Medeiros., “A Survey of Control Architectures for Autonomous Mobile Robots,” Journal of the Brazilian Computer Society, vol. 4, no. 3, pp. 35–43, April 1998, special Issue on Robotics. [3] F. Pont, S. Kolski, and R. Siegwart, “Applications of a Real-Time Soft- ware Framework for Complex Mechatronic Systems,” in IEEE/ASME International Conference on Advanced Intelligent Mechatronics, Mon- terey, California, United States, July 2005, pp. 1324–1329. [4] B. Bauml¨ and G. Hirzinger, “When Hard Realtime Matters: Software for Complex Mechatronic Systems,” Robotics and Autonomous Systems, vol. 56, no. 1, pp. 5–13, January 2008. [5] ASIMO, “Asimo Robot from Honda,” January 2010. [Online]. Available: http://asimo.honda.com [6] IRM-DLR, “Institute of Robotics and Mechatronics - German Aerospace Center (Deutsches Zentrum fur¨ Luft- und Raumfahrt; DLR),” January 2010. [Online]. Available: http://www.dlr.de/rm/en [7] T. Senoo, Y. Yamakawa, S. Mizusawa, A. Namiki, M. Ishikawa, and M. Shimojo, “Skillful Manipulation Based on High-Speed Sensory- Motor Fusion,” in IEEE International Conference on Robotics and Automation (ICRA), 2009, video Section. [8] E. Prassler and K. Nilson, “1,001 Robot Architectures for 1,001 Robots [Industrial Activities],” IEEE Robotics & Automation Magazine, vol. 16, no. 1, pp. 113–113, March 2009. [9] IEEE.1451, “IEEE Standard for a Smart Transducer Interface for Sensors and Actuators - Common Functions, Communication Protocols, and Transducer Electronic Data Sheet (TEDS) Formats,” IEEE Std 1451.0- 2007, pp. 1–335, September 2007. [10] Xenomai, “Xenomai,” January 2010. [Online]. Available: http://www.xenomai.org [11] H. Bruyninckx, “Open Robot Control Software: The OROCOS Project,” in IEEE International Conference on Robotics and Automation (ICRA), 2001. [Online]. Available: http://www.orocos.org [12] RTnet, “RTnet (Hard Real-Time Networking for Real-Time LinuxXeno- mai),” January 2010. [Online]. Available: http://www.rtnet.org [13] ORTE, “ORTE (OCERA Real-Time Ethernet),” January 2010. [Online]. Available: http://www.ocera.org/ [14] R. V. Sousa, “Roboˆ Agr´ıcola Movel´ (RAM): Uma Arquitetura Baseada em Comportamentos Hierarquicos´ e Difusos para Sistemas Autonomosˆ de Guiagem e Navegaçao,”˜ Ph.D. dissertation, Escola de Engenharia de Sao˜ Carlos da Universidade de Sao˜ Paulo, Sao˜ Carlos, Sao˜ Paulo, Brazil, 2007. [15] L. M. Pinho and F. Vasques, “Reliable Real-Time Communication in CAN Networks,” IEEE Transactions on Computers, vol. 52, no. 12, pp. 1594–1607, December 2003.