Towards Net-Centric System ofSystems in Air, Sea and Land

Ted Shaneyfelt, Sevki Erdogan, Azim Maredia, Gnanadeep Vemuri, Bhargavaram Pachala, Dong Yue, Sohel Karovalia, Ming-Zhu Lu, Shi-Zhong Yang, and Chenyu Gao Autonomous Control Engineering Center Electrical and Computer Engineering Department The University ofTexas at San Antonio, TX, USA

Near - The communication range is small and the can Abstract-This paper investigates how we can work towards only communicate with nearby . building net-centric swarms of land, sea and air robots working INF - The range is infinite and the robot can communicate together to accomplish a common goal. The goal is to bring with any robots in the working space regardless of the together swarms of robots from all three sectors to safely benefit physical size ofthe working space. mankind without increasing dangers in the process. Several platforms are explored for simulation to investigate swarm Communication Topology - Although a robot could robotics within heterogeneous environments. communicate with others randomly and arbitrarily, a structural rule is generally followed, describing a Communication Index Terms-Mobile robots, Net-centric, Simulation software, Topology. System ofsystems BROAD -Broadcast. A robot can send a message that is received by many robots. I. INTRODUCTION ADD - Addressable. A robot communicates with other HIS document identifies Network Centric System of robots using their particular addresses, which are typically TSystems of robots within a brief taxonomy of robotics predetermined. and , investigates some potential applications TREE - A robot can only communicate through a for such systems, and considers a road-map towards hierarchy. realization. Recent progress has been made in research GRAPH - Communication between robots follows some towards Net-Centric System of Systems of fault-tolerant other graph pattern. sensor networks [1] and robotic swarms. Net-Centric does not Communication Bandwidth: Data connections by cable imply totally autonomous systems, but rather using allow massive amounts of information to be transferred in a networking to enable people and systems to work together very short time. Conversely, wireless signals by submarines productively [2] and safely. A primary concern not covered in using sound waves carry only a small amount of data over a this paper is ethical use, which includes rights to privacy, relatively long period oftime. Regardless, some robots need to safety, etc. We overview some ofthe robotics developments at stop other processing in order to communicate due to limited the Autonomous Control Engineering lab at The University of processing resources. Bandwidth characterizes these aspects of Texas at San Antonio, and evaluate several robotics suites for communication cost. simulation. INF - Bandwidth limitations are negligible so that all desired communication is fast and easy. A. Robots MOTION - Robots must divert valuable processing A number ofresearchers have taken on the task ofdevising resources from other tasks to handle communication, so that taxonomies for robotics [3]-[8]. The taxonomy devised by communication is relatively expensive in terms of Dudek et al. follows [7]. processing. Collective Size - The number ofrobots in the workspace LOW -Bandwidth is narrow making communication ALONE -A single robot working alone in the workspace difficult or very expensive in terms ofprocessing. PAIR - Two robots working together in the workspace ZERO - No communication bandwidth is available. Lim - A limited number ofrobots in the working together Collective Reconfiguability - The ability of a Collective to INF -An unlimited number ofrobots working together reorganize its spatial configuration Communication Range - A robot cannot communicate with STATIC - Static arrangement; the topology is fixed. others outside ofits communication range. COMM - Coordinated rearrangement NONE -The robot cannot communicate with others under DYN - Dynamic rearrangement any condition. Processing Ability - Each unit ofthe Collective has a model ofprocessing and computation ability.

978-1-4244-2173-2/08/$25.00 ©2008 IEEE SUM -linear summation unit [9] one leader for each assigned task. Weakly centralized systems FSA - Finite state utilize multiple master robots. We are primarily concerned PDA - Push-down automaton with weakly centralized systems that are capable of self­ TME- Turing machine equivalent; this computational organization, yet respond to higher level control. model is assumed by most robotic systems. We extend this classification by considering Net-Centric Collective Composition - Three important compositions in swarms as a subclass of'weakly-connected robots. Net-centric the Collective. swarms are capable of self-organization and discovery of one IDENT - Identical; each unit of the collective is another, yet capable of multiple levels of direct remote homogeneous in both form and function, i.e., in both operator intervention. We are specifically interested in hardware and software. coordination of systems, which involves land, sea and air HOM - Homogeneous; all of the units of the Collective based swarms of robots. Each of swarm is itself a complex, essentially have the same physical characteristics. independent system, yet all of them are required to work HET - Heterogeneous; the units of the Collective have together as a coordinated System ofSystems. different physical characteristics. General speaking, this also implies difference in the physical behavior. II. ApPLICATIONS OPPORTUNITIES FOR SWARM ROBOTIC SYSTEM OF SYSTEMS B. Swarm Robots Alessandro Farinelli et aI., [2] classify of swarm A. Steps towards realization robotics based on several levels: Cooperation level, The first step towards realization is to develop actual robots knowledge level, coordination level and organization level. in the lab on a small scale to discover physical characteristics This organization is shown in Fig. 1. and technologies that cannot be so easily simulated. This is Swarm robotics is distinguished from robotics in general by already well underway at the UTSA Autonomous Control the cooperation of robots to perform a task. Classification of Engineering laboratory with land and sea based robots, with swarms as cooperative robotics is the first level in the some initial work on air based robots. Small swarms ofrobots hierarchy. are being investigated with real equipment, while scalability The second level in hierarchy deals with knowledge will be tested with simulations based on the knowledge gained possessed among the robots. Robots are either aware of other working with real robots. robots in the swarm, or unaware. Awareness allows robots to After proof-of-concept is sufficiently accomplished with cooperate well with increased efficiency. At this level, we are small swarms of real robots, simulation can be used to test primarily concerned with swarm robots that are aware of one scalability to many robots working together as a net-centric another. System of Systems. Economically it is not feasible to The third level deals with coordination, which distinguishes experiment with such large scale System of Systems in early whether or not robots act in consideration of the actions of stages of research. There is also some risk of actual robots other robots. Robotic swarms can be classified according to experiencing damage during testing in a real environment, the strength of their coordination, which is affected by the which is avoided when doing simulations. Simulation can reliability of the coordination protocol. At this level, we are provide a safe environment where correct programming can be primarily concerned with strongly coordinated robotic verified before implementing some algorithms directly on swarms. robot. A fault in programming real robots could cause damage Swarms are further classified into centralized and to robots or their environment. Some tuning can be decided distributed organization. In distributed systems each robot acts during simulation so as to reduce the likelihood ofcollision. A and makes decisions autonomously. On the other hand, ifthere simulation can provide results that are not easily is a master robot in charge of the swarm, it is called a experimentally measurable with currently available centralized system. Strongly centralized systems have only technology. Testing of robots for long periods of time can cause wearing of robots which can be prevented using Cooperation simulations. There are some drawbacks to relying on simulations as well. For example, the result of simulations may not be always accurate as expected in real time. This Knowledge calls for both simulation and development ofphysical robotics in an integrated plan.

Coordination Simulation is to be carried out to demonstrate the use of robotics in a net-centric System of Systems to profit mankind. We considered many applications, taking into account Organization negative consequences as well as benefits. For example, sending robots to dig through rubble after a major disaster in search for survivors could cause undue harm to people who INet-Centric I are trapped in the rubble ifthe robots are less able to adapt to Fig. 1. A taxonomy for swarm robotics people's cries for help or verbal cues from the disaster victims. Adapted from. Jacoffet aI., (2000)

Cold Zone Commandpost rest &: recovery 1t Fig. 3. Coast Guard Deep Water Program 1tf$ human rescue workers. For this reason, use ofrobotic swarms is potentially most helpful for covering large areas in Fig. 2. Urban Search and Rescue searching prior to the use ofhuman rescuers with more limited robotic assistance for more sensitive tasks in areas where Alternatively, overly-centralized fully-automated military or human rescue teams can safely work. police forces were considered, and the likelihood that they . Coast Guard Integrated Deep Water Program might be hijacked or misused for harm or tyranny would also be a regrettable possible consequence. The Coast Guard patrols the coast and maintains We have found far safer applications. Some applications navigational markers such as buoys and lighthouses, performs have been suggested by other researchers; other applications search and rescue operations, and patrols the waterways and involve automating existing Systems ofSystems; and we have coastal borders. An important part ofthe coast guard program also considered novel ideas ofour own. We examine a sample is to communicate and co-ordinate with each other and work of each of those categories of applications: application together. The Coast Guard Deep Water Program, illustrated in suggested by other researchers, application in the field, and Fig. 3, is an existing System of Systems that operates in air, novel application ofour own. sea, and land. This System of Systems has many tasks that could B. Urban Search andRescue potentially benefit from the use of swarm robotics. In fact, One possible application of robotic swarms is their robotic drone aircraft are already being introduced. Robots can proposed use in Urban Search and Rescue (USAR). Jacoff et cover large areas efficiently, and to go where it is unsafe for al. defines USAR as follows: "USAR is concerned with rescue human activity, or where it is impractical to achieve the activities in collapsed building or man-made structures after desired level ofpatrolling. catastrophic events, such as an earthquake or bombing" [11]. D. Harvesting Natural Resources Murphy et aI., suggest the following scenario. Make use of land, aerial and!or aquatic robots coordinating together to Manganese nodules are naturally occurring mineral deposits search for and rescue people in dangerous environments. This on the ocean floor [13], [14]. They are typically several inches scenario is shown in Fig. 2. When a disaster occurs, in diameter, and they contain metals that could be harvested Unmanned Air Vehicles (UAVs) inspect the area under commercially. Mining of these metals by divers is not consideration and send collected data to the command center. practical because of the depth at which they are located and Then the command center divides the disaster affected area the dangers and expense associated with such dives. However, into three zones: hot zone, warm zone and cold zone. The hot Autonomous Underwater Vehicles (AUVs) could be sent in zone is the area where the disaster actually occurred. The warm zone is the nearby area where robots can rendezvous. Warm zone can be defined as the area where there is the possibility ofhuman rescue teams to act without a high risk of danger. The cold zone is the surrounding area for repairing and recharging robots. The cold zone has the lowest possibility of danger. In this zone, rescue teams can put together control units to control the robots in hot zone and cold zones. After an initial aerial survey, land robots enter the dangerous area out ofsafe reach ofhuman rescue teams. Once survivors are found, they are recovered by the robots to a safe area and then transported to a nearby hospital [12]. Use of robotic swarms for search and rescue must be carefully thought out so that the robots do not become a distraction to emergency workers or unintentionally injure Fig. 4. Harvesting manganese nodules. people in distress by being less perceptive and adaptive than swarms to the seabed to search for and harvest nodules, as gna"" Microsoft VIsual Progrc1l11mlng Language I~~ If! rX shown in Fig. 4. As the nodules from the seabed are recovered, they would surface, where they could contact other vehicles such as Unmanned Air Vehicles (UAVs) to pick up the goods and to provide fuel for the submarines. The UAVs could pick up the nodules and deliver them to land-based operations, such as awaiting Unmanned Land Vehicles (ULVs) which would be able to deliver them to processing

<.ArosBlJmper facilities that might be collocated to energy sources several .Arol5Core miles inland. Geothermal energy is available on the slopes of 7" - active volcanoes such as Mauna Loa on the island of Hawaii. Fig. 6. Microsoft Robotics Studio Visual Programming Language Energy could be harnessed to provide not only electricity and mechanical power, but it could also transform abundant rainwater into hydrogen fuel. This scenario combines the use V. MICROSOFT ROBOTICS STUDIO of swarms of robots in land, air, and sea to benefit mankind while minimizing dangers. Feasibility would depend upon Microsoft Robotics Studio is not nearly as portable as many factors, including potential political implications in URBI, but it is provided for free by Microsoft. A decentralized dealing with emerging groups like the International Seabed system services based architecture allows for distributed Authority. Regardless, this System ofSystems application was wireless or web-based . selected as the primary long-term goal of our initial swarm A. Programming Languages robotics simulation effort. Microsoft Robotics Studio supports numerous text oriented languages such as C#, Visual BASIC, C++, and Python. III. ROBOTIC SWARM SIMULATION B. Visual Programming Language Several robotics simulation software suites are available to simulate robotics. As such, we investigated three of today's Microsoft Robotics Studio provides Visual Programming most prevalent robotics suites: Universal Real-time Behavior Language (VPL), reminiscent of National Instruments' Interface (URBI), Microsoft Robotics Studio and Player LabView for the novice or for the inexperienced programmers. Project. Each of these free tools is described here, and a This programming is done graphically. VPL programs mainly comparison is given in the conclusion of the paper. We are consist of boxes and arrows similar to flow-charting or data also aware of commercially available software such as flow diagrams. Fig. 6 shows the programming environment WebBots, which we did not include in this evaluation. editing a VPL program. C. Simulation IV. URBI Simulation with Robotics Studio takes advantage of any URBI is a cross-platform software suite. It runs on , AGEIA PhysX accelerator installed. Code for robots and run Mac, Windows, and other operating systems. URBI can be on either the simulator or on the actual robots. The simulation used to easily program parallel control of existing robots allows many virtual robots to be effectively built and such as Sony Aibo and . These are controlled in a life-like 3-D environment. seemingly great advantages ifURBI can be used for all our needs. However, 3D simulation ofcustom robots is a major VI. requirement that it lacks. This important disadvantage was The Player Project is a platform for robotics enough to outweigh the advantages of URBI for our and sensor systems. The Player Project is purposes. Fig. 5 illustrates the URBI architecture. URBI can composed of the Player network server and the Stage 2-D be programmed in Java, C++, and Matlab. simulator or Gazebo 3-D simulator. Player Project has traditionally been perhaps the most used robotics interface in research and education. Major intelligent robotics journals and conferences regularly publish papers featuring real and simulated robotic experiments using Player, Stage and Gazebo. Player Project is written primarily for UNIX, with support URBI Component for distributed systems, which makes it especially well suited c++ Class for UNIX-based supercomputer servers such as Huinalu "", Plugin .. operated by the University ofHawaii at Hilo. " Player supports multiple client connections over a socket Embedded Remote based interface, so it is well-suited for distributed processing. URBI Engine URBI En ine Player/Stage is available for free from SourceForge at this Fig. 5. URBI Architecture address: http://Playerstage.sourceforge.net shown in Fig. 8.

VII. HUINALU SUPERCOMPUTER The large numbers of weakly-centralized robots in a Net­ Centric system ofswarms in all ofthese environments requires considerable computing power for real-time interaction in simulation. A possible test-bed to enable human interface testing is a setup backed by supercluster to handle simulation ofthe many robots. Huinalu is a 520-processor IBM Netfinity Linux Supercluster. It consists of260 nodes, each housing two Pentium III 933 megahertz processors. Their combined theoretical peak performance is around gigaflops. Around half • a decade ago, Huinalu was the world's most powerful Linux Cluster. It has been decommissioned from The Maui High Performance Computing Center and currently it is housed in the Computer Science Department ofthe University ofHawaii o at Hilo. The original system has been modified to include a o Myranet communication interface to improve communication in the cluster. The Huinalu will be used to port simulations into a parallel environment and increase the scope of the research. Multiple Command and Control stations could be linked with the supercluster for user interaction evaluation.

Tlm@: 0 0:04:35.800 (slrrt@a :0.93) sLbs: 0 Stage \"'::;;.).3 VIII. RESULTS Fig. 7. Player Project Simulation by Dong Yue Table I shows a comparison ofthe robotic development and A. Programming Languages simulation suites that we investigated. TABLE I Player has a Transmission Control Protocol (TCP) socket­ UNITS FOR MAGNETIC PROPERTIES (SHORT TITLE HERE) based client. It uses a server model, where robot control Player Microsoft URBI programs can be written in any programming language Project Studio including C, C++, Java, TCL, and Python. As a result, Player Simulation Tool Built-In (Stage Built-In Marilou can work on virtually any computer which is connected to 2-D, and Robotics Gazebo 3-D) Studio robots with a network. Land/Sea/Air Land Land (plus)* Land Existing Robot Pioneer2DX, Lego Pioneer, Lego, Web B. Simulation Models Pioneer2AT etc. shots, Korebut, We simulated Player Project in the BSEO.236 lab, as our and Aibo, regular lab did not have UNIX available. UBANTU was used SegwayRMP Mindstorm, etc. etc. as the . Results ofthe simulation are shown in Platform UNIX-like OS Windows UNIX, MAC, Fig. 7. The dot indicates a robot position, and the solid black (including Windows. lines indicate obstacles. Linux, Mac OS X, etc) The command line input and output for a simulation is Languages Java, C++, C, C#, C++, VPL, Java, C++, Stage driver creating 1 device TCL, Python, Net.net, Iron Matlab mapping 6665.31.0 => Simulated world etc. Python, VB, pioneer. inc] [Include map. inc] [Include si etc. Reuse Software on Software on Software on o hardware hardware hardware name positionlines state 0 o Multiple Robots Yes Yes Yes name positiontext state 0 o Multiple Robot Yes Yes Yes Comm. name ranger_data state 1 1 Operator Control Yes '" Yes Yes name ranger_cfg state 0 o *Researchers at UTSA Autonomous Control Engineering laboratory are name laserdata state 1 1 currently extending Microsoft Robotics Studio to simulate underwater and name lasercfg state 0 o air environments. All three tools provide a good set of language interfaces, Stage driver creating 2 devices and allow simulation of multiple robots together. Each tool mapping 6665.4.0 => "robot1" mapp ing 6665. 6 . 0 => rrrobot.1. laser: 0 rr provides a set of existing robot models, though they all are Listening on ports: 6665 designed mainly for land-based robotics. The lack ofa built-in simulation studio where we could easily design our own Fig. 8. Portion of environment setup (input and output) from Player Project command line interface swarms overshadowed the many benefits of URBI. Good built-in simulation tools would make this project worth reconsidering. Although Microsoft Robotics Studio is [9] John Hertz, Anders Krogh, and Richard G. Palmer (1991). Introduction Windows-centric, it provides an impressive set oftools built­ to the Theory ofNeural Computation. Lecture Notes Volume I. Santa Fe Institute. Studies in the Sciences of Complexity. Addison-Wesley in and makes good use ofavailable acceleration technology on Publishing Company, Redwood City CA. the pc. For larger scale simulation, it may be necessary to [10] Gregory Dudek, Michael r. M. Jenkin, Evangelos Milios, David Wilkes, consider Player Project, which provides multiple models for 2 "A Taxonomy for Multi-Agent Robotics", Autonomous Robots, Springer Netherlands, Volume 3, Number 4/ December, 1996 and 3 dimensional modeling in simulation. [11] Adam Jacoff, Elena Messina, John Evans, "A Standard Test Course for Urban Search and Rescue Robots" Proc. Performance Metrics for IX. CONCLUSION Intelligent Systems Workshop, August 2000. [12] Robin Roberson Murphy, "Human-Robot Interaction in Rescue We have worked with three robotic development and Robotics", IEEE Transactions on Systems, Man, and Cybernetics-part simulation systems, and evaluated their usability for modeling C: Application and Reviews, vol. 34, No.2, May 2004 [13] Jenkins, Raymond W.; Jugel, M. Karl, et al The FeaSibility and net-centric robotic System of Systems. We have found that Potential Impact of Manganese Nodule Processing in the Puna and although no one solution is optimal, Microsoft Robotics Kohala Districts of Hawaii, Washington D. C. Hawaii Dept. Of Studio seems best for simulation ofunderwater environments, Planning & Economic Development and US Dept. Of Commerce, NOAA. 1981. while Player Project is well suited for running on a Linux [14] Stephen-Hassard, Q., Grabbe, E., "Hawaii and the manganese nodule Supercluster. Although the portability of URBI gave a good industry", IEEE Oceans" Volume: 9, Sep 1977 initial impression, it is sufficiently lacking in development and simulation tools to rule it out as our development platform. Our hope is that the Player Project can eventually be extended to simulate underwater and air environments in a similar manner as Microsoft Robotics Studio, yet Microsoft Robotics Studio maintains advantages in physics engine acceleration support.

ACKNOWLEDGMENT We thank Dr. Mo Jamshidi for giving us the opportunity to work together and collaborate on this project, and for his guidance, direction and support along the way. We also thank Matthew Joordens, Jefferey Prevost, Kranthi Manoj and Siu Ying Shaneyfelt for their influence and assistance to various authors.

REFERENCES

[1] Prasanna Sridhar, Asad M. Madni, and Mo Jamshidi, "Hierarchical Aggregation and Intelligent Monitoring and Control in Fault-Tolerant Wireless Sensor Networks", IEEE Systems Journal, 2007 [2] Robert 1. Cloutier, Michael J. DiMario, Hans W. Polzer, "Net-Centricity and System of Systems", System of Systems - Innovation for the 21st Century, Chapter 9 Wiley & Sons 2008 [3] As'ad Salkham, Raymond Cunningham, Aline Senart and Vinny Cahill. "A Taxonomy of Collaborative Context-Aware Systems". Presented at the Ubiquitous Mobile Information and Collaboration Systems (UMICS'06)-CAiSE'06 workshop, Luxembourg, Grand-Duchy of Luxembourg, 5-6 June, 2006 [4] Alessandro Farinelli, Luca locchi, and Daniele Nardi, "Multirobot Systems: A Classification Focused on Coordination", IEEE Transactions on Systems, Man, and Cybernetics-part B: Cybernetics, Vol. 34, No.5, October 2004 [5] Brian P. Gerkey, Maja J Matari'c, "A formal analysis and taxonomy of task allocation in multi-robot systems" In IntI. J. of Robotics Research 23(9):939-954, September2004 [6] Holly A. Yanco, Jill Drury "Classifying Human-Robot Interaction:", An Updated Taxonomy" Classifying Human-Robot Interaction: An Updated Taxonomy. Proceedings of the IEEE Conference on Systems, Man and Cybernetics, October 2004 [7] G. Dudek, M. Jenkinj E. Milios! and D. Wilkest, "A Taxonomy for Swarm Robots" Proceedings of the 1993 the IEEE/RSJ international Conference on Intelligent Robots and Systems Yokohama, Japan July 26-30,1993, [8] Ronald. C. Arkin and Tucker. Balch, Cooperative multiagent robotic systems in AI-based Mobile Robots: Case Studies ofSuccessful Robot Systems, eds., D. Kortenkamp, R. P. Bonasso, and R. Murphy, MIT Press, (1998).