Paper 1: a Network Test Bed for Highly Mobile Aerial Nodes

A Full-Scale Testbed for UAV and Ground Mobile Ad Hoc Networks Timothy X Brown, Sheetalkumar Doshi, Sushant Jadhav, Roshan-George Thekkekunel

Interdisciplinary Telecommunications Electrical and Computer Engineering University of Colorado Boulder, CO, 80309-0530 [email protected] July 31, 2004

Abstract

1 INTRODUCTION: From November 2003 and July 2004 we developed a full-scale physical testbed to study the performance of a mixed network of fixed, ground vehicle, and unmanned aerial vehicle (UAV) mounted wireless ad hoc network nodes, where the nodes are based on IEEE 802.11b wireless LAN cards. The goals of this test bed are three-fold. First, we wish to demonstrate the viability of such ad hoc networks to provide useful communication services. Second we want a tool for analyzing ad hoc networks in highly dynamic outdoor environments. Third we want to study the effects of the inclusion of UAVs in the network and how they can be used to further increase the network efficiency on the whole. In order to achieve these goals, we built a large-scale testbed with multihop communication distances up to 5km. The routing protocol is the Dynamic Source Routing (DSR) [reference] protocol implemented via Click [ref] in the Linux operating system. A monitoring agent runs on each node to record data on packets passing through the router. A robust monitoring collection architecture was developed to collect and view this data in real time. The data is also archived to allow post processing or viewing via a web-based visualization tool. The software runs on Linux laptops, and has been ported to other architectures including PDAs and a low-cost single board computer. The combination of DSR running on the single board computer with an 802.11 card and an RF amplifier we denote a mesh network radio (MNR). Some MNR have been packaged in environmental enclosures for outdoor use. Others are mounted in small UAVs that have been built for this purpose. This paper describes the testbed design in detail and our preliminary data. A full- scale series of tests are planned through October of this year which will provide a complete picture and will be included in the final version of the paper. The next section describes the motivation for building a full-scale testbed.

2 WHY HARDWARE TESTING IS IMPORTANT The vast majority of mobile ad hoc network (MANET) research has been based on theory and simulation. This testbed project has been labor intensive developing the networking components, preparing the physical testbed, and executing the experiments. Such efforts are needed in addition to simulation for several reasons:  It is difficult to model all the interactions, timing conditions, and effects of distributed peer-to-peer agents each running within a computer operating system, interacting through a complex adaptive wireless interface, over a highly variable radio environment. A hardware implementation exposes all of these issues.  The process of building the hardware test bed is instructive to learn the real challenges and limitations of MANETs.  The data collected can improve modeling for other researchers.  The hardware and software developed can be used as a basis for commercial, military, and scientific applications of MANETs. There have been other test bed implementations Our work adds several dimensions on these prior works. First, the testbed is large scale with distances spanning several kilometers. The larger distances enables better study of multihop behavior in natural environments and better shows the ability of MANETs to enable communication in environments which lack other means of communication. Second, we include aerial vehicles. UAVs are more than simply a faster node or a third dimension. They are qualitatively different nodes. The high position in the air means a UAV has better coverage and lower path loss than other nodes on the ground. UAV antenna pattern quickly tilts as the plane banks and turns causing radio dynamics beyond simple changes in position. The UAV is also a harsher environment with high vibration, large mechanical shocks, and occasional power interruptions. Other works have considered UAVs in MANETS [Gerla Paper], but, only in simulated environments and for large high-altitude UAVs. Third, we added detailed monitoring data collection. The data is placed into a database for post analysis and a web-based visualization tool will enable the MANET community to view and analyze the data for research and education purposes. The next section describes the testbed architecture.

3 PRIOR WORK ON MANET: Ad-hoc networks have been a subject of immense research, including the applications to large UAVs. Much of these investigations have been in theory and simulation with very few attempts at real world implementations of this technology. This section of the paper discusses previous related work done on MANET test beds. Over the past few years there has been extensive research on the simulation of ad hoc networks all across the world. Researchers from the University of California, LA [8] have simulated a MANET with UAVs using LANDMARK routing. Their design is based on a hierarchical routing scheme that is aimed at reducing the hops between source and destination nodes and provides support to large-scale military wireless networks. The paper describes a test bed that was simulated using a Parsec/GloMoSim platform. Using this test bed the researchers have successfully demonstrated the improved performance of a network that utilizes flat LANMAR routing and physical network hierarchy comprising of ad hoc ground radio subnets, point to point wireless long haul backbones, and Unmanned Air Vehicles (UAVs). Another paper titled “Designing and Implementing A Mobile Ad hoc Network Test bed”[11] deals with an operational ad hoc networking test bed that allows the study of various ad hoc routing in a laboratory environment with emphasis on low cost. Much of the research done is based on software assisted emulation wherein each node runs unicast and multicast routing protocols on a dynamic-on-demand basis. Their work on the test bed implementation is supposed to serve as a helpful precursor to the real world deployment of a full scale ad hoc network. All of the work done on MANETS that has been mentioned in this section so far is based purely on simulations. Real world experiments on the other hand throw a completely different challenge for researchers as variables such as radio interference, weather, terrain, node mobility, etc. can immensely alter the results of studies based on simulations. Of the few real world implementations for MANET, one of the most productive ones has been performed by researchers at CMU [5,6,7]. They have constructed and tested a tes tbed comprising of 2 stationary nodes, 5 mobile nodes that drive around the test site and a mobile node that enters and leaves the test site with a speed of about 25 to 40 km/h. The separation for the fixed nodes is a distance of about 700m. In contrast to their test bed, our goal is to monitor the MANET performance with the inclusion of UAVs in the network. The purpose of our experimentation is described by two broad scenarios. In the first scenario, an ad hoc network of ground nodes is disconnected because of distance and/or terrain. The UAV, with a better view of the nodes, maintains connectivity as an ad hoc relay. In the second scenario, a small UAV because of power and payload constraints has limited communication range which in turn may limit operational range. Ad hoc relaying between multiple UAVs extends communication range. Another paper that describes a real world implementation of MANET is A Large- scale Test bed for Reproducible Ad hoc Protocol Evaluations [10]. The paper introduces APE or an Ad hoc Protocol Evaluation test bed that the researchers have created to perform reproducible experiments to assess several different routing protocols in a real world scenario. Their test bed included three interconnected buildings on campus that allowed a distance of 174 meters for tests, and up to 37 nodes. These papers and many other that describe work related to simulated ad hoc networks discuss various aspects of MANETs. Some are aimed at introducing new wireless protocols, while some have developed test beds to test and compare various ad hoc protocols under different physical and simulated conditions. What makes AUGNet so special is that it is a real world implementation of an ad hoc network that comprises of fixed, mobile, and most importantly aerial nodes. Our goal is to study the effects of the inclusion of UAVs in the network and how they can be used to further increase the network efficiency on the whole. 4 THE AUGNET TESTBED: To better describe our test bed, we have broken it down to four areas which include the physical infrastructure, radio hardware and software, UAV platform and control, and data collection and monitoring. 4.1 Physical Infrastructure The physical infrastructure includes the test site, buildings, power, network connectivity, and UAV flight takeoff and landing facilities. Our test site is the Table Mountain Radio Quiet Zone (NRQZ) which is located 15km north of the University of Colorado’s Boulder campus. The dimensions of the site are 2.5km by 3.5km area with a north south alignment and with a site area of approximately 7 sq. km. Most of the site is flat and level with steep sides dropping to the level of the surrounding plain. The top is unobstructed for long distance communication while conversely the radio range can be easily limited by placing the radio below the top on the steep sides. An aerial photograph of the site is shown in Figure 1. A map of the site showing facilities is in Figure 2.

Figure 1: Aerial photograph of the Table Mountain test site looking west. Figure 2: Table Mountain facilities The test site buildings are connected via a fiber optic network and the site itself has a T1 backhaul to the Internet. The UAV airfield has been prepared at the north side of the site. It is approximately 100m long and 12m wide with a NNE by SSW alignment. 4.2 Radio Hardware and Software: Three areas of the radio hardware and software are described: the Mesh Network Radio, MNR packages, and base performance. 4.2.1 Mesh network radio (MNR) The mesh network radio consists of ad hoc routing software and a hardware platform to support the software. The software is Linux based and runs on any Linux platform with sufficient storage and memory. The software is designed to work with any Ethernet network interface supported by Linux. Most network interfaces are abstracted to appear as an Ethernet interface to the operating system. Therefore the software works with wired Ethernet and wireless 802.11, 802.11a, 802.11b, and 802.11g. Only 802.11b has been tested since this is the focus of the test bed research. The hardware can be any platform that supports Linux and 802.11b. A specific hardware platform was developed for this testing. The platform uses commercial off the shelf components and is shown in Figure . It is comprised of a Soekris single board computer, Orinoco 802.11b card, a Fidelity-Comtech bidirectional amplifier with up to 1W output, and a GPS. The GPS is included for monitoring purposes and is not necessary for the ad hoc routing protocol. The GPS updates a file with time and position estimates at 1 sec intervals. The components can fit in a box of dimensions 16cm x 16cm x 5cm. The MNR can operate in the temperature range from 0-60 °C with no cooling fan. Power is 12V and provided either directly or via power over Ethernet. All software resides on a compact flash (CF) card. Each CF card is identical except for an IP address that defines a node. When powered on, the operating system boots from an image stored on the CF, a RAM file system is created, and the software operates only in RAM. This mode of operation was chosen for three reasons. First, the CF card has a limited read/write life and this minimizes accesses to the card. Second, the hardware may be turned off at any time due to power interruptions and this avoids file system corruption due to partial file writes. Third, CF cards can be exchanged as needed between nodes since no operational state is stored on the card.

Figure 3: Mesh Network Radio (MNR) (left). MNR mounted in environmental enclosure for vehicle or fixed ground mounting (center). MNR mounted in UAV (foreground right) 4.2.2 MNR packages: The MNR can be mounted in different packages. For fixed ground sites and on vehicles, the MNR is mounted in the environmental enclosure shown in Figure 3. For the UAV, the MNR components are mounted on rails inside the UAV as shown in Figure 3. In addition to these versions, the ad hoc software can be operated on standard laptops running Linux. Such a node is used for the test bed gateway described in a later section. The software has also been ported to an IPaq PDA. For monitoring purposes, these nodes need to be provided with either a GPS or the ability to report their current location if they are at a fixed position. 4.3 UAV Platform and Control The UAV airframe is built from a carbon fiber monocoque construction. The wingspan is 2.4m and maximum takeoff weight is 15kg. The maximum payload is 5kg. Cruising speed is 90km/hr. The maximum measured speed is 190km/hr. The current fuel tank supports 45min of operation. Payload and avionics power is provided by batteries. The plane is controlled using standard 72MHz analog radio control (R/C). This limits operation to within a kilometer of the airfield.1 Development of the plane continues on several fronts. The fuel tank will be increased for longer flights. A plane waypoint flying system is being installed based on a 900MHz digital packet control link. This enables flight paths to be programmed and modified via computer taking the burden of direct control away from a human operator. This will enable longer range and longer duration flights. Interference tests were performed between the two R/C radios and the 802.11 payload. No interaction between the 2.4GHz and other bands could be measured with a spectrum analyzer or with a 802.11b WLAN monitoring tool. Direct measures of packet loss and throughput were unaffected by the coexistence the systems. The 72MHz R/C is affected when its antenna is placed within 10cm of the single board computer. The R/C antenna was mounted well away from the computer. 4.4 Data Collection and Monitoring Monitoring packets, like data packets from the nodes are directed to any one of the fixed nodes in the network. The fixed nodes in turn send these packets to the test bed gateway and on to the monitoring server via the internet. Remote monitoring capabilities allow us to remotely access data obtained, and to monitor the test site and activity on a real time basis. The monitoring server provides several functions. Firstly it serves to archive the monitoring data. This archived data provides additional functions like producing derived statistics. For example, the number of packets lost on a link is determined by observing the number of packets sent from the transmit radio on the link and observing the number successfully received at the receiver radio of the link. The difference between these values represents the loss rate. The monitor server also provides a playback function where different aspects of an experiment can be replayed by remote users for analysis purposes. The data is stored in a mySQL based database at the monitoring server. The playback feature uses a Java interface that enables users to analyze the data from a web brower from anywhere on the internet. A screen of the interface is shown below.

5 NETWORKING DESCRIPTION: Our wireless communications network comprises of fixed, mobile, and aerial nodes that relay packets to a Remote monitoring server via fixed gateways on the test bed. Local area connectivity is made available with units supporting 802.11b wireless transmission and mesh network routing. Each node in the network runs the Dynamic Source Routing (DSR) protocol at the network layer while communicating with other nodes via 802.11b at layer 2. We have implemented the DSR protocol ourselves using the Click Modular Router [Ref]. The reason we chose DSR over other ad-hoc protocols like AODV or TORA is because it’s routing is On-demand. In On-demand source routing which unlike Table driven routing where there are periodic updates sent from neighboring routers and not a continuous ------like protocols such as ------where routing messages are exchanged periodically between neighboring nodes. In On-demand routing a traffic source only seeks a route to a destination when it has to send data. The biggest advantage of this feature is that the nodes do not waste bandwidth in an effort to establish routes they may never use. In DSR, when a node needs to send a packet, it 1 The ability of the operator to safely perceive plane orientation rather than the radio limits the range. Operator fatigue is a factor that limits flight duration. initiates a route request process among all the nodes in the network to establish a route. DSR additionally uses source routing whereby each packet contains the ip addresses of the nodes a packet must pass through. Each node or MNR collects the network data that includes packets sent, packets received and packets discarded due to congestion induced overflow. This data is tagged with the GPS time and location and sent out at 10 second intervals. This interval ensures that the impact of the monitoring packets on other network traffic is minimal.

6 SIMULATIONS OF EXPECTED PERFORMANCE

7 ANECDOTAL RESULTS: This section details some of the major problems/issues we faced while conducting various tests. Issues: 1. To obtain a 5-hop scenario for the throughput and latency tests it was important to locate spots on the test site to place the ground nodes such that none of the intermediate nodes between the source and destination MNRs were skipped and hence a 5 hop route was guaranteed. The issue was of even greater relevance as we had to ensure that we were maintaining the maximal throughput possible while scouting for the ideal node locations. 2. Power sources for nodes: During early experimentation, the power was drawn off car batteries to the nodes via a power over ethernet unit . The biggest drawback with this setup was that every time the ignition for the car was turned off the node turned off as well. A more viable option that we have now adopted is Lead acid batteries that run off 12volts. They provide better stability, are easily portable, and ideal for our tests as the nodes can now be placed at isolated spots that may not be reachable by cars. 3. Another problem with using cars as node mounts as we noticed was that they offer a large metallic area that results in greater RF reflection/refraction that in turn reduces the RF potential of the radios considerably. 4. In terms of data collection and the monitoring software the biggest challenge was the design of the 2 layer monitoring system. The problem required a method for storing data and method for displaying near-real-time results. The solution involved the integration of two independent layers, namely data storage and real-time database querying. Java was used in the first layer to parse UDP data packets and store relevant information in a relational database when testing was taking place. Java applet and jsp/servlet technologies were used in the monitoring layer to take advantage of common web GUI features in addition to the JDBC interface with mySQL, allowing users to easily pull near-real-time test data from the database and display updates in a graphical and intuitive manner. 7.1 Test Bed Measurements: This section details the measurements obtained from tests conducted at the site so far 7.2 Baseline Ground Network: There are several factors that were measured to assess the MNR performance and are listed below. These are completed in the lab or at the test site as appropriate in order to establish expected network behavior. Measures of Performance Data Throughput Latency (communication delay) Jitter (delay variation) Communication Availability Remote Connectivity Hardware Reliability Range The range between ground nodes was measured using one MNR placed 2m above the ground and the Berkeley Variatronics Yellow Jacket held 2m above the ground. The MNR was placed at building B9 and was transmitting short ping packets to a node inside of B9. The Yellow Jacket was placed at various places around the test site and measured the received signal strength of packets. The experiment was repeated with the MNR placed on the ground so that its antenna was 0.2m above the ground (but, the Yellow Jacket still held 2m above the ground). The results are shown in Error: Reference source not found. From this data we see that the range is about 400m when the node is placed on the ground and 1200m when placed 2m above the ground. Five nodes named MNR-1 through MNR-5 were setup on the testbed, mounted 2 meters above the ground. For the range tests, the nodes had to be setup at locations where the throughput for a given link would be ideal while ensuring that the packets from a source to a given destination were routed by each of the intermediate nodes and none of them were skipped. From our range tests we have observed two types of ideal ranges. In terms of basic connectivity, tested by sending “pings” to each of the nodes in the network the Connectivity Range was found to be 1800 meters between two consecutive nodes. However in terms of throughput, this range was far large and resulted in frequent stalls in the file transfer. The distance between the nodes was thus decreased till a maximal Throughput range of 1200m was observed. The following graph describes the results obtained for the Performance vs. number of hops test. Data was transferred between nodes MNR-1 and MNR-2, then between between MNR-1 and MNR-3, and so on to the five hop transfer between MNR-1 and MNR-6. The throughput and latency measurements as a function of the number of hops were thus recorded, the results of the same are listed in table ___

Graph.

Measure Result Remarks Boot time (sec) 92 secs MNR/Laptop Network formation time (sec) 552 secs 6 MNRs at test site Ground level range: At test site, MNRs placed 6ft. Connectivity 1800 m from the ground Throughput 1200 m Air-to-ground range 3900 m at test site 2m and 150m above the ground Delay for a ground network (ms) 1 Hop 11.279 ms At test site in a 6 node 2 Hop 18.742 ms scenario. 3 Hop 22.512 ms 4 Hop 36.237 ms 5 Hop 40.891 ms throughput (bps) for the ground network: At test site as a function of 1 Hop hops 2 Hops 1494 Kbps 3 Hops 521 Kbps 4 Hops 280 Kbps 5 Hops 198 Kbps 131 Kbps throughput (pkt/sec) with UAV as one of At test site with 4 MNRs and a the nodes in the network UAV node 1 Hop 1246 Kbps 2 Hops 534 Kbps 3 Hops 234 Kbps 4 Hops 193 Kbps

8 CONCLUSION:

9 ACKNOWLEDGEMENTS This work has been supported by the L3-Comcept Corporation. The UAV planes were built and operated by the faculty and students of the Aerospace Engineering Sciences Department including Brian Argrow, Tom Bateman, Cory Dixon, Jack Elston, and Phillip Nies. Thanks to ITS/NTIA for granting permission to use the Table Mountain National Radio Quiet Zone and to John Ewan from ITS for supporting our operations on the testbed.

10 REFERENCES: 1. Timothy X Brown, Brian Argrow, Cory Dixon, Sheetalkumar Doshi, Philip Nies, “Ad hoc UAV-Ground Network(AUGNet) 2. Timothy X Brown, Brian Argrow, Sheetalkumar Doshi, Gerald Jones, “Test Bed for a Wireless Network on Small UAVs” 3. Henrik Lundgren, David Lundberg, Johan Nielsen, Erik Nordstrom, Christian Tschudin, “A Large-scale Testbed for Reproducible Ad hoc Protocol Evaluations”. 4. Sagar Sanghani, Timothy X Brown, Shweta Bhandare, Sheetalkumar Doshi, “EWANT: The Emulated Wireless Ad Hoc Network Testbed” 5. David A. Maltz, Josh Broch, and David B. Johnson, “Experiences Designing and Building a Multi-Hop Wireless Ad Hoc Network Testbed” 6. David A. Maltz, Josh Broch, and David B. Johnson. Quantitative Lessons From a Full-Scale Multi-Hop Wireless Ad Hoc Network Testbed 7. Qifa Ke, David A. Maltz, and David B. Johnson. Emulation of Multi-Hop Wireless Ad Hoc Networks. 8. Kaixin Xu, Xiaoyan Hong, Mario Gerla, Henry Ly, Daniel Lihui Gu, “LANDMARK Routing In Large Wireless Battlefield Networks Using UAVs” 9. Yongguang Zhang, Wei Li, “ An integrated Environment for Testing Mobile Ad-Hoc Networks” 10. James T. Kaba, Douglas R. Raichle, “Testbed on a Desktop: Strategies and Techniques to Support Multi-hop MANET Routing Protocol Development” 11. Zhigang Jin, Bing Liang, Yantai Shu, Oliver W.W.Yang, “Designing and Implementing A Mobile Ad hoc Network Testbed”