United States Military Academy USMA Digital Commons

West Point Research Papers

Winter 1-29-2021

Performance of Single Board Computers for Vision Processing

Curtis Manore CDT'21 United States Military Academy

Pratheek Manjunath United States Military Academy, [email protected]

Dominic Larkin United States Military Academy

Follow this and additional works at: https://digitalcommons.usmalibrary.org/usma_research_papers

Part of the Computational Engineering Commons, and the Other Computer Engineering Commons

Recommended Citation Manore, Curtis CDT'21; Manjunath, Pratheek; and Larkin, Dominic, "Performance of Single Board Computers for Vision Processing" (2021). West Point Research Papers. 378. https://digitalcommons.usmalibrary.org/usma_research_papers/378

This Conference Proceeding is brought to you for free and open access by USMA Digital Commons. It has been accepted for inclusion in West Point Research Papers by an authorized administrator of USMA Digital Commons. For more information, please contact [email protected]. Performance of Single Board Computers for Vision Processing

Curtis Manore Pratheek Manjunath Dominic Larkin United States Military Academy United States Military Academy United States Military Academy West Point, New York West Point, New York West Point, New York [email protected] [email protected] [email protected]

Abstract—With the increasing complexity of machine vision accelerator to aid in vision processing is a dedicated graphics algorithms and growing applications of image processing, how processing unit (GPU), which is specifically designed for do computers without a dedicated graphics processor perform? image processing. The GPU’s ability to perform fast matrix This research discusses the computational abilities of two low- cost single board computers (SBCs) by subjecting them to various computations in parallel makes it useful for processing multi- Visual Inertial Odometry (VIO) algorithms. The end goal of this dimensional arrays representing pixel data [3]. While GPUs research is to identify a SBC which meets the requirements of require high power consumption, recent developments in SBCs being employed on an Unmanned Aerial System for autonomous brought GPUs to the mobile computing arena. Current SBCs navigation. utilize integrated and dedicated GPUs to aid in vision process- Keywords: Single Board Computer, Odroid, Computer Vision, VIO ing while remaining a low-cost, mobile platform.

I.INTRODUCTION A. Background The advent of a credit card sized computer, the Rasp- Research and development of Small Unmanned Aerial berry Pi, has revolutionized the concept of the Internet of Systems (sUAS) has greatly benefited from the capabilities Things (IoT) [1]. This class of devices, commonly known brought on by SBCs. Most unmanned systems need to estimate as Single Board Computers, are often used as embedded their position and orientation to autonomously navigate in 3D computers by academics, industry professionals, and hobbyists space. In environments where external references (such as for applied research and tinkering. A single-board computer GNSS) are contested, inaccurate, or unavailable [4], Visual (SBC) is a device with all components such as CPU, RAM, Inertial Odometry (VIO) has emerged as a reliable alterna- storage, and network card mounted on one circuit board. tive. VIO algorithms help aid in navigation by fusing com- Hence they have a considerably smaller footprint than their puter vision and inertial measurements. These algorithms use desktop counterparts. SBCs have become a staple component monocular or stereo sensor input from a camera and provide of mobile robots. This can be attributed to their low-cost, state estimation of the robot’s position and orientation. The light-weight characteristics, and more recently – their proven architecture of most VIO algorithms is separated into two integration with the Robot Operating System (ROS). Due to parts: a front-end and a back-end. The front-end is responsible advances in software development and efficiencies gained in for processing the vision-based aspects of the algorithm. The vision processing algorithms, these embedded computers that vision processing here involves detecting, tracking, and vali- often lack the computational power of most laptops are now dating of landmarks observed from each image frame provided capable of providing the needs of computer vision for mobile by the sensor. The front-end algorithm creates landmarks from platforms. everyday objects or noticeable features in an image, such as Machine vision, also known as computer vision, involves the window corners, door frames, and table edges. Once the acquiring, processing and analyzing visual data to produce front-end identifies and validates these features, it compares an understanding of the real world as perceived by human the current features to their locations in the previous image to eyesight. Through the use of optical sensors (such as cameras) arrive at an initial state estimation for the UAS [5]. Since and algorithms, computers are able to recognize shapes, track an image contains many features, the back-end applies a motion, detect objects and estimate the pose of a body [2]. The smoothing algorithm to minimize the error between the state algorithms that serve vision processing can be computationally estimation and actual pose of the robot through methods such intensive and average CPUs in personal computers may be as a maximum a posteriori (MAP) optimization process [6]. insufficient to support computer vision. A common hardware After this stage, filtering or Inertial Measurement Unit (IMU) pre-integration takes place to improve the pose estimate. [6]. This work is supported by the Army Research Laboratory. The views and With filtering processes such as an Extended Kalman Filter conclusions contained herein are those of the authors and should not be (EKF) solving non-linear system for state estimation in the interpreted as necessarily representing the official policies or endorsements, either expressed or implied, of the U.S. Department of Defense or the U.S. front-end and MAP optimization for smoothing in the back- Government. end, a powerful processing component is needed to handle

U.S. Government work not protected by U.S. copyright the VIO algorithm’s demands. However, one needs to be 1) Processor: Clock frequency and number of processor cognizant of the computing device’s size, weight and power cores are key factors affecting the rate at which an SBC consumption when the robotic platform is an aerial system. can conduct computations. Greater number of cores increase These constraints often limit the amount of computational parallel processing capabilities and higher clock frequencies capability available on an sUAS. increase the number of instructions per second that the CPU can process, thereby leading to an increase in performance. II.RELATED WORK &LITERATURE SURVEY 2) RAM: VIO algorithms may need to allocate a large Previous attempts at comparing Single Board Computers amount of information to memory. Hence the most important have looked at processor architecture or operating system’s consideration of RAM for vision processing is simply the total kernel [7]. Research and evaluation of SBCs are typically amount of RAM. Second factor affecting memory utilization performed for one’s specific application, like with the case is the read/write speed of the chipset. DDR5 is the fastest of Hadoop Distributed File System [8] or for parallel discrete- version of SDRAM currently available but DDR4 has greater event simulation of telecommunication systems [9]. Multiple prevalence and compatibility. SBCs can also be combined in computing clusters for im- 3) Storage Media Type: SBCs can have different media proved parallel processing [10]. types and interfaces for storage. Embedded Multi-Media Card Artificial intelligence—a relatively new era in mobile (eMMC), Secure Digital (SD) card, and Serial Advanced robotics—is being implemented with Single Board Computers Technology Attachment (SATA) Solid State Drives (SSD) are to conduct modeling and planning techniques, low level con- commonly used. NVMe SSDs are still nascent in the field trol, and state estimation [11]. One of the less powerful SBCs of SBCs. eMMC and SD cards are flash storage interfaces used in this research is the Odroid XU4. Sensor fusion and whereas SSDs use SATA. While the SATA interface has faster robot navigation using natively available SLAM algorithms in read and write speeds, an SSD takes up more physical space ROS have been successfully implemented on a small ground than flash media, causing a disadvantage for use on an aerial robot with the Odroid XU4 [12]. platform. Many computer enthusiasts like to test the limits of their 4) Cost: A low-cost device makes the system more ex- hardware through the use of benchmarking tools such as pendable. If the sUAS were to crash or if the SBC were 3DMark [13]. to get damaged during prototype testing, there would not be One of the problems with using benchmarking tools is that significant budget setbacks to the project. they do not always model a computer system’s actual usage 5) Power: The peak power consumption and operating by the end-user. They mostly test only one or two aspects of a voltage feature as one of the selection criteria in the table. computer’s performance in isolation. Users of SBC have very Since the SBC would be powered from the battery on a sUAS, diverse use cases for these small computers, and we would be lesser power consumed by this payload translates to longer remiss by subjecting them to just one benchmark. We attempt flight time. to take into account several factors that might influence how 6) Size and weight: Size and weight play a major role in the SBC performs. Some variables include specific versions of the implementation of a computing device on a sUAS. Light software, libraries, device drivers and compiler options used. weight devices are always preferred for increased endurance. This research covers a breadth of characteristics for analyz- Smaller form factor devices are easier to mount and secure on ing various SBCs for the sole purpose of performing Visual sUAS frames. Inertial Odometry. 7) Graphics Processor: The Nvidia TX2 uses a dedicated 256-core CUDA GPU to aid with image processing whereas III.HARDWARE SELECTION the other SBCs use an integrated graphics card. While this Priorities for selecting a Single Board Computer will vary research project would greatly benefit from parallel processing by user and the intended application but cost, size, processor and hardware acceleration, those VIO algorithms which did and power are some of the common deciding factors. This is not support CUDA APIs could not benefit from this architec- a list of popular Single Board Computers in 2020 [14]: ture. This limitation combined with a greater power demand by GPUs caused this feature to be ranked low in our selection 1) Raspberry Pi 4 5) Asus TinkerBoard criteria. 2) Nvidia Jetson Xavier 6) Google Coral After analyzing the above metrics, two SBCs made by NX 7) Odroid XU4 HardKernel were pursued. The research focused on subjecting 3) Odroid N2+ 8) Latte Panda the Odroid XU4 and the Odroid H2+ to three VIO algorithms. 4) Intel NUC 9) UP Board IV. SYSTEM SETUP Table I lists the four platforms and their specifications that A. Hardware Infrastructure were considered for this study, prior to down-selection to two. 1) Test device: Odroid XU4 or Odroid H2+ Keeping the final application in view, the following metrics 2) 64 GB eMMC with Operating System and software primarily influenced the decision for selecting a computing 3) Logitech C270 webcam platform. 4) Intel RealSense D435i TABLE I: Single Board Computer Comparison SBC Model Clock CPU Architecture RAM Storage Media Type Cost Power (W) Weight Speed Cores (GB) ($) (g) (GHz) Odroid XU4 2 8 ARM 32-bit 2 eMMC or microSD 49 20 (5V, 4A) 60 Odroid H2+ 2.5 4 x86 64-bit 32 eMMC, microSD or SATA 119 60 (15V, 4A) 185 Raspberry Pi 4B 1.5 4 ARM 64-bit 8 MicroSD 75 15 (5V, 3A) 45 Nvidia TX2 2 4* ARM 64-bit 8 eMMC 479 57 (19V, 3A) 85 *Supplemented by 256-core CUDA GPU

B. Software Infrastructure A 64GB eMMC was using as the storage media on both the test Odroids. It was configured with the following: 1) Ubuntu 18.04 2) ROS Melodic 3) USB webcam driver and ROS wrapper [https://wiki.ros.org/usb cam] 4) Intel RealSense driver and ROS wrapper [https://github.com/IntelRealSense/realsense- ros#installation-instructions] 5) Glances system monitor [https://nicolargo.github.io/glances/] Fig. 1: MSCKF Output with Features 6) psutil Python library 7) Three VIO algorithms listed below. Three open-source VIO algorithms were used to analyze vision processing performance on the two test SBCs. 1) MSCKF: Multi-State Constraint Kalman Filter (MSCKF) is a lightweight VIO algorithm designed to run on any computational platform. The back-end of the algorithm uses an Extended Kalman Filter to estimate the 3D pose of the camera [15]. The front-end relies on Features from Accelerated Segment Test (FAST) corner detector to make use of edge-based detection. This allows MSCKF to be environment independent and recognize features and trigger events in a 3D space. Since the MSCKF camera needs edge-like features to trigger events, low texture Fig. 2: VINS-Mono Output with Features areas with no edges generate very few events. When the camera in [16] passed over texture-less areas such as back feature to provide better position estimation and tracking. walls in a room, no events were generated in the parts of Documentation and source code:[https://github.com/HKUST- the image containing the wall. This led to tracking error Aerial-Robotics/VINS-Mono] since no features are tracked over this area. MSCKF is 3) Kimera-VIO: Developed by MIT’s SPARK Laboratory the oldest VIO algorithms used. Documentation and source and members of the Distributed and Collaborative Intelligent code:[https://github.com/KumarRobotics/msckf vio] Systems and Technology (DCIST) Collaborative Research 2) VINS-Mono: The Monocular Visual-Inertial System Alliance, Kimera-VIO is an algorithm very similar to GTSAM (VINS-Mono) tracks corner features through a non-linear based approaches. Its front-end detects Shi-Tomasi corners and optimization-based sliding window estimator [15]. A KLT tracks them across frames through a Lukas-Kanade tracker sparse optical flow algorithm tracks existing features in the [18]. Once a feature is detected, Kimera conducts stereo front-end while new corner features are detected from the matching and geometric verification. In its back-end, it uses incoming image [17]. Keyframes are also selected in the front- GTSAM to add extra robustness and smoothing capabilities end. VINS-Mono pre-integrates IMU measurements before [18]. Kimera has a Robust Pose Graph Optimization (RPGO) sending it to the back-end for optimization. VINS-Mono is module that enhances the quality of pose estimation, similar compatible with the Robotic Operating System (ROS) and to features in VINS-Mono. Kimera works with ROS and has a contains an IOS version for mobile computation [15]. The 3D mesh reconstruction feature that can be used to view areas VINS-Mono package includes a pose graph optimization surveyed by the VIO algorithm. Documentation and source behavior and eventually shutdown when overheated. Temper- ature was logged to study the increase in heat generated when running VIO algorithms. These measurements were taken using a tool called Glances for real-time system monitoring and psutil Python library to log the performance metrics. Figure 4 shows the Graphical User Interface of Glances.

Fig. 3: Kimera-VIO Output with Features code: [https://github.com/MIT-SPARK/Kimera-VIO]

V. TESTING AND METHODOLOGY Three VIO algorithms were implemented on the two test Odroids: MSCKF, VINS-Mono, and Kimera-VIO. Each algo- rithm on both SBCs were subjected to the same test scenario: a rosbag file obtained from the EuRoC dataset [19]. The file used was a 180-second video recording captured in the Machine Fig. 4: Glances System Monitoring Tool Hall of ETH Zurich where sUAS are often tested. Performance was measured on the easy, medium, and hard datasets and VI.RESULTS results obtained are presented in the following section VI. The configuration parameters of the algorithms were at their default TABLE II: Algorithm Build Times settings so as to focus solely on the SBC’s computational abilities without the results being skewed by the algorithms’ Algorithm Odroid XU4 Odroid H2+ performance. The following parameters were monitored and MSCKF 5 min 3 min analyzed for each algorithm and SBC: VINS-Mono 30 min 19 min 1) Build time: This measures the amount of time the Kimera-VIO >24 hours 1 hour algorithm takes to compile successfully on the given platforms. With different computational power and processor architecture, TABLE III: Mean CPU Load and Memory Utilization build times will differ between SBCs. Lower build times indicate the ability to quickly reproduce the algorithm on Algorithm Odroid XU4 Odroid H2+ different computing devices. CPU % Mem % CPU % Mem % 2) CPU load: This is a measure of the fraction of total MSCKF 215.92 6.10 50.81 0.421 CPU cycles needed to run a process. A lower CPU load for a VINS-Mono 198.53 59.39 61.46 0.394 VIO algorithm suggests a lighter algorithm or a more capable Kimera-VIO 338.59 49.28 201.27 0.908 CPU. Here, CPU load is measured in percentage relative to the SBC itself. A. Odroid XU4 Performance 3) Memory utilization: This metric shows the amount of memory allocated to a process from the total memory available Code for MSCKF and VINS-Mono took under an hour to on the device. Higher memory utilization signifies either a build on the XU4, as shown in Table II. However, Kimera-VIO large demand from a process or insufficient memory available took over a day to compile. on the device itself. As shown in equation 1, this is a Figures 5a, 7a, and 9a illustrate the CPU load for MSCKF, measure relative to each device. If a VIO algorithm has “good” VINS-Mono, and Kimera-VIO, respectively, over the time memory utilization, it must have a consistently low percentage that the EuRoC scenario is running. Memory utilization for utilization over the duration that the algorithm runs. the Odroid XU4 is shown in Figures 5b, 7b, and 9b. Table III shows the mean and median CPU load and memory memory used per process % utilized = ∗ 100 (1) utilization for running the vision processing algorithms on the total memory on device EuRoC MH01 scenario. If the algorithm required more than 4) Temperature: Rising temperature of CPUs is a direct one process to run the algorithm, such as MSCKF’s Image side-effect of constant computational load. The Odroid XU4 Processor and Feature Tracker processes, the mean CPU load has an active heatsink whereas the Odroid H2+ has a passive was taken from the most computationally intensive process heatsink. SBCs, like most computers, tend to exhibit erratic and the memory utilization was summed across all processes. (a) Odroid XU4 MSCKF CPU Load (b) Odroid XU4 MSCKF Memory Utilization Fig. 5: MSCKF on Odroid XU4

(a) Odroid H2+ MSCKF CPU Load (b) Odroid H2+ MSCKF Memory Utilization Fig. 6: MSCKF on Odroid H2+

(a) Odroid XU4 VINS-Mono CPU Load (b) Odroid XU4 VINS-Mono Memory Utilization Fig. 7: VINS-Mono on Odroid XU4

(a) Odroid H2+ VINS-Mono CPU Load (b) Odroid H2+ VINS-Mono Memory Utilization Fig. 8: VINS-Mono on Odroid H2+ (a) Odroid XU4 Kimera-VIO CPU Load (b) Odroid XU4 Kimera-VIO Memory Utilization Fig. 9: Kimera on Odroid XU4

(a) Odroid H2+ Kimera-VIO CPU Load (b) Odroid H2+ Kimera-VIO Memory Utilization Fig. 10: Kimera on Odroid H2+

The Odroid XU4 with ARM architecture was able to computa- algorithms were able to generate a pose estimation, greatly tionally run the MSCKF and VINS-Mono algorithms with the benefiting from its x86 Architecture, faster processor, and EuRoC dataset successfully. However, VINS-Mono failed to larger RAM capabilities. Unlike on the XU4, VINS-Mono’s output a pose estimation—a crucial output for VIO algorithms. Ceres Solver worked successfully on th H2+ to generate a pose This was due to the non-linear equation solver that VINS- estimate for the algorithm, and Kimera-VIO did not have any Mono used: Ceres [20]. The Ceres Solver does not handle memory utilization errors. large, non-linear optimization problems well with a 32 bit Table IV addresses the temperature concerns with the ARM operating system. Since the Odroid XU4 only had 2GB Odroid H2+’s passive heatsink. MSCKF ran noticeably cooler of RAM, Ceres struggled solving the pose estimation problem than the other two algorithms. MSCKF is the oldest and least for VINS-Mono. computationally intensive algorithm, so its lower temperature For Kimera-VIO, CPU load exceeded 400 percent, and the is reasonable. VINS-Mono and Kimera-VIO both ran above 50 memory utilization in Figure 9b increased linearly rather than degrees Celsius. While this is noticeably warm for a passive remaining constant. This led to buffer overflow, increasing heatsink, it is still considered a safe operating condition. latency in image frames and a significant lag in pose esti- mation. When memory utilization topped 80 percent, a large TABLE IV: Temperature for Odroid H2+ drop in the CPU load occurred and the algorithm crashed. The authors believe that the Odroid XU4 cannot handle Kimera- Algorithm Mean Median VIO’s complexity and that Kimera-VIO is better suited for a MSCKF 36.7 °C 37 °C platform that has more RAM and a faster processor. VINS-Mono 55.0 °C 56 °C Kimera-VIO 52.9 °C 53 °C B. Odroid H2+ Performance Figures 6a, 8a, and 10a show charts of CPU load for the VII.CONCLUSIONS algorithms running on the Odroid H2+ platform. Since the Across the EuRoC scenarios of easy, medium, and hard, H2+ has 32GB of RAM, 5 times what the Odroid XU4 the VIO algorithms tested did not require greater processing has, the amount of memory utilized is much lower across power to navigate harder environments. However, the Odroid all algorithms, shown in Figures 6b, 8b, and 10b. This leads XU4 performed much worse than the Odroid H2+ on vision to more stable performance for the algorithms. All three processing. Real-time performance aside, the Odroid H2+’s 64-bit architecture led to faster build time prior to execution. [4] A. Badshah, N. Islam, D. Shahzad, B. Jan, H. Farman, M. Khan, The authors make the following inferences about single board G. Jeon, and A. Ahmad, “Vehicle navigation in GPS denied environment for smart cities using vision sensors,” Computers, Environment and computers for VIO: Urban Systems, vol. 77, p. 101281, Sep. 2019. [Online]. Available: 1) The amount of RAM greatly improves the ability of the http://www.sciencedirect.com/science/article/pii/S0198971517304489 [5] A. I. Mourikis and S. I. Roumeliotis, “A Multi-State Constraint SBC to conduct vision processing. Table III shows the Kalman Filter for Vision-aided Inertial Navigation,” in Proceedings decrease in total RAM used between the Odroid XU4 2007 IEEE International Conference on Robotics and Automation. and Odroid H2+. Rome, Italy: IEEE, Apr. 2007, pp. 3565–3572, iSSN: 1050-4729. [Online]. Available: http://ieeexplore.ieee.org/document/4209642/ 2) 64-bit architecture works better for VIO. The 32-bit XU4 [6] Z. Zhang, A. Suleiman, L. Carlone, V. Sze, and S. Karaman, platform caused issues with dependencies and limited “Visual-Inertial Odometry on Chip: An Algorithm-and-Hardware Co- computing capabilities, and the 64-bit H2+ did not have design Approach,” in Robotics: Science and Systems XIII. Robotics: Science and Systems Foundation, Jul. 2017. [Online]. Available: these issues when building the VIO algorithms as shown http://www.roboticsproceedings.org/rss13/p28.pdf in Table II. [7] N. Alee, M. Rahman, and R. B. Ahmad, “Performance comparison of 3) The x86 H2+ worked better than the ARM XU4. Most single board computer: A case study of kernel on arm architecture,” in 2011 6th International Conference on Computer Science Education algorithms worked with x86 architecture by default since (ICCSE), 2011, pp. 521–524. it is what most modern computers use today. On the [8] A. Adnan, Z. Tahir, and M. A. Asis, “Performance evaluation of Odroid XU4 with ARM architecture, VINS-Mono failed single board computer for hadoop distributed file system (hdfs),” in 2019 International Conference on Information and Communications to generate a pose estimate–one of the essential outputs Technology (ICOIACT), 2019, pp. 624–627. of the algorithm. The authors believe that the 32-bit [9] G. Lencse and S. Rep´ as,´ “Method for benchmarking single board ARM architecture was not able to handle the non-linear computers for building a mini supercomputer for simulation of telecom- munication systems,” in 2015 38th International Conference on Telecom- optimization solver. However, 64-bit ARM is still a munications and Signal Processing (TSP), Jul. 2015, pp. 246–251. popular choice for vision processing, such as Kimera- [10] P. J. Basford, S. J. Johnston, C. S. Perkins, T. Garnock- VIO’s preference to use a Tegra TX2 platform. Jones, F. P. Tso, D. Pezaros, R. D. Mullins, E. Yoneki, J. Singer, and S. J. Cox, “Performance analysis of single This research will continue to scale in complexity and realism. board computer clusters,” Future Generation Computer Systems, Below are two areas in which there is interest to evolve this vol. 102, pp. 278–291, Jan. 2020. [Online]. Available: project in the near future: http://www.sciencedirect.com/science/article/pii/S0167739X1833142X [11] I. Zamalloa, R. Kojcev, A. Hernandez,´ I. Muguruza, L. Usategui, 1) Hardware Implementation - Test and validate VINS- A. Bilbao, and V. Mayoral, “Dissecting Robotics - historical overview Mono and Kimera-VIO using the Odroid H2+ on a and future perspectives,” arXiv:1704.08617 [cs], Apr. 2017, arXiv: 1704.08617. [Online]. Available: http://arxiv.org/abs/1704.08617 physical quadcopter. Both VINS-Mono and Kimera-VIO [12] D. F. Tello Gamarra, A. Piccinini Legg, M. A. de Souza Leite Cuadros, are updated regularly, so they still hold relevancy in and E. Santos da Silva, “Sensory integration of a mobile robot using the visual processing today. embedded system odroid-xu4 and ros,” in 2019 Latin American Robotics Symposium (LARS), 2019 Brazilian Symposium on Robotics (SBR) and 2) Sensor complexity - Test and validate Odroid H2+ 2019 Workshop on Robotics in Education (WRE), 2019, pp. 198–203. ability to handle complex sensors like a 2D-LiDAR or [13] U. L. ©, Benchmarks 3DMark, 2020 (accessed December 24, 2020). a depth camera for VIO. VINS-Mono uses a simple [Online]. Available: https://support.benchmarks.ul.com/en/support/home [14] M. Long, Best SBCs for 2020, 2020 (accessed December 25, 2020). webcam for monocular vision whereas Kimera-VIO uses [Online]. Available: https://www.electromaker.io/blog/article/best-sbc- a stereo camera. The accuracy of the pose estimation for-ai-single-board-computer-for-artificial-intelligence for the different sensor inputs will prove an interesting [15] J. Delmerico and D. Scaramuzza, “A Benchmark Comparison of Monocular Visual-Inertial Odometry Algorithms for Flying Robots,” research discussion for future work. in 2018 IEEE International Conference on Robotics and Automation (ICRA). Brisbane, QLD: IEEE, May 2018, pp. 2502–2509. [Online]. VIII.ACKNOWLEDGMENTS Available: https://ieeexplore.ieee.org/document/8460664/ [16] A. Z. Zhu, N. Atanasov, and K. Daniilidis, “Event-Based Visual Inertial The authors would like to thank Christopher Korpela and Odometry,” in 2017 IEEE Conference on Computer Vision and Pattern the Robotics Research Center at West Point for providing Recognition (CVPR). Honolulu, HI: IEEE, Jul. 2017, pp. 5816–5824. technical expertise and equipment, Luca Carlone from MIT [Online]. Available: http://ieeexplore.ieee.org/document/8100099/ [17] T. Qin, P. Li, and S. Shen, “VINS-Mono: A Robust and Versatile SPARK Laboratory for advice on Kimera-VIO. Monocular Visual-Inertial State Estimator,” IEEE Transactions on Robotics, vol. 34, no. 4, pp. 1004–1020, Aug. 2018, arXiv: 1708.03852. REFERENCES [Online]. Available: http://arxiv.org/abs/1708.03852 [1] S. J. Johnston, M. Apetroaie-Cristea, M. Scott, and S. J. Cox, “Appli- [18] A. Rosinol, M. Abate, Y. Chang, and L. Carlone, “Kimera: an cability of commodity, low cost, single board computers for internet of Open-Source Library for Real-Time Metric-Semantic Localization and things devices,” in 2016 IEEE 3rd World Forum on Internet of Things Mapping,” arXiv:1910.02490 [cs], Mar. 2020, arXiv: 1910.02490. (WF-IoT), 2016, pp. 141–146. [Online]. Available: http://arxiv.org/abs/1910.02490 [2] A. Voulodimos, N. Doulamis, A. Doulamis, and E. Pro- [19] M. Burri, J. Nikolic, P. Gohl, T. Schneider, J. Rehder, S. Omari, topapadakis, “Deep Learning for Computer Vision: A Brief M. Achtelik, and R. Siegwart, “The EuRoC micro aerial vehicle Review,” Feb. 2018, iSSN: 1687-5265. [Online]. Available: datasets,” The International Journal of Robotics Research, vol. 35, Jan. https://www.hindawi.com/journals/cin/2018/7068349/ 2016. [3] A. HajiRassouliha, A. J. Taberner, M. P. Nash, and [20] S. Agarwal, K. Mierle, and Others, “Ceres solver,” http://ceres- P. M. F. Nielsen, “Suitability of recent hardware accelerators solver.org. (DSPs, FPGAs, and GPUs) for computer vision and image processing algorithms,” Signal Processing: Image Communication, vol. 68, pp. 101–119, Oct. 2018. [Online]. Available: http://www.sciencedirect.com/science/article/pii/S0923596518303606