Evolution of Robotic Simulators: Using UE 4 to Enable Real-World Quality Testing of Complex Autonomous Robots in Unstructured Environments Patrick Wolf a, Tobias Groll b, Steffen Hemer c and Karsten Berns d Robotics Research Lab, Dep. of Computer Science, TU Kaiserslautern, Kaiserslautern, Germany Keywords: Robotic, Simulation, Sensors, Framework, Commercial Vehicle Development. Abstract: Robotic simulators are essential for control development since they allow early testing. Additionally, trial time is tremendously reduced in comparison to a real system. In the recent past, many powerful simulation systems emerged, offering a high quality and level of realism. Still, those simulators have some shortcomings concerning the development of complex commercial vehicles that regularly operate within extremely cluttered and unstructured environments. This paper aims to reduce the gap occurring during the simulation and real- world testing to increase the expressiveness of robot control software tests. 1 INTRODUCTION of the Unreal Engine 4 into the middleware Finroc. Based on the UE 4-Finroc-API, sensors can be mod- Simulation engines are a powerful tool for the devel- eled in a more realistic way to include necessary char- opment of robotic systems. They allow early testing, acteristics that affect the robot’s navigation and per- design and sensor optimization, plus improved con- ception. Robot control behavior can be tremendously troller development before tests on the physical plat- improved through the consideration of such effects. form are performed. However, there exists a gap be- Also, the risk of tailoring a control approach towards tween testing in simulation and the real world. In the (simulation) specific data decreases. past, complex, relevant features for robot perception Sect. 2 describes state of the art and evolution of could not be completely modeled by the simulator robotic simulators followed by a description of the due to the versatility of environments. This is espe- UE 4-Finroc interface (Sect. 3). Sensors and virtual cially true for the development of autonomous com- robot setup is addressed in Sect. 4. Sect. 5 targets spe- mercial vehicles which have specialized supplements cialized actuation. Finally, two applications examples and kinematics for task fulfillment. However, the are provided (Sect. 6) and a summary and conclusion consideration of specialized sensors, sensing distur- follows (Sect. 7). bances, and the environment is essential for success- ful control development. With the Unreal Engine 4 (UE 4), a suited framework for photorealistic render- 2 RELATED WORK ing of complex scenes at the real-time performance is available. In recent literature, many UE 4-based sim- Robotic simulators are a common tool for control de- ulators have been proposed. Unfortunately, they do velopment. Depending on the application field, there not explicitly address the requirements for complex are different requirements for the simulation. Accord- commercial vehicles and sensing errors in the qual- ing to (Goodin et al., 2018), these can be grouped into ity required to reduce the simulation and real-world three major fields: testing gap. Development Environments focus on the basic The following contribution targets the integration physical modeling and environment interaction. Test Environments detailed representation of the a https://orcid.org/0000-0003-2740-7655 environment, more realistic physics and sensor b https://orcid.org/0000-0002-6214-9842 data modeling. c https://orcid.org/0000-0002-8141-3434 Empirical/semi-empirical simulators application d https://orcid.org/0000-0002-9080-1404 tailored highly realistic physics or sensor data. 271 Wolf, P., Groll, T., Hemer, S. and Berns, K. Evolution of Robotic Simulators: Using UE 4 to Enable Real-World Quality Testing of Complex Autonomous Robots in Unstructured Environments. DOI: 10.5220/0009911502710278 In Proceedings of the 10th International Conference on Simulation and Modeling Methodologies, Technologies and Applications (SIMULTECH 2020), pages 271-278 ISBN: 978-989-758-444-2 Copyright c 2020 by SCITEPRESS – Science and Technology Publications, Lda. All rights reserved SIMULTECH 2020 - 10th International Conference on Simulation and Modeling Methodologies, Technologies and Applications Simulation frameworks belonging to the first category environments. Supplementary to a robot control soft- are exemplary SimVis3D (Wettach et al., 2010) or ware framework, a simulation engine allows the mod- Gazebo (Aguero et al., 2015). Both feature a 3D sim- eling of robots and environment to simulate physical ulation including a physics and render framework as properties of those. well as a control API. SimVis3D, uses the Newton Dynamics physics engine and Coin3D for visualiza- 3.1 Finroc tion while Gazebo relies on the Open Dynamics En- gine and initially OpenGL Utility Toolkit (GLUT) but The Finroc framework for robot control provides a nowadays OGRE. Also, V-REP (Freese et al., 2010) powerful suite for the development of robot control is a multipurpose robot simulator that supports nu- systems (Reichardt et al., 2013). The central con- merous languages and controllers. The user inter- cept is the decompostion of functionality into mod- face enables a simple construction of a robotic sys- ules. A Finroc program is called part while data struc- tem and offers a default sensor suite. Yet, there are tures for data exchange are ports. They provide an limitations when it comes to bigger environments and abstraction from the actual communication technol- photo-realistic textures. As an example of second ogy, as TCP. Through information hiding, the com- and, to some extent, the third category, Virtual Au- munication stack can be easily exchanged. Finroc of- tonomous Navigation Environment (VANE) (Goodin fers inter-network communication as well as shared et al., 2017) can be mentioned. Its focus is on mili- memory-based technologies for lock-free and zero- tary, off-road unmanned ground vehicles (UGV) and copy data exchange. The Finroc-UE 4 simulation uses therefore puts strong efforts in material (reflections), Finroc data ports for communication. weather (sun/rain), dust/haze (atmosphere), and vege- tation simulation. Similarly, USARSim (Carpin et al., 3.2 Unreal Engine 4 2007), an urban search and rescue simulation build on the commercially available Unreal Engine 2.0, is of An important criterion for selecting the best-fitted restricted use. simulation engine is a high performance in visual- Unreal Engine 4 (UE 4) started to offer photo- izing outdoor landscapes and representing vehicles realistic rendering, an open-world design, and level physically correct. Gaming engines are especially of detail processing for free in academic applications suited for this purpose since they perform outstand- in 2014. The prominent CARLA (Dosovitskiy et al., ingly in representing very realistic environments and 2017) describes itself as a “simulator for autonomous have feasible realtime simulation capabilities. UE 4 driving research”. At its core, UE 4 does the main offers these features in a powerful way and is there- physics simulation and rendering work whereas the fore selected as simulation. Further, UE 4 provides framework provides a whole lot of tools and resources good support for the development of own content and for testing autonomous car driving in urban scenarios. extensions. It provides multiple possibilities for soft- This includes simulated sensor suites, UE 4 content ware modules development using an extensive C++ such as vehicles and urban maps as well as support for API and BLUEPRINT system. Physical effects use loading OpenDrive format for map generation. Sim- NVIDIA PhysX which allows fast calculations on a ilarly, AirSim (Shah et al., 2017), Sim4CV (Muller¨ GPU. Further, the material and rendering system en- et al., 2017), and Flightgoggles (Guerra et al., 2019) ables a realistic visualization of the environment. were initially developed for the aim to test computer The overall performance of a robot’s sensor is vision algorithms within the field of drone/UAV ap- strongly influenced by the sensing quality. Conse- plications and are extended for autonomous driving. quently, it is necessary to generate appropriate data Pavilion presented by (Jiang and Hao, 2019) focuses to simulate such sensors. Often applied systems are on combining UE 4 with an interface for ROS using visual sensors like cameras or laser-scanners which standard ROS messages, yet using no separate bridge especially benefit from simulation realism. UE 4 en- application and resolving the binary incompatibility ables data generation that has similar quality to the due to Run-Time Type Information (RTTI) and C++- real-world based on realistic-looking environments. exceptions. 3.3 Interface Plugin 3 SIMULATION FRAMEWORK Finroc and UE 4 share data for the transmission of sensor readings and exchange of control commands Multiple software components are applied to imple- for simulated actuators. An UE 4-Finroc plugin pro- ment and test robotic control systems in simulated vides an API for both software systems using the C++ 272 Evolution of Robotic Simulators: Using UE 4 to Enable Real-World Quality Testing of Complex Autonomous Robots in Unstructured Environments Interial Measurement Units. Inertial measure- ment units (IMU) provide acceleration readings around every translational and rotational axis of a robot. Additionally, the 3D orientation is usually available. IMU data is often prone to integration er- rors over time.