Development of In-Vehicle Distributed Computing Platform and Its Application on Around View Monitor System

中華民國第十九屆車輛工程學術研討會，桃園創新技術學院機械工程系，台灣中壢，2014 年 11 月 14 日。 The 19th National Conference on Vehicle Engineering, Nov. 14, 2014, TIIT, Jhongli, Taiwan. E – 012 ------Development of In-Vehicle Distributed Computing Platform and its Application on Around View Monitor System

Rong-Terng Juang Automotive Research & Testing Center E-mail:[email protected]

ABSTRACT design principles of the AUTomotive Open System As automakers begin to introduce new safety features, Architecture (AUTOSAR) [3], which is an open and advanced cars need significantly greater computing power. standardized automotive software architecture, this paper This paper presents an in-vehicle distributed computing proposes an in-vehicle distributed computing platform and platform and its application on around view monitor presents its application on around view monitor system. system. Based on AUTOSAR software architecture, the The rest of the paper is organized as follows. Section 2 algorithm of the around view system is divided into reviews AUTOSAR and automotive ethernet technologies. software components which are then deployed on multiple Section 3 shows the proposed in-vehicle distributed ECUs. Through a high speed ethernet, the ECU connects computing platform. Section 4 demonstrates the prototype to each other. By using this platform, application of the developed around view monitor system. Lastly, developers can put their effort on the design of various conclusions are given in Section 5. applications, without taking care of hardware, firmware, etc. Furthermore, the platform can reach high computing power by utilizing multiple ECUs. AP1, AP2,…, APn

Keywords: AUTOSAR, Automotive Ethernet, Around View Monitor System Firmware/Driver

Introduction ECU-Hardware Automakers are beginning to deliver improved driving safety and convenience through the development of Figure 1. System architecture of conventional ECU. next-generation advanced driver assistance systems (ADAS) [1]. Cars will become safer and more efficient as Technical Review they grow increasingly aware of and react to the surrounding driving environment and conditions. For This section reviews the AUTOSAR and automotive example, the new technologies covers traffic sign ethernet technologies. recognition, blind spot detection, and lane-departure (a) AUTOSAR warnings, as well as driver-alertness monitoring, assisted AUTOSAR is a worldwide development partnership of parking, night vision, etc. This means cars have to equip car manufacturers, suppliers and other companies from the with significantly greater computing power to enable more electronics, semiconductor and software industry. The advanced collision avoidance systems, and ultimately pave software implementing the automotive functionality is the way for the self-driving car. encapsulated in software components. Standardization of Conventionally, each technology is implemented in a the interfaces is central to support scalability and separate embedded control unit (ECU) inside the car. Fig. transferability of functions. Any standard-conformant 1 shows the system architecture of the conventional ECU. implementation of a software component can be integrated Depending on the complexity of the technology, one or with substantially reduced effort in a system. multiple applications run on the ECU. Consequently, Figure 2 shows the system architecture of AUTOSAR either the technology is limited by the computing power of [3]. The software components encapsulate an application the ECU or the computing power is achieved at a higher which runs on the AUTOSAR infrastructure. They have cost. well-defined interfaces. The software components As the industry moves toward more advanced driving communicate with each other through a virtual function experiences, systems will become more complex. It will be bus, which is the sum of all communication mechanisms increasingly difficult to support the growing number of provided by AUTOSAR on an abstract (technology technologies if each one is a separate ECU with its own independent) level. computer and software. The best strategy will be a In contrast to the purely virtual specification of the decentralized model that distributes functions and uses communication topology and interaction between multiple computing units to carry them out [2]. components through virtual function bus, the runtime This paper puts the focus on the development of environment provides an actual implementation for these in-vehicle distributed computing platform. Based on the artifacts. It could be said that the runtime environment provides an actual representation of the virtual concepts of

中華民國第十九屆車輛工程學術研討會，桃園創新技術學院機械工程系，台灣中壢，2014 年 11 月 14 日。 The 19th National Conference on Vehicle Engineering, Nov. 14, 2014, TIIT, Jhongli, Taiwan. E – 012 ------the virtual function bus for one specific ECU. aims to development in-vehicle networking solution that Below the runtime environment is the basic software, can leverage technology already proven in IEEE standard which provides the infrastructure for execution on an ECU. and reduce the cost of ownership. BroadR-Reach Ethernet It consists of AUTOSAR services, communication, ECU is the point-to-point ethernet PHY technology adopted by abstraction, complex device drivers, operation system, and the OPEN Alliance [5]. microcontroller abstraction. Basic software layer is BroadR-Reach Ethernet provides features that are standardized software that offers both hardware dependent comparable to the IEEE 802.3-2012 standard ethernet. It and hardware independent services to higher layers. This operates in full-duplex at 100 Mbps. Unlike the layer makes the higher layers hardware independent. convention ethernet PHY, however, BroadR-Reach PHY Memory interfaces and interfaces to communication buses uses one pair of unshielded twisted pair cable, as shown in like LIN, CAN and FlexRay are some examples. Fig. 3. The only change is physical wire, higher layers remain unchanged. BroadR-Reach Ethernet provides huge savings on cabling cost, weight, and thickness. Moreover, it meets automotive EMC/EMI requirements by adopting advanced DSP technologies.

Figure 2. System architecture of AUTOSAR [3] Figure 3. Illustration of BroadR-Reach Ethernet [5]

(b) Automotive Ethernet Besides controller area network, ext-generation ADAS The Proposed In-Vehicle Distributed Computing systems require high speed communication networks. Platform Ethernet is one of several possible technologies that can Figure 4 shows the proposed ECU system architecture. meet this demand. In fact, ethernet has now been widely It is a layered architecture which consists of three main accepted by the automotive industry as the preferred layers. The lower most layer is the ECU hardware, the interface for on-board-diagnostics in next generation of middle layer is ECU abstraction layer, and the upper most cars [4]. layer is the application layer. Ethernet was standardized by the IEEE 802.3 working group. Currently, the standardization of automotive ethernet is driven by the OPEN Alliance SIG [5], which

Application SW-C 1 SW-C 2 SW-C 3 Layer port_ port_ port_thread port_gpu Comm_rx comm_tx port_sensor

Commun. Sensor API API pthread/ OpenCL ECU OpenMP API API Android/ Abstraction Linux OS Layer Commun. Sensor Driver Driver

Multi-core GPU Transceiver Sensor ARM CPU ECU Hardware

Figure 4. Proposed ECU system architecture.

中華民國第十九屆車輛工程學術研討會，桃園創新技術學院機械工程系，台灣中壢，2014 年 11 月 14 日。 The 19th National Conference on Vehicle Engineering, Nov. 14, 2014, TIIT, Jhongli, Taiwan. E – 012 ------The layers have different responsibilities. The correction, perspective transformation, and composite application layer contains all application specific software view synthesis. Firstly, the distortion correction is to components. The ECU abstraction layer abstracts software remove fisheye affect in original input frames by applying components from hardware component. It contains ECU inverse transformation of distortion function. Secondly, the specific modules as well as general modules. In this paper, perspective transformation is to transform the four input the software modules cover CPU/GPU computing, frames so that all the views are properly registered with ethernet communication, and image sensing. Drivers are the ground plane. Lastly, the composite view synthesis is involved in the software modules to enable generalized to create a composite around view from the four input usage of different ECU hardware. The ports in application frames. layer enable communication between the software The around view monitor system is ported to the component and the software modules in ECU abstraction distributed computing platform. Figure 7 shows the system layer. architecture of the prototype of the around view monitor As shown in Fig. 5, the ECU connects to each other system, which consists of five ECUs. On the left hand side through a high speed ethernet to realize a distributed are four ECUs attached with a fisheye camera. In the computing system. The application is divided into application layer of these four ECUs, a camera software software components deployed on different ECUs. The component and a communication software component are software components can communicate with each other deployed, which are responsible for image capturing and through the well-defined ports, which provide access to frame transmission, respectively. On the right hand side is the high speed ethernet. Generally speaking, one software the other ECU receiving the four input frames. Similarly, a developer can design the application, consisting of communication software component for frame receiving interconnected software components, independent from and a display software component for displaying the the involved hardware. No knowledge about the network composite view are deployed in the application layer of is required since the software architecture hides the this ECU. network from the application. Therefore, application Beside the software components for image capturing, developers can put their effort on the design of algorithms frame transmission, and image displaying, the software of various applications. Moreover, the platform can reach component of the around view monitor system has to be high computing power by utilizing multiple ECUs. ported as well. Here, the around view monitor application is divided into two software components, namely AVM Prototype of the Developed Around View Monitor SW-C1 and AVM SW-C2, respectively. The SW-C1 System implements the distortion correction, while the SW-C2 implements the perspective transformation and composite Around view monitor system is an emerging ADAS view synthesis. These two software components are technology that assists the driver with parking the car o deployed on different ECUs, resulting in high computing safely by providing a top-down view of the 360 power by taking the advantage of parallel computing. The surroundings of the vehicle. Here, the system consists of interactions between the software components are through four fisheye cameras mounted around the car. From these virtual communication bus, which are realized by the ECU camera inputs, a composite view of the car is synthesized abstraction layer along with the ECU hardware. Therefore, and shown to the driver. system developers can put their effort on the design of the Figure 6 shows the flowchart of the developed around algorithm of around view monitor system, rather than view monitor system. Only conceptual descriptions are being dragged by image capturing, displaying, firmware/ given here, the detailed algorithm of the around view driver coding and so on. monitor system is out of the scope of this paper. The algorithm consists of three key components: distortion

Software Component 1 Software Component 2 Software Component 3

ECU Abstraction ECU Abstraction ECU Abstraction OS OS OS Layer Layer Layer

ECU-Hardware X ECU-Hardware Y ECU-Hardware Z

High Speed Data Bus

Figure 5. Proposed distributed computing platform.

中華民國第十九屆車輛工程學術研討會，桃園創新技術學院機械工程系，台灣中壢，2014 年 11 月 14 日。 The 19th National Conference on Vehicle Engineering, Nov. 14, 2014, TIIT, Jhongli, Taiwan. E – 012 ------

Distortion Correction

Perspective Transformation

Composite View Synthesis

Figure 6. Flowchart of the developed around view monitor system.

Ethernet ECU

ECU ECU

ECU

Application Camera AVM Commun. Application Commun. AVM Display Layer SW-C SW-C1 SW-C Layer SW-C SW-C2 SW-C

Virtual Bus Virtual Bus

ECU Camera Ethernet ECU Ethernet Display Abstraction API API Abstraction API API Layer Layer Camera Ethernet Ethernet Display Hardware Hardware Hardware Hardware Hardware Hardware

Figure 7. System architecture of the developed around view monitor system.

Figure 8 show the demo-car of the developed around view monitor system, where two types of ECUs are used. One is Raspberry Pi [6] and the other is Arndale Octa Board [7]. The Raspberry Pi, equipped with a single-core ARM CPU and a 500-meagpixel camera, is used to capture the wide-angle image and perform distortion correction, while the Arndale Octa Board, equipped with an eight-core ARM CPU and OpenGL image displaying, is used to perform the perspective transformation and composite view synthesis. Thanks to the enhanced computing power, the frame rate of the around view Figure 8. Demo-car of the developed around view system can reach 30 fps. monitor system.

中華民國第十九屆車輛工程學術研討會，桃園創新技術學院機械工程系，台灣中壢，2014 年 11 月 14 日。 The 19th National Conference on Vehicle Engineering, Nov. 14, 2014, TIIT, Jhongli, Taiwan. E – 012 ------Conclusions This paper has presented the development of an in-vehicle distributed computing platform, which consists of multiple ECUs connected through high speed Ethernet. Instead of running on a single ECU, the application software is divided into components and deployed among multiple ECUs. In so doing, a distributed computing platform is established and computing-intensive applications on vehicles can be achieved. Besides, this paper has also presented the porting of an around view monitor system on the developed computing platform. Resulting from the enhanced computing power, the frame rate of the around view system can reach 30 fps.

REFERENCES [1] R. Okuda, Y. Kajiwara, and K. Terashima, "A survey of technical trend of ADAS and autonomous driving," in Proceedings of 2014 International Symposium on VLSI Technology, Systems and Application, pp. 1-4, 2014. [2] A. D. Kshemkalyani and Mukesh Singhal, Distributed Computing: Principles, Algorithms, and Systems, Cambridge University Press, 2008. [3] http://www.autosar.org, The AUTOSAR Development Partnership. [4] ISO 13400 Road vehicles - Diagnostic communication over Internet Protocol. [5] http://www.opensig.org, The OPEN Alliance (One-Pair Ether-Net) Special Interest Group (SIG) [6] http://www.raspberrypi.org, The Raspberry Pi Foundation. [7] http://www.arndaleboard.org, The Arndale Board.