Honda Research Institute Europe GmbH https://www.honda-ri.de/ Analysis of a Speech-Based Intersection Assistant in Real Urban Traffic Dennis Orth, Nico Steinhardt, Bram Bolder, Mark Dunn, Dorothea Kolossa, Martin Heckmann 2018 Preprint: This is an accepted article published in The 21st IEEE International Conference on Intelligent Transportation Systems. The final authenticated version is available online at: https://doi.org/[DOI not available] Powered by TCPDF (www.tcpdf.org) Analysis of a Speech-Based Intersection Assistant in Real Urban Traffic Dennis Orth1;2, Nico Steinhardt1, Bram Bolder1, Mark Dunn1, Dorothea Kolossa2, Martin Heckmann1 1Honda Research Institute Europe, Offenbach/Main, Germany 2Institute of Communication Acoustics, Ruhr-Universitat¨ Bochum, Bochum, Germany Email: fmartin.heckmann, nico.steinhardt, bram.bolder, [email protected], fdennis.orth, [email protected] Preprint version. Paper published in: The 21st IEEE International Conference on Intelligent Transportation Systems. c 2018 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works. Analysis of a Speech-Based Intersection Assistant in Real Urban Traffic Dennis Orth∗y, Nico Steinhardty, Bram Boldery, Mark Dunny, Dorothea Kolossa∗, and Martin Heckmanny ∗Ruhr University Bochum, Faculty of Electrical Engineering and Information Technology, Institute of Communication Acoustics, Germany, Bochum Email: fdennis.orth, [email protected] yHonda Research Institute Europe GmbH, Germany, Offenbach am Main Email: fnico.steinhardt, bram.bolder, mark.dunn, [email protected] Abstract—Previously, we have presented a speech-based inter- as annoying and may lead to the driver ignoring them or section assistant prototype. The system is activated on-demand turning off the system. To overcome these limitations, we by the driver and gives afterwards, via speech, information on recently proposed the “Assistance on demand” (AOD) concept suitable gaps between the traffic vehicles approaching from the right. It is comparable to a front seat passenger which helps in [18]. Particularly challenging for drivers is turning left at the maneuver decision for an intended turn left. This system an unsignalized intersection from a subordinate road into a has assumed a more or less constant flow of the traffic. To superordinate road, especially when the traffic density is high also handle situations of more dynamic urban traffic, including [19]. In the AOD approach the idea of a cooperative front seat vehicles that may be slowing down or stopping, we have now passenger is transferred to the interaction with the assistance extended our previous approach by a dynamic vehicle model. This model predicts the future traffic vehicle state based on second- system. The driver can activate it via speech and it will order vehicle dynamics. We perform an in depth analysis of our give feedback, also via speech, on the current traffic situation system on a set of recordings under various traffic conditions. In to support her in the maneuver decision. Using speech the this analysis we compare in particular the previous and the novel system will inform the driver on possible gaps in the traffic vehicle model. Both approaches lead to a correct recommendation to make the turn. In [20] a similar approach for the use case in approximately 90% of the cases. Unexpectedly, the dynamic model does not lead to significant improvements in the system of ”turning left at a rural road with oncoming traffic” was behavior, despite its increased accuracy. applied in a simulator with positive results for the system. In contrast, the AOD approach is applied to a more demanding I. INTRODUCTION use case where a collaborative sharing of the driving task As stated in [1], [2], intersections are among the most is more beneficial. In addition, the AOD system is activated confusing and complex places in traffic, which often leads on demand therefore minimizes possible annoyance. In [21] to accidents. Therefore, with increasing technical possibilities and [22] systems are proposed which also assist in decision in the automotive industry, intersection assistance systems are making for oncoming traffic and crossing traffic, respectively, investigated to mitigate or prevent crashes at intersections [3]. by providing Head-up display (HUD) based support. Yet, These assistance systems often cover situations with oncoming in [18] we could demonstrate in a simulator study that a traffic [4], [5], but are now being extended to also handle system based on the AOD approach was well accepted by multi-directional collisions [6], [7]. Currently, those systems the participants and preferred compared to driving without mainly use LIDAR and RADAR as environment perception assistance or with a HUD based support. To further improve technology. Thus the assistance systems are limited to emer- the utility of the system, we have investigated individual gency situations only. It is expected that this disadvantage will drivers’ gap acceptance and developed methods to efficiently be mitigated in the future by applying vehicle-to-infrastructure estimate personalized gap recommendations [23], [24]. We and vehicle-to-vehicle communication approaches, which are could then show that these personalized recommendations under current research [8], [9], [10], [11]. Another reason clearly improved the acceptance of the system compared to for the limited operational range of the previous intersection identical recommendations for all drivers. They also enhanced assistants (mentioned above) is the rather low capability of the monitoring of the traffic situation and further decreased interpreting the surrounding environment. Thus, one can find the perceived workload [25]. plenty of research with regard to traffic participants’ maneuver In [26] we implemented the previously described speech- recognition [12] and intention prediction [13], [14], [15], [16], based on-demand intersection assistant in a prototype vehicle [17]. and tested it in real urban traffic to investigate which additional However, the previous mentioned assistance systems still challenges arise there for the design of the system. Based occasionally provide warnings when there is no danger. Ad- on the evaluations, we were able to design a system which ditionally, often the drivers have already perceived the danger enables tailored assistance while avoiding annoyance already and have adapted their maneuver planning accordingly. In in a wide range of traffic situations. In this work we provide both cases the warnings of the systems may be perceived a detailed analysis of one of the core elements of our system, namely the vehicle behavior estimation and prediction. It is A. General Dialog Logic the basis for the gap estimation and therefore determines what will be proposed to the driver. In [26] we assumed References of the system to traffic participants have to be that the traffic vehicles will drive at constant speed. However, unambiguous as they might otherwise confuse the driver. Due in our experiments we observed that vehicles frequently had to sensory limitations only the traffic participants dynamics to slow down and stop, e.g. due to pedestrians which were are known. With current sensors, additional features such as crossing the street. This behavior is not covered by the constant type (car, truck, pedestrian ...), size or color are not available, velocity model in [26] and leads to an underestimation or or can only be obtained at too late a point in time. Hence overestimation of the gaps. The latter is the more critical case, the system refers to all traffic participants as “vehicles.” as it could confuse the driver and generate risky situations. To achieve the required unambiguous reference, the system To overcome these limitations, we have also included in always makes reference to the traffic participant approaching our analysis a novel vehicle model based on second-order from the right with the shortest time of arrival. We will refer dynamics. In Sec. II we give a short overview of the system’s to this vehicle as the trigger vehicle (Fig. 1). In its approach components and how they work together to accomplish the several conditions are tested for the trigger vehicle, which desired assistance functionality. The dialog manager, which can trigger an announcement of the system. We refer to the controls the interaction with the driver, is described in Sec. III. vehicle following the trigger vehicle as the target vehicle as It is followed by the description of the two vehicle behavior the announcement is in most cases targeted at this vehicle. The estimation approaches (Sec. IV). After explaining our experi- reference point for the calculation of the time of arrival is in mental setup in Sec. V, we will show and discuss the results the center of the ego vehicle. We call this time ttrigger for the of our analysis (Sec. VI, Sec. VII). trigger vehicle. Consequently, we denote the time gap between the trigger and the target vehicle as tgap and measure it from II. SYSTEM OVERVIEW the front of the trigger vehicle to the front of the target vehicle. Our intersection assistant in [26] uses LIDAR sensors to Due to aforementioned sensor limitations, we currently don’t estimate the position