Visuella Förarhjälpmedel För Fjärrstyrning Av Fordon Tobias Matts Anton Sterner
Total Page:16
File Type:pdf, Size:1020Kb
LiU-ITN-TEK-A--20/011--SE Visuella förarhjälpmedel för fjärrstyrning av fordon Tobias Matts Anton Sterner 2020-05-28 Department of Science and Technology Institutionen för teknik och naturvetenskap Linköping University Linköpings universitet nedewS ,gnipökrroN 47 106-ES 47 ,gnipökrroN nedewS 106 47 gnipökrroN LiU-ITN-TEK-A--20/011--SE Visuella förarhjälpmedel för fjärrstyrning av fordon Examensarbete utfört i Medieteknik vid Tekniska högskolan vid Linköpings universitet Tobias Matts Anton Sterner Handledare Karljohan Lundin Palmerius Examinator Daniel Jönsson Norrköping 2020-05-28 Upphovsrätt Detta dokument hålls tillgängligt på Internet – eller dess framtida ersättare – under en längre tid från publiceringsdatum under förutsättning att inga extra- ordinära omständigheter uppstår. Tillgång till dokumentet innebär tillstånd för var och en att läsa, ladda ner, skriva ut enstaka kopior för enskilt bruk och att använda det oförändrat för ickekommersiell forskning och för undervisning. Överföring av upphovsrätten vid en senare tidpunkt kan inte upphäva detta tillstånd. All annan användning av dokumentet kräver upphovsmannens medgivande. För att garantera äktheten, säkerheten och tillgängligheten finns det lösningar av teknisk och administrativ art. Upphovsmannens ideella rätt innefattar rätt att bli nämnd som upphovsman i den omfattning som god sed kräver vid användning av dokumentet på ovan beskrivna sätt samt skydd mot att dokumentet ändras eller presenteras i sådan form eller i sådant sammanhang som är kränkande för upphovsmannens litterära eller konstnärliga anseende eller egenart. För ytterligare information om Linköping University Electronic Press se förlagets hemsida http://www.ep.liu.se/ Copyright The publishers will keep this document online on the Internet - or its possible replacement - for a considerable time from the date of publication barring exceptional circumstances. The online availability of the document implies a permanent permission for anyone to read, to download, to print out single copies for your own use and to use it unchanged for any non-commercial research and educational purpose. Subsequent transfers of copyright cannot revoke this permission. All other uses of the document are conditional on the consent of the copyright owner. The publisher has taken technical and administrative measures to assure authenticity, security and accessibility. According to intellectual property law the author has the right to be mentioned when his/her work is accessed as described above and to be protected against infringement. For additional information about the Linköping University Electronic Press and its procedures for publication and for assurance of document integrity, please refer to its WWW home page: http://www.ep.liu.se/ © Tobias Matts, Anton Sterner Linköping University | Department of Science and Technology Master’s thesis, 30 ECTS | Medieteknik 2020 | LIU-ITN-TEK-A--20/011-SE Vision-based Driver Assistance Systems for Teleoperation of On- Road Vehicles – Compensating for Impaired Visual Perception Capabilities Due to Degraded Video Quality Visuella förarhjälpsystem för fjärrstyrning av fordon Tobias Matts Anton Sterner Supervisor : Karljohan Lundin Palmerius Examiner : Daniel Jönsson Linköpings universitet SE–581 83 Linköping +46 13 28 10 00 , www.liu.se Upphovsrätt Detta dokument hålls tillgängligt på Internet - eller dess framtida ersättare - under 25 år från publicer- ingsdatum under förutsättning att inga extraordinära omständigheter uppstår. Tillgång till dokumentet innebär tillstånd för var och en att läsa, ladda ner, skriva ut enstaka ko- pior för enskilt bruk och att använda det oförändrat för ickekommersiell forskning och för undervis- ning. Överföring av upphovsrätten vid en senare tidpunkt kan inte upphäva detta tillstånd. All annan användning av dokumentet kräver upphovsmannens medgivande. För att garantera äktheten, säker- heten och tillgängligheten finns lösningar av teknisk och administrativ art. Upphovsmannens ideella rätt innefattar rätt att bli nämnd som upphovsman i den omfattning som god sed kräver vid användning av dokumentet på ovan beskrivna sätt samt skydd mot att dokumentet ändras eller presenteras i sådan form eller i sådant sammanhang som är kränkande för upphovsman- nens litterära eller konstnärliga anseende eller egenart. För ytterligare information om Linköping University Electronic Press se förlagets hemsida http://www.ep.liu.se/. Copyright The publishers will keep this document online on the Internet - or its possible replacement - for a period of 25 years starting from the date of publication barring exceptional circumstances. The online availability of the document implies permanent permission for anyone to read, to down- load, or to print out single copies for his/hers own use and to use it unchanged for non-commercial research and educational purpose. Subsequent transfers of copyright cannot revoke this permission. All other uses of the document are conditional upon the consent of the copyright owner. The publisher has taken technical and administrative measures to assure authenticity, security and accessibility. According to intellectual property law the author has the right to be mentioned when his/her work is accessed as described above and to be protected against infringement. For additional information about the Linköping University Electronic Press and its procedures for publication and for assurance of document integrity, please refer to its www home page: http://www.ep.liu.se/. Tobias Matts © Anton Sterner Abstract Autonomous vehicles is going to be a part of future transport of goods and people, but to make them usable in unpredictable situations presented in real traffic, there is need for backup systems for manual vehicle control. Teleoperation, where a driver controls the vehicle remotely, has been proposed as a backup system for this purpose. This technique is highly dependent on stable and large wireless network bandwidth to transmit high resolution video from the vehicle to the driver station. Reduction in network bandwidth, resulting in a reduced level of detail in the video stream, could lead to a higher risk of driver error. This thesis is a two part investigation. One part looking into whether lower resolution and increased lossy compression of video at the operator station affects driver performance and safety of operation during teleoperation. The second part covers implementation of two vision-based driver assistance systems, one which detects and highlights vehicles and pedestrians in front of the vehicle, and one which detects and highlights lane markings. A driving test was performed at an asphalt track with white markings for track boundaries, with different levels of video quality presented to the driver. Reducing video quality did have a negative effect on lap time and increased the number of times the track boundary was crossed. The test was performed with a small group of drivers, so the results can only be interpreted as an indication toward that video quality can negatively affect driver performance. The vision-based driver assistance systems for detection and marking of pedestrians was tested by showing a test group pre-recorded video shot in traffic, and them reacting when they saw a pedestrian about to cross the road. The results of a one-way analysis of variance, shows that video quality significantly affect reaction times, with p = 0.02181 at signifi- cance level α = 0.05. A two-way analysis of variance was also conducted, accounting for video quality, the use of a driver assistance system marking pedestrians, and the interac- tion between these two. The results point to that marking pedestrians in very low quality video does help reduce reaction times, but the results are not significant at significance level α = 0.05. Acknowledgments The authors would like to thank Voysys for letting us conduct our thesis there. Special thanks to our mentor Gabriella Rydenfors for putting up with us during our time at Voysys. We would also like to thank our supervisor Karljohan Lundin Palmerius for great advice during this process, as well as our examiner Daniel Jönsson. iv Contents Abstract iii Acknowledgments iv Contents v List of Figures vii List of Tables viii 1 Introduction 1 1.1 Motivation . 1 1.2 Aim............................................ 2 1.3 Research questions . 2 1.4 Delimitations . 2 2 Background 3 2.1 Autonomous Vehicles . 3 2.2 Teleoperation . 5 2.3 Advanced Driver Assistance Systems . 5 2.4 Pilot study: ADAS Categorization . 5 2.5 Voysys Software Suite for Teleoperation . 6 3 Theory 7 3.1 Visual Acuity . 7 3.2 Focal and Ambient Vision . 10 3.3 Situation Awareness . 11 3.4 Video Quality & Compression . 11 4 Video Quality & Vision in Teleoperation 12 4.1 Testing Visual Acuity in Streamed Video . 12 4.2 Lane Keeping Test . 14 4.3 Discussion . 15 4.4 Analysis of the Test Results . 19 5 Vision-based Driver Assistance Systems 20 5.1 System Overview . 20 5.2 Inference Plugin Implementation . 23 5.3 Visualization Plugin Implementation . 32 5.4 Systems Results and Performance . 33 5.5 User test . 34 5.6 Discussion . 37 6 Conclusions 43 v 6.1 Could It Have Been Made Simpler? . 44 6.2 The Work In A Wider Context . 44 6.3 Future Work . 44 6.4 Source Criticism . 45 Bibliography 46 Appendices 50 A ADAS Categorization 51 A.1 Active Systems . 51 A.2 Passive Systems . 52 A User test video description 53 vi List of Figures 3.1 Snellen chart and ETDRS chart. 8 3.2 Visual angle V - the angle spanned by the extremities A, B, of an object. 8 4.1 Visual acuity measures of the combination of each setting. 13 4.2 High resolution with different levels of compression. 14 4.3 Medium resolution with different levels of compression. 15 4.4 Low resolution with different levels of compression. 16 4.5 Car lined up to start the test course. 17 5.1 An overview of the system design. 21 5.2 The front- and back camera feeds are aligned in the streamer to make up a frame to be sent to the player. 21 5.3 ................................................ 22 5.4 Illustration of the packed RGB format. Pixel values are stored next to each other in memory.