2020 AUTONOMOUS VEHICLE TECHNOLOGY REPORT The guide to understanding the current state of the art in hardware & software for self-driving vehicles. sponsored by For the people who aim to create a better version of the future. Contributors 6 Introduction 10 How this report came to be: a collaborative effort 11 Levels of Autonomy 14 Sensing 17 Environmental mapping 18 Passive sensors 18 Active Sensors 22 Choice of Sensors 28 Geolocalization 32 Maps 33 Thinking & Learning 35 SLAM and Sensor Fusion 35 Machine Learning Methods 38 Gathering Data 40 Path Planning 42 Acting 44 Architectures: Distributed versus Centralized 45 Power, Heat, Weight, and Size challenges 47 User experience 48 Inside the vehicle 51 The external experience 53 Communication & Connectivity 55 DSRC or C-V2X 59 Use case: Autonomous Racing 62 Summary 66 About Nexperia 68 About Wevolver 70 References 72 Contributors Editor in Chief Jordan Sotudeh Akbar Ladak Joakim Svennson Cover Photographer Many thanks to Bram Geenen Los Angeles, USA Bangalore, India Norrköping, Sweden Benedict Redgrove The people at Roborace, specifically Victo- Amsterdam, The Netherlands Senior Strategic Analyst at NASA Jet Pro- Founder, CEO, Kaaenaat, which develops Senior ADAS Engineer, Function Owner London, United Kingdom ria Tomlinson and Alan Cocks. CEO Wevolver. pulsion Laboratory. autonomous robots for logistics, retail and Traffic Sign Recognition and Traffic Light Benedict has a lifelong fascination with Master International Science and Technol- security use cases, as well as Advanced Recognition at Veoneer. technology, engineering, innovation and Edwin van de Merbel, Dirk Wittdorf, Petra ogy Policy, Elliott School of International Driver Assistance Systems (ADAS) for 2- & MSc. Media Technology, Linköping Univer- industry, and is a dedicated proponent of Beekmans - van Zijll and all the other Editors Affairs, Washington DC, USA. 4- wheeler vehicles for chaotic driving sity, Sweden. modernism. This has intuitively led him people at Nexperia for their support. conditions in Asia & Africa. to capturing projects and objects at their Ali Nasseri Matthew Nancekievill Master in Electrical Engineering, Georgia Fazal Chaudry most cutting edge. He has created an aes- Our team at Wevolver; including Sander Vancouver, Canada Manchester, United Kingdom Institute of Technology. Headington, United Kingdom thetic of photography that is clean, pure Arts, Benjamin Carothers, Seth Nuzum, Lab manager at the Programming Lan- Postdoctoral researcher submersible ro- Product Development Engineer. and devoid of any miscellaneous informa- Isidro Garcia, Jay Mapalad, and Richard guages for Artificial Intelligence (PLAI) botics, University of Manchester, UK. Zeljko Medenica Master of Science, Space Studies, Interna- tion, winning him acclaim and numerous Hulskes. Many thanks for the proofreads research group at the University of British PhD. Electrical and Electronics Engineer- Birmingham, Michigan, USA tional Space University, Illkirch Graffensta- awards. and feedback. Columbia. ing, University of Manchester, UK. Principal Engineer and Human Machine den, France. Redgrove has amassed a following and Previously Chair of the Space Generation CEO Ice Nine Robotics. Interface (HMI) Team Lead at the US client base from some of the most ad- The Wevolver community for their support, Advisory Council. R&D Center of Changan, a major Chinese Shlomit Hacohen vanced companies in the world. A career knowledge sharing, and for making us Cum Laude PhD. in Engineering Physics, Jeremy Horne automobile manufacturer. Previously led Tel Aviv, Israel spent recording the pioneering technol- create this report. Politecnico di Torino. San Felipe, Baja California, Mexico research on novel and intuitive HMI for VP of Marketing at Arbe Robotics; develop- ogy of human endeavours has produce a President Emeritus of the American Asso- Advanced Driver Assistance Systems at ing ultra high-resolution 4D imaging radar photographic art form that gives viewers a Many others that can’t all be listed here Adriaan Schiphorst ciation for the Advancement of Science, Honda. technology. window into an often unseen world, such have helped us in big or small ways. Thank Amsterdam, The Netherlands Southwest Division. PhD. in Electrical and Computer Engineer- MBA at Technion, the Israel Institute of as Lockheed Martin Skunk Works, UK MoD, you all. Technology journalist. Science advisor and curriculum coordina- ing from the University of New Hampshire. Technology European Space Agency, British Aerospace MSc Advanced Matter & Energy Physics at tor at the Inventors Assistance Center. and NASA. Whether capturing the U-2 re- Beyond the people mentioned here we University of Amsterdam and the California Ph.D. in philosophy from the University of Maxime Flament Designer connaissance pilots and stealth planes, the owe greatly to the researchers, engineers, Institute of Technology. Florida, USA. Brussels, Belgium Navy Bomb Disposal Division or spending writers, and many others who share their Previously editor at Amsterdam Science Chief Technology Officer, 5G Automotive Bureau Merkwaardig time documenting the NASA’s past, present knowledge online. Find their input in the Journal. Drue Freeman Association (5GAA) Amsterdam, The Netherlands and future, Benedict strives to capture references. Cupertino, California, USA PhD. in Wireless Communication Systems, Award winning designers Anouk de the scope and scale of advancements and Contributing Experts CEO of the Association for Corporate Chalmers University of Technology, Göte- l’Ecluse and Daphne de Vries are a creative what they mean to us as human beings. Media Partner Growth, Silicon Valley. borg, Sweden. duo based in Amsterdam. They are special- His many awards include the 2009 AOP Norman di Palo Former Sr. Vice President of Global ized in visualizing the core of an artistic Silver, DCMS Best of British Creatives, and Supplyframe Rome, Italy Automotive Sales & Marketing for NXP Mark A. Crawford Jr. problem. Bureau Merkwaardig initiates, the Creative Review Photography Annual Supplyframe is a network for electronics Robotics and Machine Learning researcher, Semiconductors. Baoding City, China develops and designs. 2003, 2008, and 2009. design and manufacturing. The company conducting research on machine learning Board Director at Sand Hill Angels. Adviso- Chief Engineer for Autonomous Driving At Wevolver we are a great fan of Bene- provides open access to the world’s largest for computer vision and control at the Isti- ry Board Member of automotive companies Systems at Great Wall Motor Co. Illustrations dict’s work and how his pictures capture a collection of vertical search engines, tuto Italiano di Tecnologia, Genova, Italia. Savari and Ridar Systems, and Advisory PhD. Industrial and Systems Engineering spirit of innovation. We’re grateful he has supply chain tools, and online communi- Cum Laude MSc. Engineering in Artifi- Board Member of Silicon Catalyst, a semi- - Global Executive Track, at Wayne State Sabina Begović enabled us to use his beautiful images of ties for engineering. Their mission is to cial Intelligence and Robotics, Sapienza conductor focused incubator. University. Padua, Italy the Robocar to form the perfect backdrop organize the world of engineering knowl- Università di Roma, and graduate of the Pi Bachelor of Science in Electrical Engineer- Previously Technical Expert at Ford. Croation born Sabina is a visual and inter- for this report. edge to help people build better hardware School of Artificial Intelligence. ing, San Diego State University, MBA from action designer. She obtained a Master’s in products, and at Wevolver we support that Pepperdine University, Los Angeles. William Morris Visual and Communication Design at Iuav, aspiration and greatly appreciate that Sup- Detroit, Michigan, USA University of Venice, and a Masters in Art plyframe contributes to the distribution of Automotive engineer Education at the Academy of Applied Art, this report among their network. Rijeka, Croatia. 6 7 “It’s been an enormously difficult, complicated slog, and it’s far more complicated and involved than we thought it would be, but it is a huge deal.” Nathaniel Fairfield, distinguished software engineer and leader of the ‘behavior team’ at Waymo, December 2019 [1] 8 9 Introduction Bram Geenen Motorized transportation has changed Therefore, this report’s purpose is to How this report This report would not have been Editor in Chief, the way we live. Autonomous vehicles enable you to be up to date and un- possible without the sponsorship of CEO of Wevolver are about to do so once more. This derstand autonomous vehicles from a came to be: Nexperia, a semiconductor company evolution of our transport - from hors- technical viewpoint. shipping over 90Bn components an- es and carriages, to cars, to driverless We have compiled and centralized the a collaborative nually, the majority of which are with- vehicles, - has been driven by both information you need to understand in the automotive industry. Through technical innovation and socioeco- what technologies are needed to effort their support, Nexperia shows a nomic factors. In this report we focus develop autonomous vehicles. We will commitment to the sharing of objec- on the technological aspect. elaborate on the engineering consid- Once the decision was made to create tive knowledge to help technology erations that have been and will be this report, we asked our communi- developers innovate. This is the core Looking at the state of autonomous made for the