MASARYK UNIVERSITY FACULTY}w¡¢£¤¥¦§¨ OF I !"#$%&'()+,-./012345<yA|NFORMATICS Road lane detection for Android BACHELOR THESIS Jakub Medvecký-Heretik Brno, spring 2014 Declaration I do hereby declare, that this paper is my original authorial work, which I have worked on by my own. All sources, references and liter- ature used or excerpted during elaboration of this work are properly cited and listed in complete reference to the due source. Jakub Medvecký-Heretik Advisor: Mgr. Dušan Klinec Acknowledgement I would like to thank my supervisor, Mgr. Dušan Klinec for his con- tinual support and advice throughout this work. I would also like to thank my family and friends for their support. Abstract The aim of this bachelor thesis was to analyze the possibilities of im- age processing techniques on the Android mobile platform for the purposes of a computer vision application focusing on the problem of road lane detection. Utilizing the results of the analysis an appli- cation was created, which detects lines and circles in an image in real time using various approaches to segmentation and different imple- mentations of the Hough transform. The application makes use of the OpenCV library and the results achieved with its help are com- pared to the results achieved with my own implementation. Keywords computer vision, road lane detection, circle detection, image process- ing, segmentation, Android, OpenCV, Hough transform Contents 1 Introduction ............................ 3 2 Computer vision ......................... 5 2.1 Image processing ...................... 5 2.2 Techniques .......................... 6 2.3 Existing libraries ...................... 6 2.3.1 OpenCV . 6 2.3.2 VXL . 7 2.3.3 ImageJ . 7 2.3.4 libCVD . 7 2.4 Android ........................... 8 3 Region of interest ......................... 9 4 Segmentation ........................... 10 4.1 Thresholding ......................... 11 4.1.1 Global (static) thresholding . 11 4.1.2 Local (adaptive) thresholding . 12 Noise removal . 13 4.2 Canny edge detector .................... 14 4.3 Overview ........................... 16 5 Hough transform ......................... 17 5.1 Hough space ......................... 17 5.2 Straight line detection ................... 19 5.2.1 Algorithm . 20 5.3 Circle detection ....................... 21 5.3.1 Circles with known radii . 22 5.3.2 Circles with unknown radii . 23 5.4 Detecting non-analytical shapes . 24 6 Android application - Hough . 25 6.1 Target platform ....................... 25 6.2 Implementation ....................... 26 6.2.1 Segmentation . 26 6.2.2 Line detection . 26 Overview . 28 6.2.3 Vanishing point . 28 6.2.4 Circle detection . 29 7 Conclusion ............................. 31 1 7.1 Future development .................... 31 Bibliography ............................. 33 A FpsMeter logs ........................... 36 B Screenshot gallery ........................ 40 C Electronic attachments ...................... 42 2 1 Introduction Car manufacturers are constantly developing new innovative tech- nologies, which help drivers avoid collisions and improve safety on the road. With the arrival of cameras and various sensors the re- search in this area has greatly accelerated. Today’s modern smart- phones are compact, portable and equipped with powerful hardware sensors and high resolution cameras. If accompanied by proper soft- ware, these smartphones can be easily turned into driver assistants and help in driving safely. In order to evaluate possibly dangerous situations, such software must be able to detect and analyze the key elements in an image captured from a camera: the road, the road’s boundaries, traffic signs, potential obstacles, etc. This thesis describes the computer vision problem of road lane detection and the real-time processing of an image acquired from the smartphone’s camera on the Android mobile platform. The phases of this processing happen in the following order: 1. a color image is retrieved from the device’s camera 2. the color image is converted to grayscale 3. a binary image with visible road lanes is produced by segmen- tation of the grayscale image 4. lines are detected in the binary image using the Hough trans- form 5. the detected lines are overlaid in the color image 6. the resulting color image with detected lines is displayed on the screen of the device The second chapter of this work is an introduction to computer vision and the image processing. At the end of this chapter I compare the existing image processing libraries and I choose the most suitable one for my application. In the next chapter I identify the region of interest for the segmen- tation phase. The fourth chapter specifies the different approaches which can be used for extracting the region of interest from an image and deter- mines which technique is the most suitable one for my application. 3 1. INTRODUCTION The fifth chapter explains how the Hough transform for detecting different shapes in an image works. Different implementations of the Hough transform, their compar- ison and their integration to the resulting application are described in the sixth chapter. The last chapter provides a functionality summary of the result- ing application and outlines the possibilites of further improvement and extension of this application in the future. 4 2 Computer vision Computer vision (CV) is a field in computer science where comput- ers are used to collect visual information from real world, process, analyze and understand it, then deduce a profitable result from this information. CV is used to help humans perform their daily tasks and enhance performance in situations where CV is able to obtain far better results and can be more perceptive than human vision. Applications of CV range from tasks such as industrial machine vision systems to research into artificial intelligence and computers or robots that can comprehend the world around them [1]. One of many areas computer vision investigates is the road lane detection. 2.1 Image processing Digital image processing and analysis are the fields most closely re- lated to CV and are critically important in solving CV problems. CV’s main goal is to emulate human vision in computers and achieve results such as object recognition, feature detection, autonomous driving etc. In order to achieve that, the acquired input image must firstly be processed so that only information relevant for computa- tion of the required goal is present in the output image. This is called digital image processing (DIP). DIP is the application of transformations and filters to digital im- ages (most often 2D) which yields a processed output image. Most common uses of DIP are: reconstructing damaged or miss- ing information in an image, preparing or enhancing images for fur- ther use, registering, segmenting, identificating and measuring ob- jects in images, etc. For further introductory information into the field of Image pro- cessing I recommend a book Digital image processing [2]. 5 2. COMPUTER VISION 2.2 Techniques There is a lot of different techniques and methods in DIP which we can use to transform input images. The selection of the appropriate technique strongly depends on what result we want to achieve. Some of the DIP techniques can even be combined. If we experiment in the order in which we apply them to an image, we get different results. Specific techniques that we need to apply to an image if we want to get the output image ready for road lane detection are covered more in-depth in the next chapters. 2.3 Existing libraries There already are many existing libraries which implement most of the DIP techniques and are also highly optimized. In this chapter I mention just the most suitable ones for application on the Android mobile platform. 2.3.1 OpenCV Open Source Computer Vision Library1 (OpenCV) is an open source computer vision and machine learning software library. It was built to provide a common infrastructure for computer vision applications and to accelerate the widespread use of machine perception. The library has more than 2500 optimized algorithms. It is the most popular CV library having a user community of more than 47 thousand people and an estimated number of downloads exceeding 7 million. The library is used extensively in research groups and com- panies such as Google, Yahoo, Microsoft, Intel, IBM and others. OpenCV is written natively in C++, has interfaces in C, Python, Java and MATLAB as well and supports Windows, Linux, Android and Mac OS. The latest version is 2.4.8 which was released on 31th December 2013 [3]. 1. http://opencv.org 6 2. COMPUTER VISION 2.3.2 VXL VXL2 (the Vision-something-Libraries) is a collection of libraries de- signed for computer vision research and implementation. It is writ- ten in C++ and is designed to be portable over many platforms. The idea is to replace X with one of many letters, e.g., G (VGL) is a geometry library, N (VNL) is a numerics library, I (VIL) is an image processing library, etc. For a more detailed description of the libraries see the VXL book [4]. 2.3.3 ImageJ ImageJ3 is a public domain, Java-based image processing program developed at the National Institutes of Health. It was designed with an open architecture that provides extensibility via Java plugins and recordable macros. Custom acquisition, analysis and processing plugins can be de- veloped using ImageJ’s built-in editor and a Java compiler. User- written plugins make it possible to solve many image processing and analysis problems. ImageJ’s plugin architecture and built-in development environ- ment has made it a popular platform for teaching image processing. 2.3.4 libCVD libCVD4 is a very portable and high performance C++ library for computer vision, image, and video processing. The library is designed in a loosely-coupled manner, so that parts can be used easily in iso- lation if the whole library is not required. The emphasis is on providing simple and efficient image and video handling and high quality implementations of common low- level image processing function.