Unmanned Aerial Vehicle-Based Automobile License Plate Recognition System for Institutional Parking Lots
Total Page:16
File Type:pdf, Size:1020Kb
Proceedings of The 21st World Multi-Conference on Systemics, Cybernetics and Informatics (WMSCI 2017) Unmanned Aerial Vehicle-based Automobile License Plate Recognition System for Institutional Parking Lots Julian Dasilva, Ricardo Jimenez, Roland Schiller, Sanja Zivanovic Gonzalez Department of Mathematics and Computer Science, Barry University, Miami Shores, FL 33161, USA ABSTRACT parking areas. The public safety department struggles to cover all of the campuses numerous parking lots ef- Unmanned aerial vehicles (UAVs), also known as fectively and thus many people get away with illegally drones have many applications and they are a cur- parking, and worse unregistered vehicles are going un- rent trend across many industries. They can be used noticed due to the inefficiency of an officer having to for delivery, sports, surveillance, professional photog- manually check all decals. Beyond this, malicious in- raphy, cinematography, military combat, natural dis- dividuals can falsify a decal to make it appear as if aster assistance, security, and the list grows every day. they are indeed registered and to gain access to park- Programming opens an avenue to automate many pro- ing areas without proper authorization. In this paper, cesses of daily life and with the drone as aerial pro- we discuss how drone surveillance can be utilized for grammable eyes, security and surveillance can become automatic parking enforcement. Providing such a tool more efficient and cost effective. At Barry University, will not just speed up the process of enforcement but parking is becoming an issue as the number of people also allow for better use of public safety personnel. visiting the school greatly outnumbers the convenient Development of Automatic License Plate Recogni- parking locations. This has caused a multitude of haz- tion (ALPR) algorithms in the last decade or so, (see ards in parking lots due to people illegally parking, as [1, 3, 9, 10, 12]) has aided law enforcement on the road, well as unregistered vehicles parking in reserved ar- at campus security, at airports, casinos, etc. Univer- eas. In this paper, we explain how automated drone sity of Kansas is one of the places that uses ALPR for surveillance is utilized to detect unauthorized parking parking enforcement. Parking lots are monitored by a at Barry University. The automated process is incor- public safety person driving around the campus in a porated into Java application and completed in three vehicle that has ALPR camera mounted on the roof. steps: collecting visual data, processing data automat- The camera takes pictures of the cars, images are pro- ically, and sending automated responses and queues to cessed by ALPR, and tickets are issued if necessary. the operator of the system. PlateSmart (see [6], [8]), is developing software using Keywords: UAV, surveillance, image processing, ALPR for parking management among other things. ALPR, java application. The software works with almost any camera and pro- vides alerts of parking violations by email and text 1. INTRODUCTION messages. However, neither one of these examples and none that we found is employing drones for parking Technology continues to grow at an exponential enforcement. pace. Vehicle automation (both land and aerial) is The paper is organized as follows. Section 2 gives now a reality, as automated aircraft technology has an overview of different technology and services that shrunk down to a civilian level, and artificial intel- have been utilized. Section 3 talks about automation ligence is making leaps and bounds in multiple dis- of drone's flight path. Data obtained and results are ciplines. Research involving unmanned aerial vehicles presented in Section 4 while Section 5 presents diffi- (UAVs), also known as drones has vastly spread across culties encountered and future work. sciences both within academics and industries. From automatic fire forest monitoring [11], power line detec- 2. TECHNOLOGY tion [13], to road segment surveillance [5], the usage is expanding. We used DJI Phantom 3 Professional drone and At Barry University parking is a big issue as the utilized several existing technology/services in order amount of people visiting the school greatly outnum- to create a system that automatically returns license bers the available parking locations that one would plates of illegally parked cars. Technology/services consider convenient. This has caused a multitude of used are Litchi application, openALPR, FFmpeg and hazards in parking lots due to people illegally park- E-Verify Driving Records Search. ing, as well as unregistered vehicles parking in reserved Litchi, [4], is an Android/IOS application for DJI 153 Proceedings of The 21st World Multi-Conference on Systemics, Cybernetics and Informatics (WMSCI 2017) drones that includes numerous improvements over the 1420 ft 1 official DJI Drone application such as in-depth way- 6 point mission planning and object tracking. The 1670 ft 3180 ft 980 ft Litchi flight app allows us to easily program more 640 ft aspects of the flights because it can be run from a computer. The DJI app is only available on mobile 7 5 2070 ft 2600 ft 1580 ft devices. From a computer we can then collect all of 2580 ft the data we have gathered about a particular park- 1160 ft ing lot and build the routes on a large screen as well 2880 ft as program all the commands we need to perfectly 2580 ft capture data in a parking lot. Litchi works like an 4 2 extension of the DJI app as it still requires the DJI 660 ft app to pull its user interface, and commands. Litchi 1480 ft also facilitates the saving and naming of the various 3 waypoint missions we create that we are then able to load into the phone and fly through the app. Litchi Figure 1: Undirected weighted graph. Nodes are the stores these waypoint missions via our Litchi account. points of release of a drone. The red route represents OpenALPR, [7], is an open source Automatic Li- the shortest path. cense Plate Recognition library written in C++ with bindings in C#, Java, Node.js, Go, and Python. The library analyzes images and video streams to identify to a value larger than may increase accuracy, license plates. The output is the text representation but will increase processing time linearly (e.g., of any license plate characters. OpenALPR includes analysis count = 3 is 3x slower) support for a number of countries. The software in- cludes trained data for North American style and Eu- 6. Detection strength/strictness of license plate be- ropean style license plates allowing us to use the same fore signaling if license plate region exits. implementation in many states, countries, and conti- nents. It works by reading an image into a byte array FFmpeg, [2], is a cross-platform solution to record, in Java which it then sends to OpenALPR for pro- convert and stream audio and video. It is capable of cessing. The algorithm inside OpenALPR then tries splitting video output into standalone pictures. to get all possible coordinates for a license plate on an image, meaning there could be several different license 3. DRONE AUTOMATION plates on one image. Each one gets stored into an ob- ject with the calculated accuracy, coordinates for the The first step in successfully automating drones routes four edges of the license plate and also several guesses was to extensively map out all the parking regions of made by the algorithm what the license plate con- the school. Since it is legally required by Federal Avi- tains based on a post processed OCR reading. There ation Administration (FAA) that if the drone is de- are numerous fine tunings for the algorithm which we ployed, we have to stay within the line of sight of it, can manipulate in a config file such as: we designate a place (position), called a node, within each parking lot as a release point of the drone. It 1. Calibrating your camera improves detection ac- is a place that is easily accessible and covers one or curacy in cases where vehicle plates are captured possibly even two parking lots. Seven nodes were suffi- at a steep angle cient to cover all the parking lots at Barry University. 2. Detection will ignore plates that are too large. Walking distance between them represent weights of This is a good efficiency technique to use if the the undirected graph. Edges are then formed based plates are going to be a fixed distance away from of nodes' positions on the real map and their con- the camera nectivity. Starting from node 1, which is closest to public safety office, and returning to the same one, 3. The detection doesn't necessarily need an ex- we brute-force to find the shortest path. The shortest tremely high resolution image in order to detect path calculated is 1-2-3-4-7-6-5-1 for a total distance plates. Using a smaller input image should still of 10910 ft, see Figure 3. find the plates and will do it faster. The algo- rithm resizes the images to a fixed resolution. We Routes are created by using our previously mapped used 1280 by 720, and got good results. out parking lots and then programming waypoints set at GPS coordinates. The methodology for creating a 4. Whether we want to use masks. This can be ex- route is that the drone must effectively capture all the panded in the future. parking spaces as well as potentially illegal parking ar- 5. OpenALPR can scan the same image multiple eas while avoiding obstructions. We take into consid- times with different randomization. Setting this eration that cars may be parked either facing forward 154 Proceedings of The 21st World Multi-Conference on Systemics, Cybernetics and Informatics (WMSCI 2017) or in reverse.