QUADCOPTER MONITORING USING FACIAL RECOGNITION
Visvesva raya Technological University, Bel agavi.
PROJECT REPORT on “QUADCOPTER MONITORING USING FACIAL RECOGNITION ”
Project Report submitted in partial fulfillment of the requirement for theaward of the degree of Bachelor of Engineering in Electronics and Communication Engineering For the academic year 2019-20
Submitted by
USN Name 1CR16EC080 MEGHANA R 1CR16EC104 PALLAVI KN 1CR16EC120 RAJALAKSHMI R 1CR16EC133 RESHMA
Under the guidance of MRS. ASWINI.N Assistant professor Department of ECE CMRIT ,Bengaluru
Department of Electronics and Communication Engineering CMR Institute of Technology, Bengaluru – 560 037
QUADCOPTER MONITORING USING FACIAL RECOGNITION
DEPARTMENT OF ELECTRONICS AND COMMUNICATION ENGINEERING
CERTIFICATE This is to certify that the dissertation work “QUADCOPTER MONITORING USING FACIAL RECOGNITION”carried out by MEGHANA R, PALLAVI KN, R AJALAKSHMI R ,RESHMA USN: 1CR16EC080, 1CR16EC104, 1CR16EC120, 1CR16EC133 and bonafide student s of CMRIT in partial fulfillment for the award of Bachelor of Engineering in Electronics and Communication Engineering of the Visvesvaraya Technological University , Belagavi,during the academic year 2019- 20 . It is certified that all corrections /suggestions indicated for internal assessment have been incorporated in the report deposited in the departmental library. Theproject report has been approved as it satisfies the academic requirements in respect of Project work prescribed for the said degree.
Signature of Guide Signature of HOD Signature of Principal
______Mrs. Aswini.N, Dr. R. Elumalai Dr. Sanjay Jain Assistant professor, Head of the Department, Principal, Dept. of ECE., Dept. of ECE., CMRIT , CMRIT ,Bengaluru. CMRIT ,Bengaluru. Bengaluru .
External Viva Name of Examiners Signature & date 1. 2
QUADCOPTER MONITORING USING FACIAL RECOGNITION
ACKNOWLEDGEMENT
The satisfaction and euphoria that accompany the successful completion of any task would be incomplete without the mention of people who made it possible, whose consistent guidance and encouragement crowned our efforts with success.
Weconsider it asour privilege to express the gratitude to all those who guided in the completion of the project.
Weexpress my gratitude to Principal, Dr. Sanjay Jain, for having provided me the golden opportunity to undertake this project work in their esteemed organization.
Wesincerely thank Dr. R. Elumalai ,HOD , Department of Electronics and Communication Engineering, CMR Institute of Technology for the immense support given to me.
Weexpress my gratitude to our project guideMRS.ASWINI. N, Assistant professor, MRS.PAPPA.M, Associate professor & program coordinator,MR. MANOHARVALLETI,Assistant professor for their support, guidance and suggestions throughout the project work.
Last but not the least, hearty thanks ourparents and friends for their support.
Above all, Wethank the Lord Almighty for His grace on us to succeed in this endeavor.
Department of ECE, CMRIT, Bangalore 2019-20 3 QUADCOPTER MONITORING USING FACIAL RECOGNITION
Table of contents
ABSTRACT ...... 7
CHAPTER 1 ...... 8
INTRODUCTION ...... 18
CHAPTER 2 ...... 12
LITERATURE SURVEY ...... 12
CHAPTER 3 ...... 13
3.1 KK 2.1.5 MICROCONTROLLER ...... 13
3.2 BRUSHLESS DC MOTOR ...... 14
3.3 GYRO AND ACCELEROMETER SENSOR...... 14
3.4 ZIGBEE ...... 14
3.5 LIPO BATTERY ...... 15
3.6ELECTRONICAL SPEED CONTROLLER ...... 17
CHAPTER 4 ...... 188
SOFTWARE ...... 188
4.1 RASPBERRY PI MODULE...... 188
4.28 MP MONOCULAR CAMERA ...... 189
4.3 32 GB SD CARD ...... 189
4.4 3.5 TOUCH SCREEN DISPLAY ...... 21
4.5PYTHON 3.6 ...... 218
4.6 OPEN CV...... 218
4.7RASPBIAN OS ...... 218
4.8LPC2148 MICROCONTROLLER ...... 23
4.9 GPS LITE MODULE ...... 24
Department of ECE, CMRIT, Bangalore 2019-20 4 QUADCOPTER MONITORING USING FACIAL RECOGNITION
4.10 20X4 LCD MODULE...... 25
4.11 GSM800A SIM CARD ...... 26
4.12 FLAME SENSOR ...... 187
4.13 KEIL M4 ...... 188
4.14 FLASH MAGIC...... 188
CHAPTER 5 ...... 30
5.1QUADCOPTER ...... 30
5.2 IMAGEPROCESSING ...... 30
5.3 SENDING ALERT ...... 31
CHAPTER 6 ...... 33
RESULTS AND DISCUSSION6 ...... 33
CHAPTER 7 ...... 33
APPLICATION AND ADVANTAGES5 ...... 34
7.1APPLICATION5 ...... 34
7.2ADVANTAGES5 ...... 35
CHAPTER 8 ...... 377
CONCLUSION AND FUTURE SCOPE ...... 377
REFERENCES ...... 388
APPENDIX A ...... 399
Department of ECE, CMRIT, Bangalore 2019-20 5 QUADCOPTER MONITORING USING FACIAL RECOGNITION
LIST OF FIGURES
Figure no. Title Page no.
FIG 3.1 KK 2.1.5 MICROCONTROLLER 13
FIG 3.2.1 BRUSHLESS DC MOTOR 14
FIG 3.2.2 PROPELLERS 14
FIG 3.5.1 LIPO BATTERY 15
FIG 3.6.1 ELECTRONICAL SPEED CONTROLLER 16
FIG 4.1.1 RASPBERRY PI MODULE 18
FIG 4.2.1 8 MP MONOCULAR CAMERA 19
FIG 4.3.1 32 GB SD CARD 19
FIG 4.4.1 3.5 TOUCH SCREEN DISPLAY 20
FIG 4.8.1 LPC2148 MICROCONTROLLER 23
FIG 4.9.1 GPS LITE MODULE 25
FIG 4.10.1 20x4 LCD MODULE 26
FIG 4.11.1 GSM800A SIM CARD 27
FIG 4.12.1 FLAME SENSOR 28
FIG 4.13.1 KEIL M4 29
FIG 5.1 METHODOLOGY 30
FIG 5.1.1 DRONE 31
FIG 5.3.1 SENDING ALERT 32
FIG 6.1 PROTOTYPE 33
FIG 6.2 MEGHANA DETECTED 33
FIG 6.3 RAJ DETECTED 34
FIG 6.4 ALERT MESSAGE SENT 34
Department of ECE, CMRIT, Bangalore 2019-20 6 QUADCOPTER MONITORING USING FACIAL RECOGNITION
ABSTARCT
Maintaining surveillance is no easy job, neither is chasing criminals nor looking for kids or elderly people who are lost in crowded areas. This report aims to propose a viable solution for the aforementioned issue. The proposal encompasses the working of an Unmanned Aerial Vehicle (UAV) with an add-on of face recognition software. Unmanned Aerial Vehicles (UAVs) or drones are often used to reach remote areas or regions which are inaccessible to humans. Equipped with a large held of view, compact size, and remote-control abilities, drones are deemed suitable for monitoring crowded or disaster-hit areas and performing aerial surveillance. While research has focused on area monitoring, object detection, and tracking, limited attention has been given to person identification, especially face recognition, using drones. The main aim of this paper is to identify and locate missing persons and most wanted criminals by inputting the database. And when their faces are identified an alert and exact location of the person is sent. This helps in increasing surveillance. The application of this project can be extended to look humans in closed environments- such as exhibition halls, pattern tracking, and public safety surveillance. UAVs are the future and using them for such applications help reduce the human efforts, thereby making things easier and human error-free.
Department of ECE, CMRIT, Bangalore 2019-20 7 QUADCOPTER MONITORING USING FACIAL RECOGNITION
CHAPTER 1
INTRODUCTION
Emerging technologies have increased the demand of products having more features. Growth of population has raised the problem of security where surveillance is an effective measure. Increasing in demand of drones used for surveillance has given rise to the idea of face recognition drone. Where the drone recognizes the person in the crowd. The main objective of this software is to recognize and locate missing persons, children and most wanted criminal with the help of drone camera video input and report their location to the police station. Distinguishing facial features, detection and tracking face objects from video is a challenging task. Real time facial recognition is done by using raspberry pi (machine learning code). Face recognition software compares and identifies the scanned face with the inputs from the database and provides with all the inputted details including name. This database contains record of facial images of missing person and wanted criminals. If the both the faces have greater resemblance then the person location is sent to police station. An unmanned aerial vehicle (UAV) commonly known as a drone is an aircraft without a human pilot on board and a type of unmanned vehicle. UAVs are a component of an unmanned aircraft system (UAS); which include a UAV, a ground- based controller, and a system of communications between the two. The flight of UAVs may operate with various degrees of autonomy: either under remote control by a human operator or autonomously by onboard computers. Advantages of this system is that even though face recognition software can be installed in stationary cameras, drone provides an advantage of mobility and flying at low altitudes. Retrieval of data would be easier. Less time would be required for to maintain, update and delete records. So, surveillance become easier.
Department of ECE, CMRIT, Bangalore 2019-20 8 QUADCOPTER MONITORING USING FACIAL RECOGNITION
CHAPTER 2
LITERATURE SURVEY
Apurv Saha, Student Member IEEE, Akash Kumar, Aishwarya Kumar Sahuhas presented a brief idea about advancements in drones using Raspberry PI. A camera is attached with drone which helps PI to capture images and then PI can process it further to recognize person. The drone can be controlled from server room. The video can be seen live in the server room and simultaneously stored in the server. The main aim of the project is to recognize the face of person using drone camera and load all information about that person sitting in the server room. It is useful for biometric attendance, for military operations at remote areas and for surveillance purpose. Prototype is designed with high-tech specifications where it is made more stable and noise reduction feature is added. Depending upon the criteria more feature can be added.
Matthew C Fysh and Markus Bindemann has proposed the deployment of unmanned aerial vehicles (i.e., drones) in military and police operations implies that drones can provide footage that is of sufficient quality to enable the recognition of strategic targets, criminal suspects, and missing persons. On the contrary, evidence from Cognitive Psychology suggests that such identity judgements by humans are already difficult under ideal conditions, and are even more challenging with drone surveillance footage. In this review, we outline the psychological literature on person identification for readers who are interested in the real-world application of drones. We specifically focus on factors that are likely to affect identification performance from drone-recorded footage, such as image quality, and additional person-related information from the body and gait. Based on this work, we suggest that person identification from drones is likely to be very challenging indeed, and that performance in laboratory settings is still very likely to underestimate the difficulty of this task in real-world settings.
Isha Kalra and Maneet Singh has proposed Unmanned Aerial Vehicles (UAVs) or drones are often used to reach remote areas or regions which are inaccessible to humans. Equipped with a large field of view, compact size, and remote-control abilities, drones are deemed suitable for monitoring crowded or disaster-hit areas, and performing aerial
Department of ECE, CMRIT, Bangalore 2019-20 9 QUADCOPTER MONITORING USING FACIAL RECOGNITION surveillance. While research has focused on area monitoring, object detection and tracking, limited attention has been given to person identification, especially face recognition, using drones. This research presents a novel large-scale drone dataset, Drone SURF: Drone Surveillance of Faces, in order to facilitate research for face recognition. The dataset contains 200 videos of 58 subjects, captured across 411K frames, having over 786K face annotations. The proposed dataset demonstrates variations across two surveillance use cases: (i) active and (ii) passive, two locations, and two acquisition times. DroneSURF encapsulates challenges due to the effect of motion, variations in pose, illumination, background, altitude, and resolution, especially due to the large and varying distance between the drone and the subjects. This research presents a detailed description of the proposed DroneSURF dataset, along with information regarding the data distribution, protocols for evaluation, and baseline results.
From paper of Z. Zaheer, A. Usmani, E. Khan and M. A. Qadeer, it is seen that in today’s world, there is a growing need for surveillance in order to maintain the decorum at a place and ensure the safety and security of its people. An aerial surveillance system will be worthwhile in this regard. This paper describes how an aerial surveillance system can be built using an unmanned aerial vehicle or a drone. We start by delineating the features of our aerial surveillance system and then discuss some of the technologies that we have used in building it. After that we mention how we have incorporated those technologies into a drone and have made them work together harmoniously in order to achieve our desired aerial surveillance system. This system will be a convenient and efficient alternative to current surveillance systems. It can be used in peace keeping activities and also real time monitoring of a place at any time of the day. The aim is to provide fast and efficient surveillance at an affordable rate so that it can be used widely at private, institutional and governmental level. A. A. Shah, Z. A. Zaidi, B. S. Chowdhry and J. Daudpoto has implemented facial monitoring system by embedding face detection and face tracking algorithm found in MATLAB with the GPIO pins of Raspberry pi B by using RasPi command such that the array of LEDS follows the facial movement by detecting the face using Haar classifier, tracking its position in the range assigned using the eigenfeatures of the face, which are detected by eigenvectors of MATLAB and by face tracking, which is been carried by geometrical transformation so that motion and gesture of the face can be
Department of ECE, CMRIT, Bangalore 2019-20 10 QUADCOPTER MONITORING USING FACIAL RECOGNITION followed. By doing so we are opening up new way of facial tracking on a live streaming by the help of Viola Jones algorithm and an IR camera.
L. A. Elrefaei, A. Alharthi, H. Alamoudi, S. Almutairi and F. A1-rammah has proposed a criminal detection framework that could help policemen to recognize the face of a criminal or a suspect is proposed. The framework is a client-server video-based face recognition surveillance in the real-time. The framework applies face detection and tracking using Android mobile devices at the client side and video-based face recognition at the server side. This paper focuses on the development of the client side of the proposed framework, face detection and tracking using Android mobile devices. For the face detection stage, robust Viola-Jones algorithm that is not affected by illuminations is used. The face tracking stage is based on Optical Flow algorithm. Optical Flow is implemented in the proposed framework with two feature extraction methods, Fast Corner Features, and Regular Features. The proposed face detection and tracking is implemented using Android studio and OpenCV library, and tested using Sony Xperia Z2 Android 5.1 Lollipop Smartphone. Experiments show that face tracking using Optical Flow with Regular Features achieves a higher level of accuracy and efficiency than Optical Flow with Fast Corner Features. From J. Zhu and Z. Chen face detection is widely used in interactive user interfaces and plays a very important role in the field of computer vision. In order to build a fully automated system that can analyze the information in face image, there is a need for robust and efficient face detection algorithms. One of the fastest and most successful approaches in this field is to use Haar-like features for facial appearance and learning these features by AdaBoost algorithm. The key advantage of a Haar-like feature over most other features is its calculation speed. Due to the use of integral images, a Haar-like feature of any size can be calculated in constant time, which greatly accelerates the detection speed, while AdaBoost algorithm is a good way to select a good set of weak learners to construct a strong classifier. In this paper, a real time face detection system using framework of Adaboost and Haar-like feature is developed. In the end, the experiments show high performance in both accuracy and speed of the developed system.
From Hsu, H., Chen, K, Drones are also known as unmanned aerial vehicles (UAV) which are aircrafts which can perform autonomous pilot. They can easily reach locations which are too difficult to reach or dangerous for human beings and collect
Department of ECE, CMRIT, Bangalore 2019-20 11 QUADCOPTER MONITORING USING FACIAL RECOGNITION images from bird's-eye view through aerial photography. Enabling drones to identify people on the ground is important for a variety of applications, such as surveillance, people search, and remote monitoring. Since faces are part of inherent identities of people, how well face recognition technologies canbe used by drones becomes essential for future development of the above applications.
In this paper, we conduct empirical studies to evaluate several factors that may influence the performance of face detection and recognition techniques on drones. Our findings show that the current face recognition technologies are capable of recognizing faces on drones with some limits in distance and angle, especially when drones take pictures in high altitudes and the face image is taken from a long distance and with a large angle of depression. We also find that augmenting face models with 3D information may help to boost recognition performance in the case of large angles of depression.
From Satayanaryana, S, Niharika, D, Sravya, P and BhanuPriya, G, Because of increasing security concerns in real time biometric applications, face recognition is the suitable solutions, whenever we compare a greater number of solutions. Examine the latest advanced approaches to work and detect intrusion in face processing. Unwanted vehicle serving is the privacy issue in sensor-oriented frameworks for processing different technologies like video surveillance and unnamed vehicle areas in protecting technologies. Traditionally some types of intrusion and detection systems were proposed to do efficient security concerns for face authentication and detection in biometric applications. Lack of automatic track process in traditional systems, in this paper we propose and develop drone advanced primitives for autonomous communication. They can easily extraction location where the faces of the human available with similar features present in ground. By enabling drones, it is identified human in aerial photography of images for efficient people search and remote monitoring in face reorganization. Using SIFT (Scale Invariant Feature Transform) for efficient feature extraction in face detection based on drones in privacy technologies. By doing so, it removes the need for key point recognition on quality images and provides our strategy more effective to lighting changes than related techniques from the literature. Our experimental results show effective face reorganization results in real time security issues with feasible environment.
Department of ECE, CMRIT, Bangalore 2019-20 12 QUADCOPTER MONITORING USING FACIAL RECOGNITION
CHAPTER 3
HARDWARE
3.1 KK 2.1.5 MICROCONTROLLER
FIG 3.1 KK 2.1.5 MICROCONTROLLER
SPECIFICATIONS:
V Size: 50.5mm x 50.5mm x 12mm
V Weight: 21 grams (Inc. Piezo buzzer) V IC: Atmega644 PA V Gyro/Acc: 6050MPU V Auto-level: Yes V Input Voltage: 4.8-6.0V V AVR interface: standard 6 pin. V Signal from Receiver: 1520us (5 channels)
QUADCOPTER MONITORING USING FACIAL RECOGNITION
V Signal to ESC: 1520us V Firmware Version 1.6
3.2 BRUSHLESS DC MOTOR
FIG 3.2.1 BRUSHLESS DC MOTOR FIG 3.2.2 PROPELLERS
3.3 GYRO AND ACCELEROMETER SENSOR
Gyro sensors, also known as angular rate sensors or angular velocity sensors, are devices that sense angular velocity. Angular velocity. In simple terms, angular velocity is the change in rotational angle per unit of time. Angular velocity is generally expressed in deg/s (degrees per second).
An accelerometer is an electromechanical device used to measure acceleration forces. Such forces may be static, like the continuous force of gravity or, as is the case with many mobile devices, dynamic to sense movement or vibrations. Acceleration is the measurement of the change in velocity, or speed divided by time.
3.4 ZIGBEE
Zigbee is an IEEE 802.15.4-based specification for a suite of high-level communication protocols used to create personal area networks with small, low-power digital radios, such as for home automation, medical device data collection, and other low-power low- bandwidth needs, designed for small scale projects which need wireless connection.
QUADCOPTER MONITORING USING FACIAL RECOGNITION
Hence, Zigbee is a low-power, low data rate, and close proximity (i.e., personal area) wireless ad hoc network. The technology defined by the Zigbee specification is intended to be simpler and less expensive than other wireless personal area networks (WPANs), such as Bluetooth or more general wireless networking such as Wi-Fi. Applications include wireless light switches, home energy monitors, traffic management systems, and other consumer and industrial equipment that requires short-range low-rate wireless data transfer. Its low power consumption limits transmission distances to 10–100 meters line- of-sight, depending on power output and environmental characteristics. Zigbee devices can transmit data over long distances by passing data through a mesh network of intermediate devices to reach more distant ones. Zigbee is typically used in low data rate applications that require long battery life and secure networking (Zigbee networks are secured by 128 bit symmetric encryption keys.) Zigbee has a defined rate of 250 Kbit/s, best suited for intermittent data transmissions from a sensor or input device.
3.5 LIPO BATTERY
FIG 3.5.1 LIPO BATTERY
A lithium polymer battery, or more correctly lithium-ion polymer battery (abbreviated as LiPo, LIP, Li-poly, lithium-poly and others), is a rechargeable battery of lithium-ion technology using a polymerelectrolyte instead of a liquid electrolyte. High conductivity semisolid (gel) polymers form this electrolyte. These batteries provide higher specific energy than other lithium battery types and are used in applications where weight is a critical feature, like mobile devices and radio-controlled aircraft.
3.6 ELECTRONICAL SPEED CONTROLLER
QUADCOPTER MONITORING USING FACIAL RECOGNITION
FIG 3.6.1 ELECTRONICAL SPEED CONTROLLER
An electronic speed control follows a speed reference signal (derived from a throttle lever, joystick, or other manual input) and varies the switching rate of a network of field effect transistors (FETs) . By adjusting the duty cycle or switching frequency of the transistors, the speed of the motor is changed. The rapid switching of the transistors is what causes the motor itself to emit its characteristic high -pitched whine, especially noticeable at lower speeds.
Different types of speed controls are required for brushed DC motors and brushless DC motors . A brushed motor can have its speed controlled by varying the voltage on its armature. (Industrially , motors with electromagnet field windings instead of permanent magnets can also have their speed controlled by adjusting the strength of the motor field current.) A brushless motor requires a different operating principle. The speed of the motor is varied by adjusting the timing of pulses of current delivered to the several windings of the motor.
Brushless ESC systems basically create three-phase AC powe r, like a VFD variable frequency drive, to run brushless motors . Brushless m otors are popular with radio controlled airplane hobbyists because of their efficiency, power, longevity and light
QUADCOPTER MONITORING USING FACIAL RECOGNITION weight in comparison to traditional brushed motors. Brushless DC motor controllers are much more complicated than brushed motor controllers.
The correct phase varies with the motor rotation, which is to be taken into account by the ESC: Usually, back EMF from the motor is used to detect this rotation, but variations exist that use magnetic (Hall effect) or optical detectors. Computer-programmable speed controls generally have user-specified options which allow setting low voltage cut-off limits, timing, acceleration, braking and direction of rotation. Reversing the motor's direction may also be accomplished by switching any two of the three leads from the ESC to the motor.
Department of ECE, CMRIT, Bangalore 2019-20 17 QUADCOPTER MONITORING USING FACIAL RECOGNITION
CHAPTER 4
SOFTWARE
4.1 RASPBERRY PI MODULE
FIG 4.1.1 RASPBERRY PI MODULE
The Raspberry Piis a series of small single-board computers developed in the United Kingdom by the Raspberry Pi Foundation to promote teaching of basic computer science in schools and in developing countries. The original model became far more popular than anticipated, selling outside its target market for uses such as robotics. It now is widely used even in research projects, such as for weather monitoring because of its low cost and portability. It does not include peripherals (such as keyboards and mice) or cases. However, some accessories have been included in several official and unofficial bundles.
After the release of the second board type, the Raspberry Pi Foundation set up a new entity, named Raspberry Pi Trading, and installed Eben Upton as CEO, with the responsibility of developing technology. The Foundation was rededicated as an educational charity for promoting the teaching of basic computer science in schools and developing countries.
QUADCOPTER MONITORING USING FACIAL RECOGNITION
The Raspberry Pi is one of the best-selling British computers. As of December 2019, more than thirty million boards have been sold. Most Pi’s are made in a Sony factory in Pencoed, Wales, while others are made in China and Japan.
4.2 8 MP MONOCULAR CAMERA
FIG 4.2.1 8 MP MONOCULAR CAMERA
The Raspberry Pi Camera v2 is a high quality 8-megapixel Sony IMX219 image sensor custom designed add-on board for Raspberry Pi, featuring a fixed focus lens. In terms of still images, the camera is capable of 3280 x 2464-pixel static images, and also supports 1080p30, 720p60 and 640x480p90 video.
4.3 32 GB SD CARD
FIG 4.3.1 32 GB SD CARD
QUADCOPTER MONITORING USING FACIAL RECOGNITION
A memory card or memory cartridge is an electronic data storage device used for storing digital information, typically using flash memory. These are commonly used in portable electronic devices, such as digital cameras, mobile phones, computers, tablets, PDAs, portable media players, video game consoles, synthesizers, electronic keyboards and digital pianos.
4.4 3.5 TOUCH SCREEN DISPLAY
FIG 4.4.1 3.5 TOUCH SCREEN DISPLAY
Atouch screen displayis a both input and output device and normally layered on the top of an electronic visual display of an information processing system. A user can give input or control the information processing system through simple or multi-touch gestures by touching the screen with a special stylus or one or more fingers. Some touchscreens use ordinary or specially coated gloves to work while others may only work using a special stylus or pen. The user can use the touchscreen to react to what is displayed and, if the software allows, to control how it is displayed; for example, zooming to increase the text size.
The touchscreen enables the user to interact directly with what is displayed, rather than using a mouse, touchpad, or other such devices (other than a stylus, which is optional for most modern touchscreens).
Touchscreens are common in devices such as game consoles, personal computers, electronic voting machines, and point-of-sale (POS) systems. They can also be attached to
QUADCOPTER MONITORING USING FACIAL RECOGNITION computers or, as terminals, to networks. They play a prominent role in the design of digital appliances such as personal digital assistants (PDAs) and some e-readers.
4.5 PYTHON 3.6
Python is a general purpose and high-level programming language. You can use Python for developing desktop GUI applications, websites and web applications. Also, Python, as a high-level programming language, allows you to focus on core functionality of the application by taking care of common programming tasks.
4.6 OPEN CV
OpenCV-Python is a library of Python bindings designed to solve computer vision problems. OpenCV-Python makes use of NumPy, which is a highly optimized library for numerical operations with a MATLAB-style syntax. All the OpenCV array structures are converted to and from NumPy arrays.
OpenCV is used for all sorts of image and video analysis, like facial recognition and detection, license plate reading, photo editing, advanced robotic vision, optical character recognition, and a whole lot more.
4.7Raspbian OS
Raspbian is a free operating system based on Debian optimized for the Raspberry Pi hardware. An operating system is the set of basic programs and utilities that make your Raspberry Pi run.
Raspberry Pi OS is a free operating system based on Debian, optimised for the Raspberry Pi hardware. Raspberry Pi OS comes with over 35,000 packages: precompiled software bundled in a nice format for easy installation on your Raspberry Pi.
4.8 LPC2148 MICROCONTROLLER
The LPC2148 microcontroller is designed by Philips (NXP Semiconductor) with several in-built features & peripherals. Due to these reasons, it will make more reliable as well as
Department of ECE, CMRIT, Bangalore 2019-20 21 QUADCOPTER MONITORING USING FACIAL RECOGNITION the efficient option for an application developer. LPC2148 is a 16-bit or 32-bit microcontroller based on ARM7 family.
The main features of LPC2148 include the following.
V The LPC2148 is a 16 bit or 32-bit ARM7 family-based microcontroller and available in a small LQFP64 package. V ISP (in system programming) or IAP (in application programming) using on-chip boot loader software. V On-chip static RAM is 8 kB-40 kB, on-chip flash memory is 32 kB-512 kB, the wide interface is 128 bits, or accelerator allows 60 MHz high-speed operation. V It takes 400 milliseconds time for erasing the data in full chip and 1 millisecond time for 256 bytes of programming. V Embedded Trace interfaces and Embedded ICE RT offers real-time debugging with high-speed tracing of instruction execution and on-chip Real Monitor software. V It has 2 kB of endpoint RAM and USB 2.0 full speed device controller. Furthermore, this microcontroller offers 8kB on-chip RAM nearby to USB with DMA. V One or two 10-bit ADCs offer 6 or 14 analog inputs with low conversion time as 2.44 µs/ channel. V Only 10-bitDAC offers changeable analog output. V External event counter/32-bit timers-2, PWM unit, & watchdog. V Low power RTC (real time clock) & 32 kHz clock input. V Several serial interfaces like two 16C550 UARTs, two I2C-buses with 400 Kbit/s speed. V 5 volts tolerant quick general-purpose Input/output pins in a small LQFP64 package. V Outside interrupt pins-21. V 60 MHz of utmost CPU CLK-clock obtainable from the programmable-on-chip phase locked loop by resolving time is 100 µs. V The incorporated oscillator on the chip will work by an exterior crystal that ranges from 1 MHz-25 MHz V The modes for power-conserving mainly comprise idle & power down.
Department of ECE, CMRIT, Bangalore 2019-20 22 QUADCOPTER MONITORING USING FACIAL RECOGNITION
V For extra power optimization, there are individual enable or disable of peripheral functions and peripheral CLK scaling.
FIG 4.8.1 LPC2148 MICROCONTROLLER
4.9 GPS LITE MODULE
A GPS tracking unit is a navigation device normally carried by a moving vehicle or person or animal that uses the Global Positioning System (GPS) to track the device's movements and determine its location. The recorded location data can either be stored within the tracking unit or transmitted to an Internet-connected device using the cellular (GPRS or SMS), radio, or satellite modem embedded in the unit. This allows the location to be displayed against a map backdrop either in real time or when analysing the track later, using GPS tracking software. Data tracking software is available for smartphones with GPS capability.
A GPS "track me" essentially contains a GPS module that receives the GPS signal and calculates the coordinates. For data loggers, it contains large memory to store the coordinates. Data pushers additionally contain a GSM/GPRS/CDMA/LTE modem to transmit this information to a central computer either via SMS or GPRS in form of
QUADCOPTER MONITORING USING FACIAL RECOGNITION
IPpackets. Satellite-based GPS tracking units will operate anywhere on the globe using satellite technology such as Global Star or Iridium. They do not require a cellular connection.
FIG 4.9.1 GPS LITE MODULE
4.10 20x4 LCD MODULE
A liquid-crystal display (LCD) is a flat panel display, electronic visual display, or video display that uses the light modulating properties of liquid crystals. ... 20x4 means that 20 characters can be displayed in each of the 4 rows of the 20x4 LCD, thus a total of 80 characters can be displayed at any instance of time.
LCD accepts two types of signals, one is data, and another is control. These signals are recognized by the LCD module from status of the RS pin. Now data can be read also from the LCD display, by pulling the R/W pin high. As soon as the E pin is pulsed, LCD display reads data at the falling edge of the pulse and executes it, same for the case of transmission.
QUADCOPTER MONITORING USING FACIAL RECOGNITION
FIG 4.10.1 20x4 LCD MODULE
4.11 GSM800A SIM CARD
The SIM800A Quad-Band GSM/GPRS Module with RS232 Interface is a complete Quad-band GSM/GPRS solution in an LGA (Land grid array) type which can be embedded in the customer applications. SIM800A support Quad-band 850/900/1800/1900 MHz, it can transmit Voice, SMS and data information with low power consumption.
With a tiny size of 100 x 53 x 15 mm, it can fit into slim and compact demands of custom design. Featuring and Embedded AT, it allows total cost savings and fast time-to-market for customer applications.
The SIM800A modem has a SIM800A GSM chip and RS232 interface while enables easy connection with the computer or laptop using the USB to the Serial connector or to the micro-controller using the RS232 to TTL converter. Once you connect the SIM800A modem using the USB to RS232 connector, you need to find the correct COM port from the Device Manager of the USB to Serial Adapter.
QUADCOPTER MONITORING USING FACIAL RECOGNITION
FIG 4.11.1GSM800A SIM CARD
4.12 FLAME SENSOR
A sensor which is most sensitive to a normal light is known as a flame sensor. That’s why this sensor module is used in flame alarms. This sensor detects flame otherwise wavelength within the range of 760 nm – 1100 nm from the light source. This sensor can be easily damaged to high temperature. So, this sensor can be placed at a certain distance from the flame. The flame detection can be done from a 100cm distance and the detection angle will be 600. The output of this sensor is an analog signal or digital signal. These sensors are used in firefighting robots like as a flame alarm. A flame-sensor is one kind of detector which is mainly designed for detecting as well as responding to the occurrence of a fire or flame. The flame detection response can depend on its fitting. It includes an alarm system, a natural gas line, propane & a fire suppression system. This sensor is used in industrial boilers. The main function of this is to give authentication whether the boiler is properly working or not. The response of these sensors is faster as well as more accurate compare with a heat/smoke detector because of its mechanism while detecting the flame.
QUADCOPTER MONITORING USING FACIAL RECOGNITION
FIG 4.12.1 FLAME SENSOR
4.13 KEIL M4
The µVision IDE combines project management, run-time environment, build facilities, source code editing, and program debugging in a single powerful environment. µVision is easy-to-use and accelerates your embedded software development. µVision supports multiple screens and allows you to create individual window layouts anywhere on the visual surface.
The µVision Debugger provides a single environment in which you may test, verify, and optimize your application code. The debugger includes traditional features like simple and complex breakpoints, watch windows, and execution control and provides full visibility to device peripherals.
With the µVision Project Manager and Run-Time Environment you create software application using pre-build software components and device support from Software Packs. The software components contain libraries, source modules, configuration files, source code templates, and documentation. Software components can be generic to support a wide range of devices and applications.
QUADCOPTER MONITORING USING FACIAL RECOGNITION
FIG 4.13.1 KEIL M4
4.14 FLASH MAGIC
Flash Magic is a PC tool for programming flash based microcontrollers from NXP using a serial or Ethernet protocol while in the target hardware.
QUADCOPTER MONITORING USING FACIAL RECOGNITION
CHAPTER 5
WORKING
The proposed system methodology is flying drone and recognising the wanted persons face which is already stored in the database and send an alert message to the registered phone number.
FIG 5.1 METHODOLOGY
We can classify the working to three parts Quadc opter, Image processing and sending alert.
5.1 QUADCOPTER
V KK2.1.5 Flight controller purpose is to stabilize the aircraft during flight. V To do this it takes the signal from the 6050MPU gyro/acc (roll, pitch, and yaw) then passes the signal to the Atmega644PA I C. V The Atmega644PA IC unit then processes these signals according to the user’s selected firmware (quadcopter) and passes control signals to the installed Electronic Speed Controllers (ESCs). V These signals instruct the ESCs to make fine adjustments to the DC motor’s rotational speed which in turn stabilizes your multi -rotor craft.
QUADCOPTER MONITORING USING FACIAL RECOGNITION
V KK2.1.5 Multi-Rotor control board also uses signals from your radio systems receiver (Rx)
V These signals to the Atmega644PA IC via the aileron, elevator, throttle and rudder inputs. V Once this information has been processed the IC will send varying signals to the ESCs which in turn adjust the rotational speed of each motor to induce controlled flight (up, down, backward, forward, left, right, yaw).
FIG 5.1.1 DRONE
5.2IMAGE PROCESSING:
V Raspbian OS is stored using 32GB SD card. V Here wanted person faces are stored as an input with their record. V Drone with camera is made to fly and real time images are captured and sent to raspberry pi module. V For the input image, we apply face detection to detect the location of a face in the image. V Then when face is identified we pass it to our deep neural network. V The Face Net deep learning takes an input image and returns a 128-d embedded vector that is the image representation in a Euclidean space of 128 dimensions. V The neural network computes the 128-d embeddings for each face and triplet loss function is done.
QUADCOPTER MONITORING USING FACIAL RECOGNITION
V Anchors and positive are assumed to be closer and pushes negative image farther and face recognition is done.
5.3 SENDING ALERT :
V The whole operation is controlled by the LPC2148 microprocessor. V GPS is used to track the quadcopter. It finds the location when at least 4 GPS satellites in line of sight to a receiver on the earth. It uses GPGGA format to fetch data. V When the face is recognized, GPS provides the latitude and longitude information of the vehicle. V GSM/GPRS module is used for establishing communication link between a computer and a GSM-GPRS system. V GSM modem is used to send the SMS to the registered phone numbers. V In addition, we are using flame sensor,if there is fire in that place it also sends same kind of alert to registered numbers
FIG 5.3.1 SENDING ALERT
QUADCOPTER MONITORING USING FACIAL RECOGNITION
CHAPTER6
RESULTS AND DISCUSSION
The model below Fig 5.1 is the final stage Quadcopter having specific features as described.
FIG 6.1 PROTOTYPE
V In the drone face recognizing feature is added. And the software is working in sync with the UAV. V In real time using Raspberry pi cameratwo persons (Meghana and Raj) faces are captured. V About 30 images of their faces where captured and stored in the database with their names. V Now we consider the wanted person to find is Raj. V Drone was made to fly in specific area to find the wanted person. V So, when Meghana’s face was captured by the camera it recognized the face. But it didn’t send any alert messages.
QUADCOPTER MONITORING USING FACIAL RECOGNITION
FIG6.2 MEGHANA DETECTED
• Same way when Raj’s face was captured by the camera it recognized the face
FIG6.3 RAJ DETECTED
• And alert message is sent to the registered mobile number with the latitude and longitude information.
FIG 6.4 ALERT MESSAGE SENT
QUADCOPTER MONITORING USING FACIAL RECOGNITION
CHAPTER 7 APPLICATION AND ADVANTAGES
7.1 APPLICATION
V MILITARY AND LAW ENFORCEMENT : Quadcopter unmanned aerial vehicles are used for surveillance and reconnaissanceby military and law enforcement agencies, as well as search and rescue missions in urban environments.One such example is the aeryon scout created by Canadian company aeryon labs, which is a small UAV that can quietly hover in place and use a camera to observe people and objects on the ground . V PHOTOGRAPHY: The largest use of quadcopters in the USA has been in the field of aerial imagery. Quadcopter UAVs are suitable for this job because of their autonomous natureandhuge cost savings. Drones have also been used for light-painting photography. V DRONE DELIVERY: A delivery drones an autonomous vehicle often an unmanned aerial vehicle ( UAV), Used to transport packages, food or other goodsUAVs can transport medicines and vaccines, and retrieve medical samples, into and Out of remote or otherwise inaccessible regions. V HUMANITARIAN OPERATIONS: Quadcopters are being used for a wide variety of humanitarian applications fromDisaster relief to animal conservation. V ART: Quadcopters have also been used in various art projects including but not limited toDrone photography. At least one drone has demonstrated the feasibility of paintingGraffiti on a wall with spray paint. They may be used in performance art with newDegreesof positional control that allows for new uses of puppets, characters, lights and cameras. V SPORT: Quadcopters are used all over the world for racing (also known as “down racing”) and freestyle events. Racing and freestyle quadcopters are built for aped andagility.Racing and freestyle drones tend to be relatively small in size,with 250mm betweenthepropeller shafts and/or 5-6-inch props being the usually upper end of the size scale.
Department of ECE, CMRIT, Bangalore 2019-20 34 QUADCOPTER MONITORING USING FACIAL RECOGNITION
7.2 ADVANTAGES The main merit of quadcopters and similar unmanned aerial vehicles is their small size, due to which they could traverse in narrow conditions.
The use drones have tremendously grown in a short span of time owing to the long flying time in contrast to the manned aircrafts. A human pilot, drones, can operate for significantly longer without fatigue than airplanes.Moreover,drone operators van easily hand off controls of a drone without any operational downtime. They are remote controlled,so no danger will be there to the crew.
They contain a whole lot of widespread applications,in day to day lives, domestic purposes and national to international purposes.
V It has successfully struck militants of AI Qaeda and other terrorist groups.
V Helps detect forest fires.
V Monitor environmental data (i.e., that population of animals.)
V Can seek missing children and felons.
V It is also suitable for indoor applications.
V Drones are very popular because mass media networks patronize it’s functionally and efficiency when capturing videos and images.
V Four small rotors have smaller diameter than once large helicopter rotor.
V Take less damage to rotors.
V No need for a tail rotor which generates no lift.
V Easier to build four small blades compared to large ones.
V Does not require mechanical linkages to change the pitch angle at the blades as it spins.
V Due to ease in construction and control, they are used in amateur model aircrafts project.
V They can traverse through difficult terrains because of their small size and there is less risk of damage too.
Department of ECE, CMRIT, Bangalore 2019-20 35 QUADCOPTER MONITORING USING FACIAL RECOGNITION
V They can save lives. They greatly putting military manpower in combat(in harm). They are significantly cheaper and cost in fuel and maintenance is way lower than regular airplanes.
V Quadcopters are smaller and are able to fly lower than traditional airplanes and the risk level to military hardware is comparatively very low.
V Drones increase surveillance, reconnaissance, and general military intelligence.
V Drones contain more pinpoint precision and accuracy from larger distances, which in turn reduce the collateral damage to civilians and infrastructure.
V Drones are easier and faster to deploy than most alternatives.
.
Department of ECE, CMRIT, Bangalore 2019-20 36 QUADCOPTER MONITORING USING FACIAL RECOGNITION
CHAPTER 8
CONCLUSION AND FUTURE SCOPE
Drones will soon take on be an imperative existence in the coming future. They will be seen taking up larger notes for a variety of jobs including business in the immediate future. They could become a part of our daily lives, from smallest retails like delivering groceries to changing the way farmers manage their crops to revolutionizing private security, or maybe even aerial advertising. Today,quadcopters are capturing news video,recording vacation travel logs, filming movies,providing disaster relief, surveying real estate and delivering packages. In innumerable advantages of drones lead to their growth in a short span time. They have a few demerits but those can be rectified. Today most drones are controlled by either software or other computer programs. The components of a drone also vary based on what type of work needs to be done and how much payload needs to be carried.
Face Recognition incorporated with drones creates a valuable contribution in security arena. It is useful for police operations in secret missions. The drone will be able to detect people going through places where human reach is not possible. It can also be used for attendance purpose by just flying the drone throughout the room. Further this can be made more accurate and used in our daily life.
Department of ECE, CMRIT, Bangalore 2019-20 37 QUADCOPTER MONITORING USING FACIAL RECOGNITION
References
[1] Apurv Saha, Akash Kumar, Aishwary Kumar Sahu, "Face Recognition Drone”, 2018 3rd International Conference for Convergence in Technology (I2CT) IEEE. [2] Ankush Pandita, Rushikesh Dahenkar, Jyotiram Dalve, Shubham Kumar, Shubham Kumar, “Missing Person Identification and Tracking for Intelligent Video Surveillance” International Journal of Innovative Research in Computer and Communication Engineering (ISSN), March 2017. [3] Isha Kalra ; Maneet Singh ; Shruti Nagpal ; Richa Singh ; Mayank Vatsa; P. B. Sujit ,” DroneSURF: Benchmark Dataset for Drone-based Face Recognition” 4International Conference on Advanced Machine Learning Technologies and Applications . [4] Z. Zaheer, A. Usmani, E. Khan and M. A. Qadeer, "Aerial surveillance system using UAV", 2016 Thirteenth International Conference on Wireless and Optical Communications Networks (WOCN ), 2016. [5] A. A. Shah, Z. A. Zaidi, B. S. Chowdhry and J. Daudpoto, "Real time face detection/monitor using raspberry pi and MATLAB", 2016 IEEE 10th International Conference on Application of Informationand Communication Technologies (AICT) , 2016. [6] L. A. Elrefaei, A. Alharthi, H. Alamoudi, S. Almutairi and F. A1-rammah, "Real-time face detection and tracking on mobile phones for criminal detection", 2017 2nd International Conference on Anti- Cyber Crimes (ICACC), 2017. [7] J. Zhu and Z. Chen, "Real Time Face Detection System Using Adaboost and Haar-like Features", 2015 2nd International Conference on Information Science and Control Engineering, 2015. [8] Hsu, H., Chen, K.: Face recognition on drones: issues and limitations. In: Proceedings of the first workshop on micro aerial vehicle networks, systems, and applications for civilian use, pp. 39–44 (2017) [9] Satayanaryana, S., Niharika, D., Sravya, P., BhanuPriya, G.: Advanced face detection to trend analysis in image processing based on drone primitives. Int. J. Pure Appl. Math. 115 (6), 31–36 (2017)
Department of ECE, CMRIT, Bangalore 2019-20 38 QUADCOPTER MONITORING USING FACIAL RECOGNITION
Appendix A Table of contents List of figures
Department of ECE, CMRIT, Bangalore 2019-20 39