Intelligent Virtual Assistant for Visually Impaired People Using Raspberry Pi S

Home , Virtual assistant

Research Article Volume 10 Issue No.3 Intelligent Virtual Assistant for Visually Impaired People using Raspberry Pi S. Sivakumar1, T. M. Pavithra2, K. Pooja Shree3, S. Reshma4 Assistant Professor1, Student2, 3, 4 Department of ECE Jeppiaar SRR Engineering College, Chennai, India

Abstract: An OCR (Optical Character Recognition) system which may be a branch of computer vision and successively a sub-class of AI. Optical character recognition is that the translation of optically scanned bitmaps of printed or hand transcription into audio output by using of Raspberry Pi. OCR's are developed for many world languages are already under efficient use. This method extracts moving object region by a mixture-of-Gaussian based-background subtraction method. A text localization and recognition are conducted to accumulate text information. To automatically localize the text regions from the image, a text localization and Tesseract algorithm by learning gradient features of stroke orientations and distributions of edge pixels in an Ada boost model. Text characters in the localized text regions are then converted into binary codes and recognized by optical character recognition software. The recognized text codes are given as output to blind users. Performance of the proposed text localization algorithm. As the recognition process is completed, the character codes within the document are processed using Raspberry pi device on which recognize character using Tesseract algorithm and python programming, the audio output is listened. This model is mounted on a shopping trolley to help visually challenged people to shop individually.

Keywords: Capturing Image, Distance measuring, OCR Software, Tesseract Algorithm, Final Audio output.

I. INTRODUCTION document that was printed on paper are time-consuming and labour-intensive. It is widely wont to convert books and Visually challenged people face many problems with documents into electronic files to be used in storage and accessing printed text, which includes problems with document analysis. OCR makes it is possible to use techniques alignment, focus, accuracy, mobility, and efficiency. We like MT, text-to-speech and text mining to the capture/ present a smart device that assists the visually impaired which scanned page. The final recognized text document is fed to the effectively and efficiently reads the paper-printed text. The output devices, depending on the choice of the user. The proposed project uses the methodology of a camera based output device are often a headset connected to the raspberry pi assistive device which will be employed by people to read or a speaker which may spell out the text document loud. Text document. The main idea of this project is for implementing image capturing technique in an embedded II. LITERATURE SURVEY system based on Raspberry Pi board. In this project, we proposed a text read out a system for the visually challenged. 1. Mrs. Shilpa Reddy K, Mounika S. k, Pooja K, Sahana N. The proposed system uses a camera image as input and the Text to Speech for the Visually Impaired”, IRJCS Journal, image is processed using OCR software module and the output 2017. is given in audio format. A methodology is implemented to recognition sequence of characters, and the line of reading. As Abstract a part of the software development, the Open CV (Open source Computer Vision) libraries are utilized to do image Perusing is prime in day by day life for everyone. Outwardly capture of text, to do the character recognition. Most of the debilitated people can read just by utilizing unique access technology tools built for people with blindness and applications by them like Braille dialect. The disadvantage of limited vision is built on the two basic building blocks of OCR this framework is that every item doesn't give the content in software and Text-to-Speech (TTS) engines. optical character Braille. In this paper, they have proposed an assistive content recognition (OCR) is the translation of captured images of perusing system to help outwardly impaired people to peruse printed text into binary codes. OCR may be a process which writings from different questions in their day by day lives. At associates a symbolic meaning with objects(letters, symbols a first, we catch the image of the specified, pre-handling is number) with the image of a character. It is defined because performed thereon. Pre-handling incorporates steps like dark the process of converting scanned images of machine printed scale and binarization, question of intrigue acknowledgment. into a computer process able format. An optical Character In the proposed framework, we are making the utilization of recognition is additionally useful for visually impaired people OTSU calculation to change over the dim scale picture into that cannot read Text document, but got to access the content binarized one. The content districts from the caught picture of the Text documents. An optical Character recognition is are then separated and perceived by utilizing optical used to digitize and reproduce texts that have been produced character acknowledgment programming (OCR).The principle with the non-computerized system. Digitizing texts also helps calculation in OCR to be specific MODI is utilized here. This reduce storage space. Editing and Reprinting of a Text extricated content of different textual styles and sizes then can

IJESC, March 2020 24843 http:// ijesc.org/ be perceived independently and afterward consolidated in a words. By the utilization of varied Mat lab commands i.e. word giving its yield as sound utilizing Text-to-discourse wavered, subplot etc. the waves are sampled and extracted to utilizing the SAPI libraries. get the desired result. This method is straightforward to implement and involves much lesser use of memory spaces. 2. MallapaD.Gurav, Shruti S. Salimath, Shruti B. Hatti, Vijayalaxmi I. Byakod, ShivaleelaKanade “A Reading aid Disadvantages: for the Blind People using OCR and OpenCV”, IJSRET Journal 2017. Braille system is very slow and not very practical.

Existing OCR systems are not automatic. Abstract Require full-fledged computers to run effective. Optical character recognition (OCR) is used to detect printed or typed characters using photoelectric devices and computer Cell phone does not allows the user to read mail, receipts, files software. It converts images of typed or printed text into machine encoded text from scanned document. In this research IV. PROPOSED SYSTEM these printed text are converted into audio output. OCR is employed in machine process like cognitive computing, text to In this system we've described a prototype system to read speech, key data and text mining. It is mainly utilized in the printed or typed texts on hand-held objects for guiding blind sector of research in Character recognition, AI and computer persons. In order to unravel the common aiming problem for vision. The recognition process is completed using OCR the blind users, we've proposed a motion-based method to detect character code in text files are processed using Raspberry Pi the thing of interest. The automatic ROI detection and text device on which it recognizes character using tesseract localization algorithms were independently evaluated to make algorithm and python programming and audio output is sure effectiveness and robustness of the entire system. We listened. For pattern recognition and to perform Document subsequently evaluated this prototype system of assistive text Image Analysis (DIA) we use information in grid format in reading using images of hand-held objects captured by ten virtual digital library’s design and construction. This research blind users face to face. Two calibrations were applied to mainly deals on the OCR based automatic book reader for the organize for the system test. First, we instructed blind users to visually impaired using raspberry pi. Raspberry pi features a put hand-held object within the camera view. Since it's Broadcom System On a Chip (SOC) which incorporates ARM difficult for blind users to aim their held objects, we employed compatible CPU and an on chip graphics processing unit a camera with a fairly wide angle. Secondly, in an applicable GPU. It promotes Python programming as main programming blind-assistive system, a text localization algorithm might language. prefer higher recall by sacrificing some precision. By using ultrasonic sensor, we can measure the distance between the 3. Miss. Kirti P. Bhure, Mrs. J. D. Dhande” Object blind people and obstacle then the distance will be played Detection Methodologies for Blind People” IJRIT Journal through earphones. OCR is used for converting images of 2017. typed, handwritten or printed text into machine-encoded text, whether form a scanned document, a photo of a document. It Abstract is widely used to produce the information entry from printed paper. It is a method of analyzing printed texts so that they can Vision is the most important sense. Image plays vital role be easily edited, searched, stored, displayed online and utilized within the human perception of the encompassing in machine process like cognitive computing, machine environment. Industry has created a spread of computer vision translation and text-to-text.This module can also be mounted products and services by developing new electronic on a shopping trolley which can help the visually challenged technologies for the blind so as to beat the difficulties. Digital people shop individually on super markets. Four buttons is image processing is that the field which processes the digital added along with this model. These four buttons are assigned image by using computer. An increasing interest in developing with four basic food products and a DC motor and a driver is technologies attempts to assist visually impaired people in fixed at the wheel of the shopping trolley. The buttons, DC their daily lives. It is shown that the thing identification is that motor and driver is connected to Raspberry Pi's General the difficult task for visually impaired people. Although there Purpose Input Output (GPIO) pins and it is coded accordingly. are many applications which will be used for this task, there If the buttons is pressed. It moves according to the program are still limitations that need more improving. For this reason, coded. The UV sensor calculates the distance between this paper provides the survey and an analysis of varied the object of interest and the camera. The camera captures the evaluations for the technologies that utilized in the thing image of the product and the output can be heard through the identification task. For the visually impaired the thought of headphone. sensory substitution are often used.

Advantages: III. EXISTING SYSTEM The automatic system for reading text books are enforced that This paper presents a method to design a Text to Speech not solely converts written books to the digital text. conversion module by the use of Mat lab by simple matrix However conjointly reads them as Associate in Nursing audio operations. Firstly, by the utilization of microphone some output. This planned algorithmic rule will a lot of effectively similar sounding words are recorded employing a record handle advanced background and multiple patterns. Character program within the Mat lab window and recorded sounds are recognition associates in Nursing art of police investigation, saved in .wav format in the directory. The recorded sounds are segmenting and distinctive characters from image. The then sampled and therefore the sampled values are taken and advancement of automation method improves the interface separated into their constituent phonetics. The separated between man and machine in several applications. syllables are then concatenated to reconstruct the specified It is cheap.

IJESC, March 2020 24844 http:// ijesc.org/ VI. BLOCK DIAGRAM "general purpose computer". In July 2017, sales reached nearly 15 million.

VI. BLOCK DIAGRAM DESCRIPTION Webcam: The power supply is given to the 5 voltage micro USB connector of raspberry pi through the Switched Mode Power Supply (SMPS). The SMPS converts the 230 voltage AC supply to 5V DC. The online camera is connected to the USB port of raspberry pi. Raspberry pi has an Operating System named RASPBION which process the conversions. The audio output is taken from audio jack of the raspberry pi. The converted speech output is amplified using an amplifier. The web is connected to the Ethernet port in raspberry pi. The page to be read is placed on a base, and therefore, the camera is concentrated to capture the image. The captured image is processed by the OCR software which is installed in Raspberry pi. The captured image is converted to text by the OCR software. The text is converted into speech by the Text to Speech engine. The ultimate output is given to the amplifier from which it's connected to the speaker. Speaker also can get replaced by a headphone for convenience. Webcam could also be a little digital video camera directly or VII. HARDWARE DESCRIPTION indirectly connected to a computer or network .A webcam is a data input device that captures digital images. These are Raspberry Pi: transferred to the pc. A Webcam is generally connected by a The Raspberry Pi may be a series of small single-board USB cable or similar cable, or built into computer hardware, computers developed within the uk. It was developed to such as laptops. Webcam are known for their low promote the teaching of basic computer science in schools and manufacturing cost and their high flexibility, making them the in developing countries. The original model became far more lowest-cost form of video telephony. Despite the low cost, the popular than expected, selling outside its target market. It resolution offered at the present is quite impressive, with low- doesn't include peripherals (such as keyboards, mice and end webcams offering resolutions of 320*240,medium cases). However, some accessories are included in several webcams offering 640*480 resolution and high-end web official and unofficial bundles. According to the Raspberry Pi cams offering 1280*720(720p) or maybe 1920*1080(1080p) Foundation, over 5 million Raspberry Pi were sold by resolution. Other popular uses which includes security February 2015, making it the best-selling British computer. By surveillance, computer vision, video broad casting.The video November 2016 they had sold 11 million units, and 12.5 streams provided by webcams provided by webcams can be millon by March 2017, making it the third best-selling used for a number of purposes by using appropriate software.

IJESC, March 2020 24845 http:// ijesc.org/ Ultrasonic sensor: suggests that it's going to be connected to a router or on to Ultrasonic sound which vibrates at a frequency that is above another computer (without the necessity for a crossover cable). the range of human hearing. Ultrasonic sensor, use a single to send a transducer pulse and to receive the echo. This sensor USB hub determines the distance by measuring the time lapse between In order to attach additional devices to the RPi, you'll want to the sending and receiving the ultrasonic pulse. It consists of get a USB hub, which can allow multiple devices to be used. It four pin Vcc, power supply, echo, trigger. is recommended that a powered hub is employed - this may provide any additional power to the devices without affecting the RPi itself. A USB 2.0 model is recommended. USB 1.1 is Pin Pin Description ok for keyboards and mice, but might not be fast enough for No. Name other accessories.

1 Vcc The Vcc pin powers the sensor, Drivers typically with +5V L293D is also a twin H-bridge motor driver integrated circuit(IC).Motor drivers act as current amplifiers since they 2 Trigger Trigger pin is an Input pin. This take a low-current management signal and offer high-current pin has to be kept high for 10us signal. This higher current signal is utilized to drive the motors to initialize measurement by in forward or backward direction. In its common mode of sending US wave. operation, two DC motors area unit typically driven at the same time, both in forward and reverse direction. The motor 3 Echo Echo pin is an Output pin. This operation of 2 motors area unit typically controlled by input pin goes high for a period of logic at pins 2&7 and 10&15. Input logic 00 and eleven can time which will be equal to the stop the corresponding motor. Input logic 01 and ten can rotate time taken for the US wave to in right-handed and anticlockwise directions severally. return back to the sensor. DC Motor 4 Ground This pin is connected to the A DC motor is employed to drive a mechanical load. In this Ground of the system. model, a separately excited DC generator acts as the load. The load on the DC motor is adjusted by varying the generator field current. By increasing the sector current of the DC generator, the load on the DC, motor increases, and thus the armature current increases.

VIII. FLOW OF PROCESS

IMAGE CAPTURING AND PRE-PROCESSING: The video is captured by using the web-camera and the frames

HDMI to VGA Converter: from the video are segregated and that is undergone to the preprocessing. First, get the objects interest is extracted HDMI- High definition multimedia interface. from the camera image and it converted into gray image. Use cascade classifier for recognizing the character from the VGA- Video graphic array. It is the standard monitor or object. The work associated with a cascade classifier includes display interface used in most PC’s. two major stages, they are training and detection.

HDMI TO VGA CONVERTER is used to connect the Automation Text Extraction: monitor and the raspberry kit to convert the image into audio In order to handle complex backgrounds, two novel feature output. which maps to extracts text features based on the stroke orientations and edge distributions. Stroke is defined as the Power Supply: uniform region with the bounded width and significant extent. The unit is powered via the microUSB connector (only the These feature maps are combined to build an Ada-boost based power pins are connected, so it'll not transfer data over this text classifier. The actual transduction of text and image data connection). A standard modern phone charger with a into audio is the process of OCR output. microUSB connector will do, providing it can supply a minimum of 700mA at +5Vdc. Check your power supply's Text Recognition and Audio Output: ratings carefully. Suitable mains adaptors are going to be Text recognition is performed by the OCR from the localized available from the RPi Shop and are recommended if you're text regions. A text region labels the minimum rectangular unsure what to use. space for the accommodation of characters within it, that the border of the text region contacts the sting boundary of the Cables: text characters. However, this experiment show that OCR Video cable alternatives: o HDMI-A cable o HDMI-A cable + generates higher performance text regions area unit 1st allotted DVI adapter o Composite video cable o Composite video correct margin areas and binaries to segment text characters cable + SCART adaptor.Audio cable (not needed if you from background.The recognized text codes area unit recorded employ the HDMI video connection to a TV). Ethernet/LAN in script files. Then, use the Microsoft Speech code cable (Model B only) Development Kit to load these files and show the audio output. Blind users will alter speech rate, volume and tone Internet Connectivity: consistent with their preferences. They're designed to simply This may be through an Ethernet or LAN cable or a USB WiFi interface with dedicated pc systems by victimization identical adaptor. The RPi Model B Ethernet port is auto-sensing which USB technology that's found on most computers.

IJESC, March 2020 24846 http:// ijesc.org/ IX. RASPBIAN OPERATING SYSTEM display unit connected to raspberry pi3. currently regenerate text ought to be regenerate to voice to listen to name as voice Raspbian is an OS supported Debian optimized for the through ear phones connected to audio jack port. Ultrasonic Raspberry Pi hardware. An operating system contains a set of sensors area unit used for obstacle detection and calculation of basic programs which are used to run Raspberry Pi 3. distance between the obstacle and also the visually impaired Raspbian OS contains 35,000 packages, recompiled software person. It consists of transmitter and receiver. The transmitter bundled during a nice format for straight forward installation emits eight forty kilohertz pulse, this pulse once touch the on Raspberry Pi 3. Raspbian OS is one among the official obstacle is received back at the receiver, it records the time Operating systems available for free of charge to download taken by the emitted pulse to come back at the receiver finish. and use. The system is predicated on Debian Linux and is The system is self-made in warning the user regarding the optimized to figure efficiently with the Raspberry Pi presence of obstacles in their path. It will find any object at computer. Debian is very lightweight and makes a great choice intervals a pre-specified minimum distance. For out tests, we for the Pi. Operating systems for the Raspberry Pi can be tend to set the minimum distance price to zero.1m. The system installed on a Micro-SD, Mini-SD or SD card, depending on announces the space calculated in real time in meters or the board and available adapters. The Micro-SD slot is located centimeters through headphones. on the bottom of a Raspberry Pi 3 board. The Raspbian desktop environment is understood because the “Lightweight XII. CONCLUSION: X11 Desktop Environment.” Lightweight X11 Desktop Environment could also be a free desktop environment. This In this analysis, we got described an epitome system to scan makes especially suitable for use on older or system on a chip written communication and hand-held objects for serving to computer. the blind people. To extract text regions from advanced backgrounds, we have got projected a totally distinctive text X. FLOW CHART FOR OVERALL MECHANISM localization formula supported models of stroke orientation, and edge distributions. The corresponding feature maps estimate the worldwide structural feature of text at every element. Block patterns project the projected feature maps of an image patch into a feature vector. A adjacent character grouping is performed to calculate candidates of text patches prepared for text classification. Associate Ada boost learning model is used to localize text in camera-based footage. OCR is utilized to perform word recognition on the localized text regions and work into audio output for blind users. Throughout this analysis, the camera acts as input for the paper. As a result of the Raspberry Pi board is high-powered the camera starts streaming. The streaming data square measure attending to be displayed on the screen victimization interface application. Once the item for text reading is placed earlier than the camera then the capture button is clicked to supply image to the board. Mistreatment Tesseract library the image square measure attending to be converted into data, and conjointly the data detected from the image square measure attending to be shown on the standing bar. The obtained data square measure attending to be pronounced through.

XIII. FUTURE SCOPE

In the future, we can use more robust and the efficient XI. WORKING DESCRIPTION algorithm to read the images and separate the text from the images. The captured images was blurring, and then also we Once capture button is clicked this technique captures the will deblurred the image in the less time and can separate the image placed before of the net camera that is connected to info efficiently to convert them to the speech. raspberry pi3 through USB. once choosing the method button, the captured label image undergoes Optical Character XIV. REFERENCES Recognition (OCR) Technology. OCR technology permits the [1]. E. Cardillo, V. Di Mattia, G. Manfredi, P. Russo, A. De conversion of scanned pictures of written text or symbols into Leo, A. Caddemi, G. Cerri “An Electromagnetic Sensor text or data that may be understood or emended employing a bug. during this system for OCR technology TESSERACT Prototype to Assist Visually Impaired and Blind People in Autonomous Walking”,IEEEJournal,2017. library were used. mistreatment this library, the information are regenerate to audio. Camera acts as main vision in [2]. Mrs.Shilpa Reddy K, Mounika S.k,Pooja K , Sahana N sleuthing the label image of the merchandise or board then Text to Speech for the Visually Impaired”, IRJCS Journal, image is processed internally and separates label from image 2017. by mistreatment open CV library and eventually identifies the merchandise and known product name is pronounced through [3]. Mallapa D.Gurav, Shruti S. Salimath, Shruti B. Hatti, voice. currently it identifies received label image is regenerate Vijayalaxmi I. Byakod, Shivaleela Kanade “A Reading aid to text by mistreatment tesseract library. Once the known label for the Blind People using OCR and OpenCV”, IJSRET name is regenerate to text and regenerate text is showed on Journal 2017.

IJESC, March 2020 24847 http:// ijesc.org/ [4].Suchita Wankhade, Mrunali Bichukale, Shruti Desai, Shraddha Kamthe, Archana Borate “Smart Stick for Blind People with Live Video Feed”, IRJETJournal, 2017.

[5]. Miss. Kirti P. Bhure, Mrs. J. D. Dhande "Object Detection Methodologies for Blind People” IJRIT Journal 2017.

[6]. Ms. Athira Panicker “Smart Shopping assistant label reading system with voice output for blind using raspberry pi” International Journal of Advanced Research in Computer Engineering & Technology (IJARCET) Oct 2016.

[7]. K. Audenaert, H. Peremans, Y. Kawahara and J. Van Campenhout, "Accurate ranging of multiple objects using ultrasonic sensors" IEEE JOURNAL ,2016.

[8]. A. Carullo and M. Parvis, "An ultrasonic sensor for distance measurement in automotive applications" in IEEE Sensors Journal, Aug 2016.

[9]. William A. Ainsworth “A system for converting English text into speech on Audio and Electro acoustics”, Jun 2016.

IJESC, March 2020 24848 http:// ijesc.org/