International Journal of Research in Engineering, Technology and Science, Vol. VIII, Issue III, March 2018 www.ijrets.com, [email protected], ISSN 2454-1915

DEVELOPING CLOUD-BASED APP FOR VISUALLY IMPAIRED PEOPLE TO READ THE TEXT

Dr. Sagar Tambe1, Mr. Suraj Chandgude2, Mr. Shubham Sodewad2, Mr.Shriraj Kesarwani2 and Mr. Akshay Wakchaure2

Assistant Professor1 , UG Student2, PREC, Department Of Computer Engineering, Loni, India

ABSTRACT: For the outwardly hindered portion of the populace, the failure to peruse has a substantive negative effect on their personal satisfaction. Printed content (books, magazines, menus, names, and so forth.) still speaks to a sizable bit of the data this gathering needs unhindered access to. Henceforth, creating techniques by which content can be recovered and perused so anyone can hear to the visually impaired is basic. In this work, we talk about the plan and usage of two assistive stages, in one, we join today’s' cell phone abilities with the points of interest offered by the quickly developing cloud assets, and alternate uses a more temperate approach making utilization of financially savvy microcontrollers. Both methodologies make us of an Optical character Recognition (OCR) motor on the cloud and utilize neighborhood assets for the Text-to-Speech (TTS) transformation. Models are effectively created and tried with good outcomes.

KEYWORDS: Optical character Recognition (OCR), Text-to-Speech (TTS), Visual Inability, , Text Extraction

• INTRODUCTION In Existing System researchers have attempted to ease the burden on blind people by proposing various techniques that converts text to audible sounds. Tyflos is a pair of glasses that had cameras attached to the side, earphones, and a microphone. Voice commands can be used to guide the user and direct the platform. Some commands include “move paper closer,” “move paper up,” “move paper up, right” from the device to the user, and “rewind paragraph,” “forward paragraph,” and “volume up” from the user to the device. However, the voice might not function perfectly in a noisy environment, rendering it limited to indoor use. Finger Reader is a wearable ring with a camera on the front.

1 Dr. Sagar Tambe, Mr. Suraj Chandgude, Mr. Shubham Sodewad, Mr.Shriraj Kesarwani and Mr. Akshay Wakchaure International Journal of Research in Engineering, Technology and Science, Vol. VIII, Issue III, March 2018 www.ijrets.com, [email protected], ISSN 2454-1915 The world of print information such as newspapers, books, sign boards, and menus remain mostly out of reach to visually impaired individuals, in an effort to seek an answer to this persistent problem, an assistive technology based solution, referred in this project. We propose Mobile phones are one of the most commonly used electronic gadgets today. Here, we intend to develop a modular and friendly application using cloud based OCR platform and the built in Android TTS for producing an audible result of the text file. Motivation: The world of print information such as newspapers, books, sign boards, and menus remain mostly out of reach to visually impaired individuals, in an effort to seek an answer to this persistent problem, an assistive technology based solution, referred to in this paper as Read2Me. • LITERATURE SURVEY 1. Nagaraja L, Nagarjun R S, Nishanth Anand, Nithin D, "Vision based Text Recognition using Raspberry Pi", National Conference on Power Systems & Industrial Automation (NCPSIA), 2015. Human communication today is mainly via speech and text. To access information in a text, a person needs to have vision. However those who are deprived of vision can gather information using their hearing capability. The proposed method in this paper is a camera based assistive text reading to help blind person in reading the text present on the text labels, printed notes and products [1]. This paper involves Text Extraction from image and converting the Text to Speech converter, a process which makes blind persons to read the text. This is the first step in developing a prototype for blind people for recognizing the products in real world, where the text on product is extracted and converted into speech. This is carried out by using Raspberry pi, where portability is the main aim which is achieved by providing a battery backup and can be implemented as a future technology. The portability allows the user to carry the device anywhere and can use any time. 2. Nagarathna and Sowjanya V. M, "Product Label Reading System For Visually Challenged People", International Journal of and Information Technology Research ISSN 2348-120X (online), April - June 2015. This paper presents a camera-based label reader to help blind persons to read names of labels on the products. Camera acts as main vision in capturing the video of the product then video captured is split into frames. From sequence of frames one frame is taken and processed internally to separate label from frame using text detection algorithm. The received label image is converted to text by using OCR (Optical Character Recognizer). Once the identified label name is converted to text and converted text is output to blind users in speech. 3. Roberto Netoa and Nuno Fonsecaa, "Camera Reading For Blind People ", International Conference on Health and Social Care Information Systems and Technologies , 2014. Blindness makes life rather difficult for people who suffer from this health problem, but the use of technology can help in some day-to-day tasks. In this paper, the present work focuses the development 2 Dr. Sagar Tambe, Mr. Suraj Chandgude, Mr. Shubham Sodewad, Mr.Shriraj Kesarwani and Mr. Akshay Wakchaure International Journal of Research in Engineering, Technology and Science, Vol. VIII, Issue III, March 2018 www.ijrets.com, [email protected], ISSN 2454-1915 of a photo-to-speech application for the blind. This project is called Camera Reading for Blind People, and its ultimate purpose is the development of a mobile application that allows a blind user to "read" text (a sheet of paper, a signal, etc.). To achieve that, a set of frameworks of Optical Character Recognition (OCR) and Text to (TTS) are integrated, which enables the user, using a smartphone, to take a picture and hear the text that exists in the picture. 4. Robert Keefer and Nikolaos Bourbakis, "Interaction with a Mobile Reader for the Visually Impaired", IEEE International Conference on Tools with , 2009. While reading devices for the visually impaired have been available for many years, they are often expensive and difficult to use. In an effort to prototype a device comprised of inexpensive components, this paper present a highly interactive method for processing document images. The voice user interface (VUI), document image segmentation and framing, and targeted image registration methods of a mobile reading device are presented. Initial experimental results on a set of newspaper pages are also presented. 5. Roy Shilkrot, Pattie Maes, Jochen Huber and Suranga C. Nanayakkara, " FingerReader : A Wearable Device to Support Text Reading on the Go ", Singapore University of Technology and Design, 2016. Visually impaired people report numerous difficulties with accessing printed text using existing technology, including problems with alignment, focus, accuracy, mobility and efficiency. This paper presents a finger worn device that assists the visually impaired with effectively and efficiently reading paper-printed text. In this paper introduce a novel, local-sequential manner for scanning text which enables reading single lines, blocks of text or skimming the text for important sections while providing real-time auditory and tactile feedback. The design is motivated by preliminary studies with visually impaired people, and it is small-scale and mobile, which enables a more manageable operation with little setup. 6. Rajalaskhmi P, Deepanraj S, Arun Prasath M. and Dinesh Kumar S, "Portable Camera Based Visual Assistance For Blind People", ARPN Journal of Engineering and Applied Sciences, APRIL 2015 . This paper presents a portable camera based visual assistance prototype for bind people to identify currency notes and also helps them to read printable texts from the handheld objects. To read printable texts, an efficient algorithm that combines an Optical Character Recognition (OCR) with Hierarchical optimization is used. In Pattern Recognition OCR every character is localized and separated then the resulting character image is sent to a pre-processor to reduce noise and to perform normalization. Certain characteristics will be extracted from the character for comparison. After comparison the identified characters are grouped and reconstructed to form original text strings, then the output is given to the speech engine to perform text to voice conversion. For identification of currency notes a novel recognition system is developed using SIFT (Scale Invariant Feature Transform) to improve precision and accuracy. The input image undergoes pre-processing and thereafter the distinct features are

3 Dr. Sagar Tambe, Mr. Suraj Chandgude, Mr. Shubham Sodewad, Mr.Shriraj Kesarwani and Mr. Akshay Wakchaure International Journal of Research in Engineering, Technology and Science, Vol. VIII, Issue III, March 2018 www.ijrets.com, [email protected], ISSN 2454-1915 extracted and compared with the templates from the database. The resulting outcome is given through Earphone to the blind users.

• SYSTEM ARCHITECTURE Our proposed system uses android device that gives voice commands to guide the user and direct the platform. The user only needs to point at the text that they would like to read, and the device will use local sequential text scanning in order to read each line of text progressively. OCR and TTS tools were integrated into an application in order to capture images and return audio as output back to the user.

Figure1. System Architecture.

Modules 1. Image capture with the camera : The camera of the mobile device is critical to use the application, since it will be essential for the user to take the picture of the brackets containing text that will be recognized and synthesized. The device used in the project is an Android smartphone. The application will run in full screen mode. The captured image will be the one displayed on the screen and will be send for proccessing. 2. Pre-Processing : Preprocessing of the image includes following stages : i) Image Acquisition : The Image acquisition component collects scenes containing objects of interest in the form of images. ii) Gray Scale Conversion : 4 Dr. Sagar Tambe, Mr. Suraj Chandgude, Mr. Shubham Sodewad, Mr.Shriraj Kesarwani and Mr. Akshay Wakchaure International Journal of Research in Engineering, Technology and Science, Vol. VIII, Issue III, March 2018 www.ijrets.com, [email protected], ISSN 2454-1915 To make the system more robust i.e. for the purpose of reducing the processing time of the overall process, the input is converted into Gray Scale. Preprocessing of document images is the way of using mature image processing techniques to improve the quality of images. Its purpose is to enhance and extract useful information of images for later processing purposes. iii) Edge detection : Edge detection is a set of mathematical method which aim at identifying point in a image at which image brightness changes sharply are typically organized into a setup for the line segment turned are edges. iv) Binarizing the image : The reduced image has to be binarized by erasing out all pixels outside the bounding boxes.(i.e.) All pixels outside the bounding box will be set to white(0) and those within the box will be set to black(255). v) Segmentation of the image : To do this the text candidates are extracted from the grayscale image. Segmentation concludes by enhancing the contrast on the obtained text image. vi) Text Extraction : An automatic text extraction algorithm is implemented to detect the region containing the label text. 3. Optical Character Recognition (OCR) : Optical character recognition, usually designated by the acronym OCR, is the process of recognition and automatic conversion of existing characters in the written-support image into the text format, which can then be used in various applications. The OCR has been widely studied and has displayed considerable advances regarding the performance and accuracy of the obtained results . 4. Text-To-Speech (TTS) : Voice synthesis, defined as TTS (acronym for Text-To-Speech), is a computer system that should be able to read aloud any text. The use of TTS aims to produce human voice artificially. Voice synthesis is a complex process . TTS synthesis makes use of techniques of Natural Language Processing. Since the text to be synthesized is the first entry of the system, it must be the first to be processed. • MATHEMATICAL MODEL Let S be the whole System, S = { I, P, O } I = input P = procedure O = Output • I = { I0, I1, I2 } I0 = {Login} Login = {Username, Password} I1 = Capture Image I2 = Upload Image 5 Dr. Sagar Tambe, Mr. Suraj Chandgude, Mr. Shubham Sodewad, Mr.Shriraj Kesarwani and Mr. Akshay Wakchaure International Journal of Research in Engineering, Technology and Science, Vol. VIII, Issue III, March 2018 www.ijrets.com, [email protected], ISSN 2454-1915 • P = { P0, P1, P2, P3, P4, P5, P6, P7 } P0 = Image Aquisition P1 = Gray Scale Conversion P2 = Edge detection P3 = Image Binarization P4 = P5 = Text extraction P6 = Optical Character Recognition P7 = Text-to-Speech Conversion • O = { O0, O1 } O0 = Text Output O1 = Audio Output

• Applications 1. It can be used by visually impaired individual to understand printed data. 2. It can be also used by Deaf people to read data converted from voice. • SYSTEM REQUIREMENT • HARDWARE RESOURCES REQUIRED 1. System : Pentium IV 2.4 GHz. 2. Hard Disk : 40 GB. 3. Floppy Drive : 1.44 Mb. 4. Monitor : LCD. 5. Mouse : Logitech. 6. Ram : 512 Mb. 7. Android phone

• SOFTWARE RESOURCES REQUIRED 1. : and above. 2. IDE : Android studio 3. Programming Language : Android, Java 4. Database : SQL

• Conclusion and Future Work We have discussed about the outline and usage of a frameworks that add to the capacity of the visually- challenged to share with us access to printed message paying little mind to its arrangement. The

6 Dr. Sagar Tambe, Mr. Suraj Chandgude, Mr. Shubham Sodewad, Mr.Shriraj Kesarwani and Mr. Akshay Wakchaure International Journal of Research in Engineering, Technology and Science, Vol. VIII, Issue III, March 2018 www.ijrets.com, [email protected], ISSN 2454-1915 frameworks make utilization of the cloud as a practical and viable asset for character acknowledgment. Framework is less demanding to utilize. However, the accuracy of the mobile in the conversion efforts is better, primarily due to the high determination camera worked in the gadget. In future upgrades of this work, we expect this will enhance its precision. We anticipate more work will be delivered in this basic territory of assistive innovation, and venture that future versatile devices will have simple to utilize and worked in component as perusing helps for the visually impaired, comparative, to the portable based arrangement displayed here.

• References

1. World health organization official website. August 2014. [Online]. Available at http://www.who.int/mediacentre/factsheets/fs282/en. Accessed on March 4, 2015.

2. Nagaraja L, Nagarjun R S, Nishanth M Anand, Nithin D, "Vision based Text Recognition using Raspberry Pi", National Conference on Power Systems & Industrial Automation (NCPSIA), 2015.

3. Nagarathna and Sowjanya V. M, "Product Label Reading System For Visually Challenged People", International Journal of Computer Science and Information Technology Research ISSN 2348-120X (online), April - June 2015.

4. Roberto Netoa and Nuno Fonsecaa, "Camera Reading For Blind People ", International Conference on Health and Social Care Information Systems and Technologies , 2014.

5. Robert Keefer and Nikolaos Bourbakis, "Interaction with a Mobile Reader for the Visually Impaired", IEEE International Conference on Tools with Artificial Intelligence, 2009.

6. Roy Shilkrot, Pattie Maes, Jochen Huber and Suranga C. Nanayakkara, " FingerReader : A Wearable Device to Support Text Reading on the Go ", Singapore University of Technology and Design, 2016.

7. Rajalaskhmi P, Deepanraj S, Arun Prasath M. and Dinesh Kumar S, "Portable Camera Based Visual Assistance For Blind People", ARPN Journal of Engineering and Applied Sciences, APRIL 2015 .

8. THOMAS, S. " Natural Sounding Text-To-Speech Synthesis Based On Syllable-Like Units ", Department Of Computer Science And Engineering Indian Institute Of Technology Madras, 2007.

9. Elmore, M. And Martonosi, M. " A morphological image preprocessing suite for ocr on natural scene images ", 2008.

10. " Read2Me: A Cloud- based Reading Aid for the Visually Impaired ", The Master of IEEE Projects, 2015.

7 Dr. Sagar Tambe, Mr. Suraj Chandgude, Mr. Shubham Sodewad, Mr.Shriraj Kesarwani and Mr. Akshay Wakchaure