Mobile Device Apps for People with Hearing Loss Expanding the Horizons of Hearing Access

By Larry Medwetsky Mobile Device Apps for People with Hearing Loss Expanding the Horizons of Hearing Access

Wherever one goes nowadays, people are using mobile devices or tablets. One of the virtues of these devices is the ability to download from the myriad applications (apps), in effect, allowing mobile devices to literally become an all-in-one system. This is part two of Dr. Medwetsky’s article on exploring the world of smartphone apps for people with hearing loss.

In the last issue of Hearing Loss Magazine I discussed various hearing screening/testing and personal sound amplification apps. In this issue I review some of the incredible apps that have made life more convenient for individuals with hearing loss. It truly is an amazing time and all of us should take advantage of the oppor- tunities that have opened up to us. These apps not only can help make communication easier but also allow for greater access to important alerts, and even for addressing hearing related issues such as tinnitus. Writing this article has been a tremendous learning experience. As someone who witnessed the introduction of the personal computer into the marketplace, I have been astounded by what is now available. Any mention of specific products does not imply any form of endorsement. And, in many cases I am relying on the accuracy of the product’s website, thus, I apologize for any false claims.

Communication Enhancement Applications This category includes a variety of apps that enhance communication access for individuals with hearing loss.

Face-to-Face Communication This category only includes apps that allow for direct communication without any augmentative support, such as captioning or the use of an intermediary. The latest Android and Apple operating systems provide high-quality video and a fast frame rate, which in turn, allow for effective communication for people with hearing loss

26 Hearing Loss Magazine Visit us at hearingloss.org and follow @HLAA on Twitter who benefit from speechreading when time of this article, the software was using the phone. in beta-testing. Another app that is not yet • FaceTime is an app that is included available but offers unique features is among the various iPhone/iPad op- RogerVoice. With RogerVoice, the tions and works right out of the box. individual with hearing loss calls or • Google Hangouts is an app that receives calls on their smartphone that allows for up to 10 individuals to are instant transcriptions of what the communicate with each other and other speaker is saying, regardless of can be downloaded from either the their language. Unlike other speech rec- App store (for iPhone or iPad) or ognition software that involves being Google Play (for Android devices) trained to a single voice, this software • Skype is another face-to-face app is meant to work with any voice, thus that can be downloaded to either errors can occur in the transcription. Apple or Android mobile devices. However, basic conversations—such as Video calls are available for 1:1 An example of mobile captioning ordering on the phone or making an communication (and more than appointment are less prone to errors. 1:1 on laptops/desktops, although An app that supplements the does allow for group audio calls) Real-Time Voice Recognition aforementioned mobile communica- Imagine going to a lecture or noisy tion apps is VoxSci for Mobiles. This All of the above are generally restaurant and catching all of the spo- app transcribes voicemails into text and free, except for some options of which ken information on the screen of your delivers them to the mobile device as a user might want to take advantage smartphone or tablet. Live Caption is a text (SMS) message and/or email (e.g., long distance audio calls). an example of a voice recognition app with MP3 attachment. A related app that has been developed for use in is Glide, which allows for video mes- Mobile Captioning everyday settings, whereby an individ- saging—be it live or kept for later. One recent innovation is analogous to ual speaks into the mobile device with voice-over, real-time captioned services the voice transcribed into real-time for landline phones (such as CapTel text. An adaptation of this approach is and CaptionCall). In this scenario, to have the talker speak into a Blue- individuals who use their voice but tooth microphone paired with the can’t hear well on the regular voice mobile device, thus, allowing a talker phone talk into their mobile device to be up to 30 feet away from the user’s or tablet and communicate with smartphone, yet still being able to see others via the use of a relay service. the text on the phone. Rather than the communications Let Me Hear Again is an assistant (CA) typing what the hear- app that allows for face-to-face chat, Transcence Voice recognition app ing person is saying (which requires converting voice into text (up to 41 allowing for group conversations a highly trained stenographer), the languages). The app can also transcribe CA who has “trained” the software instructions or directions into text mes- to recognize his or her voice, repeats sages for later recall, and even save class (shadows) what he or she hears from lectures to phone in the form of text. the hearing person. One voice recognition app that The software then transcribes appears to be promising for use at the CA’s spoken information for relay meetings or large group settings is to the screen of the mobile phone or Transcence. This app can transcribe tablet used by the person with hearing speech from various individuals onto loss. These apps are free upon regis- screen in near real time. To make this tration and depending on the service possible, the app connects to several chosen are available for either Android phones and activates their mics to cap- and/or iPhone. Companies providing ture what everyone is saying, then uses RogerVoice voice-to-text allowing for this service include ClearCaptions, voice recognition software to assign transcription in multiple languages CaptionCall Mobile, and Hamilton the text for each person in the group a Mobile CapTel. color for their speech bubbles. At the continued on page 28

Like HearingLossAssociation on Facebook November/December 2015 27 Mobile Device Apps • ViBe app allows one to choose RECREATION ENHANCEMENT APPLICATIONS continued from page 27 contacts and set different vibration patterns for each of them, thus, allowing the user to know who is Subtitled Apps for Movies Alerting Apps calling without even taking the and Television shows Hearing the doorbell, the phone ring, phone out of their pocket A number of apps, many of them or the fire alarm are everyday sounds free, allow you to download and view that are important to detect but might Hearing Loss Simulator subtitles for movies and TV shows. be difficult for an individual with The Hearing Loss Simulator app A downside to these free apps is that hearing loss. A number of apps have developed by Starkey is a wonderful they have some limitations and the been developed that enable the mobile tool to demonstrate how speech and presence of ads. Some of these apps do device user to be alerted to important environmental sounds might sound to not play the actual show but display environmental sounds. These apps can different individuals with hearing loss. synced subtitles while one is simulta- be used not only as a support at home, One can choose to use this as a general neously watching a movie or show on but also in a hotel room or any other education tool or find an audiogram television, or when one is at a movie enclosed environment. The following that most closely resembles your hear- theater. Some of these apps support are some of these apps: ing loss to demonstrate how speech more than 20 languages so it not might sound to you. restricted only to English users. • Braci Pro is an app that according to its website is able to detect up to a Synced Lyrics thousand sounds at a distance up to I have always loved listening to music. 35 meters away and recognize these However, it has always been frustrating sounds within 2-4 seconds. It is also to me that my hearing loss usually does compatible with the Pebble smart not allow me to understand the words watch or any other Android wear. in most tunes. Recently, downloadable apps have been developed for syncing • MyEardroid has been developed to lyrics to music on media sites such as detect and identify common sounds. Pandora, Shazam, Spotify, YouTube, Diverse sound notification modes etc. Some of these sites provide lyrics are incorporated, such as vibration, as an inherent feature of the app (such text and image. as Pandora, Shazam, and YouTube). Hearing loss simulation using the There are also a number of sites • Otosense comes with a general Starkey Hearing Loss Simulator app that do not provide lyrics, but one can library of sounds as well as be trained overcome this by using an external app to identify unique alerts in one’s to sync lyrics with the music site (such home or elsewhere. as syncing Musixmatch to Spotify). In addition to syncing the lyrics to the music, these apps have the ability to identify the name of the song title and music artist. Having this feature truly has added to my enjoyment of music.

Tinnitus Relief Tinnitus (i.e., ringing in the ears) is defined as the perception of sound in the absence of an external sound source and affects approximately 10 percent of the population (Kochkin, Tyler & Born, 2011). The goal of tinnitus management is to make symptoms less prominent and distressing. Current philosophies include a combination Example of the Braci Pro app displaying some of the sounds that can be detected of education/counseling, amplification and serve as alerts to individuals with hearing loss. and sound enrichment. The purpose

28 Hearing Loss Magazine Visit us at hearingloss.org and follow @HLAA on Twitter of sound enrichment is to provide sup- best in masking or suppressing their plementary sounds (often broadband) tinnitus. Each of Whist’s controls • Multi Messaging Service (MMS): to help clients focus their attention adjusts easily-understood character- The iPhone supports MMS which away from their tinnitus. A number istics of the sound. The individual enables users to send text or multi- of apps have been developed to assist is able to match the pitch, noisiness media to one or more individuals, individuals who are seeking relief from and loudness of the tinnitus to find thus, allowing for instant messaging, their tinnitus. The following is a review the best sound providing tinnitus chat rooms, and social networking. of some of the tinnitus apps that are relief. In turn, the user has controls available. for presenting either tones or noise, • FaceTime: As discussed earlier, adjusting the level/balance, and FaceTime allows for effective com- • Phase-Out allows the individual whether the sound is to be presented munication for individuals who rely to determine the frequency and continuously or in pulsed fashion. on speechreading for augmenting intensity of their tinnitus. The app the auditory signal when using subsequently plays back sounds that Built-In Features for Users of the phone or communicating via are 180° reversed phase waves for one Apple (iPhone and iPad) and American Sign Language. hour. The app manufacturer recom- Android Mobile Devices mends doing this over a number of Numerous features are already included Additional features can be days or few weeks. with the purchase of a mobile device. accessed on smartphones Perhaps the most important is one that via the following steps: • Phonak Tinnitus Balance (PTB) is required much lobbying and eventual intended to be part of a personalized legislation, that of hearing aid compati- 1. Select Settings tinnitus management program pro- bility. For individuals with hearing aids 2. Select General vided by a hearing care professional who used cell phones when they first familiar with the management of came out, many of them experienced a 3. Select Accessibility tinnitus. The PTB app is the sound buzzing in their hearing aids due to in- component of the program. Sounds compatible cell phones. Without going These features include: can be selected from the client’s into detail, most cell phones are now personal smartphone music library, able to provide a high-quality signal • Left/Right Balance: If an indi- or from the list of default sounds without any interference when used vidual has poorer hearing in one ear within the app. The individual by someone wearing hearing aids. versus the other, Apple devices allow rates how effective each sound is in for volume adjustment of the right/ achieving its desired effect (sooth- Apple iPhone and iPad left channels ing, background or interesting) and In addition to features such as sound/ the overall ratings can be reviewed ringer volumes, the following are some • Mono Audio: Songs are usually re- with the hearing care professional to of the built-features included on recent corded in stereo (i.e., separate tracks reach the optimal sound plan. If one iPhones/iPads: that play in the right and left chan- uses Phonak hearing aids, one can nels). This means that an individual stream the sounds over Bluetooth • Voice Recognition Software with a hearing loss in only one ear via a Phonak ComPilot directly to (VRS): Apple’s built-in VRS is (or who has much better hearing in one’s ears. Siri. This feature allows the user to one ear) can miss out on music or control the device or pose queries by audio content from the audio chan- • Tinnitus Masker (Explosive Apps) voice, as well as serve as a form of nel being delivered to the poorer ear. is designed to drown out sounds a dictating machine. Unlike many Mono audio allows the iPhone to associated with tinnitus. The sounds VRS trained to recognize specific direct both audio channels to each used by Tinnitus Masker are typically voices, Siri (as well as Android’s ear, thus an individual with a unilat- soothing, focusing on the frequency VRS) makes no attempt to model eral hearing loss can avoid losing a areas usually most associated with individual voices. Rather it records whole channel of sound tinnitus. Examples of sounds include short pieces of the incoming speech, white noise, rain drizzle, crickets, sends them off to a powerful Cloud • Subtitles and Captioning: By and waves on the beach. computer which compares the sound activating this option, videos of any pattern with a database of thousands kind that have built-in subtitles or • Whist provides relief for tinnitus of people speaking thousands of closed captioning can be displayed sufferers by allowing them to find/ words, and sends back the text which on the video screen. create/store sounds they find works produces the best statistical fit. continued on page 30

Like HearingLossAssociation on Facebook November/December 2015 29 Mobile Device Apps from page 29 if turned on, one can set the look of the captioning • Adjust the volume L/R balance Online • Visible and Vibrating Alerts: • Turn the Mono Audio feature on/off In addition to vibrating in silent • Turn on/off a baby cry detector Learning mode, the iPhone can be set to • Turn on/off Auto Haptic. When on, display a full screen image or photo the phone vibrates in time to sound for incoming calls, text messages, when listening to music or watch- Registration is open for the HLAA Hearing Loss Support emails, and calendar items. ing videos (a feature that some deaf individuals might enjoy) Specialist Training (HLSST) Under the Accessibility option, Concluding Thoughts HLSST is an online, self-paced one can also proceed to Step 4, and training program which provides choose “Hearing Aids.” In this mode It truly is amazing what is now avail- core knowledge for individuals who one can check “Bluetooth” to pair the able to people with hearing loss who work with people with hearing loss. cell phone to one’s hearing aids, as well use mobile devices. I have learned so The training consists of four classes as activate the “Hearing Aid Mode,” much while writing this article, and I with as many as fourteen lessons in which increases the cell phone’s electro- plan to take advantage of many of the each class. Lessons are composed magnetic field strength for effective use built-in features that come with my of multiple readings and captioned with hearing aids that have t-coils. smartphone as well as access some of videos. The classes are: the downloadable apps I have discussed. I hope these articles have been Class I: Hearing Loss: The Basics Android Devices of interest and that you have obtained Android phones/tablets include many Class II: Coping with of the same features as the iPhone/iPad; information that you can put to good Hearing Loss such as Left/Right balance, Mono use. It’s exciting to imagine what Audio, and Visible and Vibrating Alerts. the next few years will bring. I look Class III: Hearing Assistive It also possesses built-in voice recogni- forward to these new apps breaking Technology and Services tion software that can be accessed via down communication barriers in ways Class IV: Advocacy, Resources pressing the microphone icon. I could not have imagined just a few years ago. and the Law Well, I’m off to go listen to some To access the other built-in features, You may register for one class at the following steps must be carried out: music and use one of my syncing apps a time or register for the entire to sing along; hopefully, no one is program at once for the discounted 1. Select Settings around, otherwise, they’ll remind me price of $225. For complete details, of my terrific singing voice—Not! HLM table of contents, and registration 2. Under System, choose Accessibility information, visit hearingloss.org/ Larry Medwetsky, content/hlsst. 3. Select Hearing under Categories Ph.D. associ- Questions? Email Nancy Macklin ate professor can In this category, one has access to: at [email protected]. be contacted at Gallaudet Uni- • Pulse notification light for incoming versity at larry. notifications or calls medwetsky@ • Select hearing aids to improve the gallaudet.edu. sound quality of the phone when used with hearing aids with t-coils • Turn on/off the captioning program;

Apps Mentioned in this Article For the website addresses for the apps mentioned in this article, go to the online version of this article at hearingloss org>HearingLossMagazine> Current Issue. Click on the title of Dr. Medwetsky’s article and see the live links at the end of the article.

30 Hearing Loss Magazine Visit us at hearingloss.org and follow @HLAA on Twitter Apps Mentioned in this Article For the website addresses for the apps mentioned in this article, go to the online version of this article at hearingloss. org>HearingLossMagazine>Current Issue. Click on the title of Dr. Medwetsky’s article and see the live links at the end of the article. The following lists the URLS for various products discussed in this article. Unless indicated within parentheses, all of the apps are available for both Apple iPhone/iPad or android devices.

ALERTING APPS MOBILE CAPTIONING

Braci Pro ClearCaptions http://braci.co/ http://www.clearcaptions.com/

MyEardroid CaptionCall Mobile http://www.tecnalia.com/en/myeardroid/what-is- https://www.captioncall.com/CaptionCall_Solution myeardroid.htm (Android only) CaptionCall-Mobile.aspx (Apple iPad 2 or later devices only) Otosense http://www.betaboston.com/news/2014/09/08/otosense- Hamilton Mobile CapTel is-an-app-that-hears-and-learns-the-sounds-of-the-world/ http://www.hamiltoncaptel.com/smartphone/what_is_app. ($7.99 for either Apple or Android devices) html

ViBe app Real-Time Voice Recognition https://play.google.com/store/apps/details?id=com. Glide base2apps.vibes&hl=en (Android only) http://www.glide.me/ FACE-TO-FACE VIDEO CALLS Live Caption http://www.livecaptionapp.com/ Face Time ($3.99 to download) http://www.apple.com/ios/facetime/ (Apple only, easily accessed by an icon on the iPhone) Let Me Hear Again https://play.google.com/store/apps/details?id=appinventor. Google Hangouts ai_drsharonbaisil.LetMEHearAgain_Free_US https://plus.google.com/hangouts (Android only-free or Pro version for $9.99 to download)

Skype RogerVoice https://support.skype.com/en/faq/FA10613/what-do-i- https://www.kickstarter.com/projects/olivierjeannel/ need-to-make-a-group-videocall?q =Group+video+chat rogervoice-phone-engaging-the-world-with-the-hard (Android only) HEARING LOSS SIMULATOR Transcence Starkey Hearing Loss Simulator http://www.engadget.com/2014/10/15/transcense-app- https://itunes.apple.com/us/app/hearing-loss-simulator/ speech-deaf/ id398352094?mt=8 VoxSci for Mobiles ($1.99 for Apple only) http://www.voxsci.com/

continued on page 32

Like HearingLossAssociation on Facebook November/December 2015 31 Mobile Device Apps from page 31 TINNITUS RELIEF

Phase-Out iTunes SUBTITLE APPS FOR MOVIES http://appcrawlr.com/app/search?go=go&q=tinnitus&devic e=iphone (Apple only) http://www.technorms.com/44159/ automatically- download-movie-subtitles-on-android Phonak Tinnitus Balance Google Play https://play.google.com/store/apps/details?id=com.phonak. SYNCED LYRIC SITES tinnitus&hl=en

Pandora Google Play Phonak Tinnitus Balance iTunes https://play.google.com/store/apps/details?id=com.pandora. https://itunes.apple.com/us/app/tinnitus-balance/ android&hl=en id621383952?l=de&ls=1&mt=8

Pandora iTunes Tinnitus Masker (Explosive Sounds) iTunes https://itunes.apple.com/us/app/pandora-radio/ https://itunes.apple.com/us/app/tinnitus-masker/ id284035177 id301353562?mt=8 Shazam (Apple only) http://www.shazam.com/apps Whist YouTube Google Play http://www.sens.com/products/whist-custom-sound- https://play.google.com/store/apps/details?id=com.google. therapy-for-tinnitus/ android.youtube&hl=en (Free for Apple via iTunes; $1.99 for Android via Google Play) YouTube iTunes: https://itunes.apple.com/us/app/youtube/ id544007664?mt=8

SYNCING LYRIC—EXTERNAL SITES

Musixmatch https://play.google.com/store/apps/details?id=com. musixmatch.android.lyrify&hl=en (Android only)

Sound Hound Music Search Google Play http://appcrawlr.com/android/soundhound-2 ($5.99 to download)

Sound Hound Music Search iTunes http://appcrawlr.com/ios/soundhound-2

32 Hearing Loss Magazine Visit us at hearingloss.org and follow @HLAA on Twitter