Let’s Talk about Blind Accessibility By Adriane Kuzminski Overview

• 1. History of blind accessibility in tech • 2. Blind-accessible design principles • 3. The accessibility community and developing Personas • Group discussion 1. Blind Accessibility in Tech History of Screen Readers for VI Users

• Speech synthesis projects aimed at blind users: • 1976 – Kurzweil Reading Machine (OCR – optical character recognition) • 1980s – Talking Typewriter by IBM Research (first blind employee in 1914 [1] )

• Command-line interface (CLI) computers: • Accessible to blind users with screen readers, like the IBM Screen Reader in 1986 [2]

• Popularity of Graphical User Interfaces (GUIs) – new set of challenges for blind users: • Icons were not able to labeled, incompatible with screen readers • Original Macintosh had an inaccessible keyboard with no arrow keys

• Screen readers compatible with GUI computers: • Commercially available with IBM’s Screen Reader/2 (1994) [3] and ALVA Access Group’s outSPOKEN for Macintosh (1989) [4] and Windows (1994) [5] The Macintosh Keyboard released in January 1984 had no arrow keys

The Macintosh Numeric The 1986 Macintosh Keypad (Sept 1984) Plus Keyboard included arrow keys Modern Screen Readers for VI Users

• Screen readers: • Desktop: VoiceOver for Mac; JAWS, NVDA, Narrator, etc. for Windows • Mobile: VoiceOver for iOS; TalkBack and Voice Assistant for Android • Consoles: Xbox One – Narrator for home screen, Xbox API for in-game text • Games: Polly for Amazon Web Services and Lumberyard, MonoGame, Python, proprietary • Game Engines: • DAWS:

No native screen reader for engine or Screen reader accessible on Screen reader accessible with game implementation yet Mac with Flo Tools (2016) SWS + Osara extensions (2015) More Improvements in Blind-Accessible Media

• Virtual assistants (Alexa, Cortana, Siri, Google Home): • 21% of the US population owns as smart speaker as of January 2019 [1] • Computers, smart phones, consoles, smart TVs, & watches have built-in virtual assistants – over 7 billion devices expected with virtual assistants by 2020 [2] • Virtual assistants have improved in their affability and flexibility in interpreting commands, making them more likely to be used regularly by sighted people – could allow “default” accessibility for users with visual impairments (however, virtual assistants ≠ screen readers) Recent Improvements in Blind-Accessible Media

• Audio Description: • Theaters: • Final Rule (2016) requires a certain amount of audio description equipment in US theaters [2] • Television: • FCC requires the top 5 networks to broadcast 87.5 hours of audio-described content per quarter (2017) – up from 50 hours [2] • Streaming: • Netflix adds an average of 22 newly audio-described titles per month (total 931 titles in April 2019) [3] - received backlash over Daredevil series in 2015 and took action • iTunes has over 1,000 audio-described titles as of April 2019 [4] • As of 2016, all Pixar films are audio-described on Disney Movies Anywhere app [5] • Americans with Disabilities Act (ADA): Hulu only has a “plan” to start adding audio description, class action lawsuit filed in 2017 [6] The Communications and Video Accessibility Act

• Part of FCC - not under ADA (“public places”), Section 508 or WCAG 2.0 • Games: • The CVAA requires text-to-speech (TTS) for devices with advanced communications services to include electronic messaging services like in-game chat on consoles as of 1 January 2019 • Xbox One is clear frontrunner: TTS support for in-game chat in Halo Wars 2, Forza Motorsport 7, Forza Horizon 4 and Sea of Thieves, as well as voiced menus in Crackdown 3 and The Division 2 • Crackdown 3 also employs the Xbox One’s “Let games read to me” accessibility option for accessible UI elements, though limited • Other games have developed their own solutions, such as the platform- agnostic TTS solution in Apex Legends Recent Timeline of Blind-Accessible Games

The IGDA Game Papa Sangre is Injustice: Gods A Blind Legend Microsoft’s Zach Accessibility Special released and uses Among Us is patched receives nearly Quarles writes about Interest Group is binaural audio to be with an Accessibility $50,000 USD in designing sound for established [1] played without a Mode for blind crowd funding plus blind players in Killer screen [3] gamers [5] public assistance [7] [8] Instinct [10] Academic AAA Indie AAA Indie CVAA….

Acad Academi Indie Indie AAA Indie Q4 AAA Q4 2003 emic 2005 2010 2011 2013 c 2014 2015 2017 2018 2019

Academic Indie Indie Indie AAA

Audiogames.net, Swamp is released Skullgirls is patched Frequency Missing, Madden NFL 18 website for VI as an audio-only with text-to-speech a blind-accessible becomes blind gamers, is created by multiplayer first- compatibility; Tolk point-and-click accessible through Sander Huiberts and person shooter [4] support added in adventure, was haptic feedback [11] Richard van Tol [2] 2016 [6] released [9] Who Can Benefit from Blind Accessibility? Blind/Low Vision • 36 million people worldwide are blind [1] • 217 million are moderately to severely visually impaired • 188.5 million people are mildly visually impaired • Audiogames.net has over 125,000 registered users [2]

Aging Gamers 50+ • US – 1 in 4 gamers are over the age of 50 (2018) [3] • Australia – 43% of people over 64 years old play games (2017) [4]

Dyslexia New language learners [6] • 1 in 10 people have dyslexia [5] • Duolingo alone has 300 million active users (2018) Benefits Outside Gameplay

• Quality of life: Accessible games… • promote technological independence/job skills: research, education, design, retail, customer service, etc. • encourage people to learn to program using blind-accessible game engines (MonoGame, Python) and integrated development environments (IDE) like Visual Studio • create shared experiences and help blind and sighted people relate to each other • expose players to cultural expression and Art in video games • encourage gamers with disabilities to express themselves through game dev and streaming

• Tech advancements: • Game accessibility improved design for new tech like interactive car audio, virtual assistants, hands-free/eyes-free devices, voice user interfaces (VUI), e-readers, VR audio (reduce strain of subtitles/icons, motion sickness) • Reach a wider audience, both disabled and abled There are blind accessible solutions for almost every other type of media…

Books Film & Television Internet • Audiobooks • Audio Description • Screen Readers • Screen readers • Web Content Accessibility Guidelines (WCAG 2.0) So why should we ignore games? Challenges

• Culture • Sighted people are sensitive to audio output they don’t know how to control (like text-to-speech, pop ups with audio) – interaction improved with flexible commands of virtual assistants • Sighted people in general have stronger memory capabilities for visual and tactile stimuli compared to auditory stimuli (Bigelow, Poremba) [1] • Myths about the blind community [2] • Industry • Game, UI, and UX design are visually-oriented fields • Sound design is often added in the later stages of development • Game audio often focuses on aesthetics – though games like Apex Legends, Killer Instinct and Overwatch have innovative solutions for portraying gameplay info • Text-to-speech is crucial for blind accessibility but still in early stages for in-game menus/UI elements [3] • Sound design and screen reader implementation are handled by separate departments • Mobile – developing for TalkBack isn’t the same as for VoiceOver 2. Blind-Accessible Design Blind-Accessible Design

• Accessibility: • “Giving as many players as possible the best opportunity of completely experiencing a game” – Josh Straub of DAGER System [1]

• Blind-accessible audio can help players: • Learn to play and control the game • Understand the story • Be more engaged and informed players

• Can player understand …where they are and can go? …what they are supposed to do? …how they are intended to feel? Associated Fields

• Interaction Sound Design/Sonic Interaction Design • Employing sound to convey information, meaning and aesthetic/emotional qualities in interactive contexts [1] • User Experience Design (UX) • Enhancing user satisfaction by improving usability, accessibility and pleasure in interacting with product [2] • … not technology driven – focuses on human experience • … not anecdotal • … not synonymous with usability or User Interface design • Usability • Product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use. [3] • User Interface Design • Visually-oriented – but values gameplay information over immersion • User control, user memory load, consistency Prototype – Tower Defense 2D assets https://assetstore.unity.com/packages/templates/packs/tower-defense-2d-85612

https://player.vimeo.com/video/251516171 Instructional Information

• “What should I do?” • Purpose: Help players learn game with audible menus, icons, tutorials and actions • Use methods to enhance player’s memory • Logical, consistent, predictable patterns • Repetition – “one sound, one function” Repetition and Patterns

• Warnings – Iconic “!” alert from Metal Gear Solid series; nearby collectables in Uncharted 2; interaction cues in Injustice 2; zombie hordes in Left 4 Dead • Confirmations – hit other player in MarioKart; full shadow/instinct meters in Killer Instinct • Progress – puzzle complete in Zelda series; enemy/friendly radio in Metal Gear Solid series, puzzle comments in Uncharted series • Unique character sounds – Overwatch, Mortal Kombat, Injustice Prioritize mix and informational dialogue/text

• Mix: (Prioritize what player should notice) • Overwatch – mixed by threat level instead of realism; sounds from enemies are more noticeable than sounds from teammates; friendly/enemy ultimate abilities (no occlusion) [1] • Killer Instinct – focused on player movements: footsteps, jumps, landing, Foley movements, etc. “Ambient loops and sweetener layers that didn’t provide strong player feedback were backed off a bit.” [2] • Apex Legends – distant gunshots are louder if enemy points directly at you [3] • Non-story text: • Be brief as possible • Give player choice of screen reader if possible for varying speech rates Alternate sound types

• Use both spoken text and iconic sounds • Teach gameplay with spoken text and allow UI sounds to represent groups of information • Hearthstone – cards have a picture and description, eventually description is memorized and decision can be made by seeing the picture • 2DTD – Alternated between iconic sounds and spoken words • TTS for selecting menus, towers, # of enemy/gold • Iconic SFX to confirm selections Positional Information

• “Where am I, and where should I go?” • Purpose: Make sure player orientation is clear and that they can confirm their location and direction whenever they need to Leading Direction

• Simulated acoustics: • Use detailed positioning with 5.1 imaging/Dolby Atmos, HRTF, obstruction, occlusion, reverb, VR audio plugins (dearVR, Aspic Engine, Steam Audio) • Stereo panning: • Let players know where they are on screen for 2D platforms (Killer Instinct) • Let players know which side of enemy they’re on (Mortal Kombat) • Touch-activated panning: • Find items by dragging finger on screen (Frequency Missing) • Sound beacons: • Player follows or avoids sounds (Papa Sangre, Sight Unseen) • Player follows characters (A Blind Legend) Confirming Direction

• Cardinal directions: • Indicate whether facing east, west, etc. (A Hero’s Call, Swamp) • Sonar (Alien: Isolation, A Hero’s Call) • Orientation descriptions: • Describe what a player touches on mobile (Frequency Missing) • Describe objects in front of the player in first-person and VR games (Accessible Realities) • Warnings of danger/off-limits areas: • Warn when leaving play area/battle arena (MGS: Ground Zeroes) • Warn if no enemies remain (Fallout: New Vegas) Confirming Location

• Memorable SFX and leading paths: • Unique doors or footstep surfaces that indicate correct path much like the red climbing paths in the Uncharted games and locator paths in Dead Space • Memorable ambiences: • Use detailed audio cues or ambiences specific to a contained area (Silent Hill 2, Far Cry 5) • Audio description of environments: • Describe rooms and notable story details (Frequency Missing, story games for Alexa/Earplay) • Don’t underestimate mental mapping: A Hero’s Call Emotional Information

• “What should I feel?” • Purpose: Player should not be distracted from task or caused unintentional anxiety. Add/Remove Detail

• Relaxing Music/SFX: (for gamers with autism/SPD) • Allow players to concentrate with unobtrusive music (Civilization, Bejeweled Stars) • Audio sliders: • Allow players to remove sounds that are distracting or that they are sensitive to that they can reintroduce later (Pit People) • Cinematic audio description (AD): • Describe cinematics to the standards of television audio description Challenges

• Audio Sliders: • Hard to standardize – sounds triggering SPD are different for everyone • Audible UI: • Repetition can be annoying • UI sounds can be hard to memorize • Expectations for UI sound are far behind graphical user interface (GUI) • SFX and spoken text take up the soundscape • Location: • Realism ≠ accessible • Hearing information ≠ player control • Position ≠ direction Accessible Genres

• Games that allow players to make one choice at a time and allow enough time to make an informed decision can be blind-accessible. • Point and click adventures: Frequency Missing • Turn-based strategy games: A Hero’s Call, Entombed • Turn-based sports games: Madden 2018 • Trading card games: games like Hearthstone • Side-view fighting games: Mortal Kombat, Injustice, Skullgirls • Interactive story games: audio games for Alexa and Earplay • Text adventures: King of Dragon Pass, games by Choice of Games • Visual novels: STEINS;GATE, Narcissu Other methods • Tweakable font sizes: Life is Strange • Haptic feedback: Madden 18 • Keyboard accessibility: Spelunky • Control game clock: Eagle Island • Assisted steering/easy mode: MarioKart 8, • Aim assist, auto-target, re-orientation: Far Cry 5: New Dawn? • Turn-based combat: A Hero’s Call • Remove quick time events/timers: Deadpool, LA Noire • Reset/rewind: GRIP, Forza Motorsport 7 • Refreshable/full-page Braille displays • Braille typing for touchscreens Core Elements for Accessible Sound

Speed Flexibility Memory load Notifications/ Emotion and reinforcements storytelling

Image modified from www.vecteezy.com Speed • Be concise and place key terms first • Allow dialogue to be interrupted if not Be concise and allow interruptions crucial • Sounds that are often interrupted should be shortened or ducked • Rarer, more gratifying sounds can last longer

33 Flexibility • Allow players to choose speed of Allow players to choose the dialogue speed and clarity of sounds • Include audio sliders focused on reducing distractions/anxiety Working memory • Tutorials and pacing Limit what needs to be • Familiarization memorized in certain • Real-life amounts of time • Genre-specific • Standardization (vs. audio branding) • Logical, consistent and predictable patterns • Repetition/“one sound, one function” • Alternate between dialogue and SFX • Allow access to help Working memory • Memory load: Arrow keys, Enter, Escape, G = gold, E = enemies, upgrade defender Limit what needs to be successful, no gold left memorized in certain amounts of time • Familiar SFX: Enemy footsteps and panning, enemy deaths, defender deaths, battle horn

• Dialogue: type of defender, selected towers, gold accrued, enemies remaining Signs and Feedback Signs and Feedback: Let players know what to • Don’t overdo reminders expect and what you expect • Hierarchy of importance of them • Allow player to initiate certain info • Ducking • Use clear patterns to deter wrong actions Story and Emotion • Use voice actors for story content Find a balance between • Use screen readers for: storytelling and instructions • Dry instruction (menus, tutorials) • When speedy text-to-speech can offer players an advantage: https://www.vincit.fi/en/blog/software -development-450-words-per-minute/

Image modified from www.vecteezy.com 3. People and Personas Accessibility Advocates

Brandon Cole SightlessKombat Tomasz Tworek IllegallySighted @superblindman @SightlessKombat @lirin111 @BGFH79 Blind Twitch/YouTube Streamers

Kephas Meghan Dornbrock Ross Minor Liam Erven Steve Saylor blindgamer102

@TheKephas @meglish @ross_minor @Liamerven @stevesaylor @blindgamer102 New Blind-Accessible Games

Reverge Labs Nifty Sound Earplay Iris Meyer, Out of Sight Blind Sparrow Hansjörg Mikesch Games Interactive

Skullgirls Frequency Jack Ryan, Feer A Hero’s Call Upcoming… Missing Jurassic Park Turn 10 Studios

Presentation Audio Lead:

Responsibilities: Audio owner for User Experience, Cinematic Presentation, Voiceover, and Accessibility “When you hear ‘someone might want this’ you know you’re about to hear a really bad design decision.”

– Alan Cooper

44 Wyatt

Image modified from www.vecteezy.com

Ted

Image modified from www.vecteezy.com Jasmine

Image modified from www.vecteezy.com Wyatt • Wants long paragraphs of text to be accessible through audio

Jaws Demo - YouTube

Image modified from www.vecteezy.com Wyatt • Wants long paragraphs of text to be accessible through audio

Skyrim Mod: Read Books Aloud - YouTube

Image modified from www.vecteezy.com Ted • Wants all text to be accessible • Wants control over the volume of dialogue and others sounds

Image modified from www.vecteezy.com Pit People

52 Jasmine • Wants to know… • …where enemies are coming and going

Image modified from www.vecteezy.com Jasmine • Wants to know… • …where enemies are coming and going • …when enemies have arrived

Image modified from www.vecteezy.com Jasmine • Wants to know… • …where enemies are coming and going • …when enemies have arrived • …how many enemies remain and how much gold she has

Image modified from www.vecteezy.com Jasmine • Wants to know… • …where enemies are coming and going • …when enemies have arrived • …how many enemies remain and how much gold she has • …how to begin the game quickly without having to memorize too much

Image modified from www.vecteezy.com Jasmine • Wants to know… • …where enemies are coming and going • …when enemies have arrived • …how many enemies remain and how much gold she has • …how to begin the game quickly without having to memorize too much • …how to navigate around the map quickly

Image modified from www.vecteezy.com Jasmine • Wants to know… • …where enemies are coming and going • …when enemies have arrived • …how many enemies remain and how much gold she has • …how to begin the game quickly without having to memorize too much • …how to navigate around the map quickly • …how she can be most immersed in the story Image modified from www.vecteezy.com Overview • Accessibility – not about limiting your creative vision but removing unintentional “designed disabilities” so more people can play • Wearing a blind fold doesn’t make you a blind gamer • Don’t sacrifice accessibility for immersion – immersion is subjective • Accessibility is an iterative, evolving process • Accessibility needs to be backed by real needs, real experiences and real research • Games don’t have to be audio-only to be blind-accessible; allow more than one way to access gameplay information • Consider accessibility of consoles, stores, development tools and your own website • Test with gamers with disabilities… • Individuals like Brandon, Sightless and Jesse • AbleGamers Player Panels • The Accessibility User Research Collective • Disability Rights Organizations Further • Microsoft Guide Dogs • Audio Pizza Podcast by Garth Humphreys considerations • imitone by interactopia LLC • Accessible Realities by Zohar Gan Accessible Tools and • Sable Game Creator by Ebon Sky Studios Non-game Tech: (blind-accessible game engine) • Eagle Island – low vision accessibility • Audiogame Jam every October run by James Kyle • Talk resources can be found at: www.smashclay.com Upcoming blog post for Audiokinetic/MIGS talk

• IllegallySighted YouTube (business cards): www.youtube.com/user/IllegallySighted

• Brandon Cole’s website: www.brandoncole.net

• SightlessKombat’s website/#TranscribingGames: www.sightlesskombat.com www.transcribinggames.sightlesskombat.com

Resources • Tomasz Tworek’s accessible games list: forum.audiogames.net/viewtopic.php?id=10067 Also, follow Ian Hamilton at @_ianhamilton for up-to-date news about game accessibility and • Polygamer interview with Joseph Bein from the CVAA and consider attending the Game Out of Sight Games: Accessibility Conference in London 2019 and www.polygamer.net/2018/01/17/ San Francisco 2020 pg74-joseph-bein/ Formal Guidelines

• Game Accessibility Guidelines: • http://gameaccessibilityguidelines.com/ • AbleGamers APX (Accessible Player Experiences): • https://accessible.games/accessible-player-experiences/ • Audio Description Guidelines: • http://www.acb.org/adp/guidelines.html • http://audiodescription.co.uk/uploads/general/itcguide_sds_audio_desc_word 3.pdf Additional Open Educational Resources (OER)

Look out for more and future OERs from the Game Accessibility Special Interest Group of the International Game Developers Association here: https://igda-gasig.org/oer/ Thank you

Adriane Kuzminski @smashclayaudio [email protected] www.smashclay.com