Accessible Software
- Enable users with disabilities to use your software with their preferred workflow - Leverage existing accessible technologies - Support OS accessibility clients - VoiceOver (macOS), Narrator (Windows) Accessible Software
- Names and descriptions for controls - Programmatic access to UI elements and text - UI keyboard navigation - Hierarchical UI model - Indicate keyboard focus - Client notifications - Focus change - Windows opening/closing - Control value change Anatomy of a JUCE App juce::JUCEApplicationBase juce::ComponentPeer juce::TopLevelWindow MainComponent juce::Label juce::TextButton NSApplication NSWindow NSView
Why?
- juce::ComponentPeer manages the native window (HWND, NSView, Window, etc.) - juce::Components rendered within the context of the peer - not backed by native UI elements - Consistency - Code re-use Why?
- OS and accessibility clients only “see” the top-level peer - Need to inform them about the rest of the UI
Accessibility Client Why?
- OS and accessibility clients only “see” the top-level peer - Need to inform them about the rest of the UI
Native APIs
Accessibility Client Apple
- NSAccessibility protocol (Objective-C)
- Accessible UI elements must adopt this protocol - Many AppKit controls adopt NSAccessibility and provide default implementations for all properties and methods (NSView, NSButton, NSSlider, etc.) - Custom controls can adopt a subset - NSAccessibilityButton, NSAccessibilitySlider, NSAccessibilityGroup, ... Apple
@protocol NSAccessibilitySlider
@required
- (nullable NSString *)accessibilityLabel;
- (nullable id)accessibilityValue;
- (BOOL)accessibilityPerformIncrement;
- (BOOL)accessibilityPerformDecrement;
@end Apple
- Assumes custom control is backed by an NSView - Direct NSView subclass - Existing AppKit control subclass - Non-view backed controls need to subclass NSAccessibilityElement class Apple
@interface NSAccessibilityElement : NSObject
- Represents a control not backed by an NSView in the accessibility hierarchy - Subclass, adopt a role, and add it to the root NSView via accessibilityChildren property - Implement NSAccessibility properties and methods required for your control Apple
NSAccessibilityButtonRole
NSAccessibilityStaticTextRole accessibilityParent accessibilityFrame accessibilityParent accessibilityTitle accessibilityFrame accessibilityValue accessibilityTitle isAccessibilityFocused accessibilityValue setAccessibilityFocused: accessibilityPerformPress: Apple
- NSAccessibilityPostNotification() - Sends notification to any observing assistive applications (not via standard NSNotificationCenter)
- NSAccessibilityCreatedNotification - NSAccessibilityFocusedUIElementChangedNotification - NSAccessibilityWindowCreatedNotification - NSAccessibilityValueChangedNotification - ... Microsoft
- UI Automation (UIA) framework
- COM interface - Supported by standard Win32 and XAML controls Microsoft
- IRawElementProviderSimple - Host window, control pattern type and control properties - IRawElementProviderFragment - Bounds and navigation to other fragment elements - IRawElementProviderFragmentRoot - Fragment retrieval - focused, at position Microsoft
HWND
WM_GETOBJECT
IRawElementProviderFragmentRoot IRawElementProviderSimple
IRawElementProviderFragment IRawElementProviderFragment IRawElementProviderSimple IRawElementProviderSimple Microsoft
- Control patterns
- Enable accessibility client to manipulate and query information on a control type - Providers subclass and implement control patterns for specific controls: - IWindowProvider, IInvokeProvider, ITextProvider, ... - Returned via IRawElementProviderSimple::GetPatternProvider() Microsoft
- Control properties
- Enable accessibility client to retrieve information about a control such as name, type, bounds etc. - IRawElementProviderSimple::GetPropertyValue() called with a PROPERTYID: - UIA_NamePropertyId, UIA_ControlTypePropertyId, UIA_IsKeyboardFocusablePropertyId, UIA_HasKeyboardFocusPropertyId, UIA_IsEnabledPropertyId, ... - Returns a valid value if the property is supported by the control Microsoft
UIA_ButtonControlTypeId
UIA_ValuePatternId UIA_TextControlTypeId UIA_InvokePatternId
UIA_ValuePatternId UIA_NamePropertyId UIA_HasKeyboardFocusPropertyId UIA_NamePropertyId Navigate() Navigate() get_BoundingRectangle() get_BoundingRectangle() SetFocus() Microsoft
- UiaRaiseAutomationEvent() - Notifies any subscribed listeners
- UIA_AutomationFocusChangedEventId - UIA_Window_WindowOpenedEventId - UIA_Window_WindowClosedEventId - UIA_AutomationPropertyChangedEventId - UIA_Text_TextChangedEventId - ...
JUCE
- Unified API for exposing required information and control for JUCE UI elements - Support all standard JUCE widgets - Custom handlers for your own components - API for posting accessibility client notifications JUCE
- Already contains a lot of the required functionality - juce::Component tree navigation - JUCE <-> native screen bounds conversion - UI programmatic control e.g. juce::Button::triggerClick() - Keyboard focus traversal - juce::Component state - focus, enablement, visibility
- Bridge JUCE code and native libraries JUCE
- NSAccessibility - AppKit framework (Objective-C) - JUCE Obj-C++ wrapper
- UI Automation - Win32 COM interface - juce::ComBaseClassHelper - handles COM ref-counting, QueryInterface() etc. JUCE
- juce::AccessibilityHandler class - Platform-independent roles - Properties - State - Control values - Control actions - Navigable text support - Table/cell support - Native implementation under the hood JUCE
juce::AccessibilityHandler
NSAccessibility protocol UI Automation
NSAccessibilityElement IRawElementProvider
VoiceOver Narrator JUCE
juce::AccessibilityHandler
NSAccessibility protocol UI Automation
NSAccessibilityElement IRawElementProvider ?
VoiceOver Narrator Further Work
- Support mobile platforms, Linux - Direct access to screen reader (notifications, announcements) - Windows 10 native menu bar support - Desktop push notification support - Mobile haptics support - System colour setting access - high contrast, inverted - Native scroll behaviour Questions?
- https://forum.juce.com/ - [email protected] - Discord - Ed Davies #4745 - JUCE Q&A - 19/11/2020 20:10 UTC, Track C