Brlapi: Simple, Portable, Concurrent, Application-Level Control of Braille Terminals Samuel Thibault, Sébastien Hinderer

BrlAPI: Simple, Portable, Concurrent, Application-level Control of Braille Terminals Samuel Thibault, Sébastien Hinderer

To cite this version:

Samuel Thibault, Sébastien Hinderer. BrlAPI: Simple, Portable, Concurrent, Application-level Con- trol of Braille Terminals. The First International Conference on Information and Communication Technology and Accessibility - ICTA 2007, Apr 2007, Hammamet, Tunisia. pp.27–31. inria-00135946

HAL Id: inria-00135946 https://hal.inria.fr/inria-00135946 Submitted on 9 Mar 2007

HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est archive for the deposit and dissemination of sci- destinée au dépôt et à la diffusion de documents entific research documents, whether they are pub- scientifiques de niveau recherche, publiés ou non, lished or not. The documents may come from émanant des établissements d’enseignement et de teaching and research institutions in France or recherche français ou étrangers, des laboratoires abroad, or from public or private research centers. publics ou privés. inria-00135946, version 1 - 9 Mar 2007 auatrrt rvd ohabaletria n a a and for terminal usual braille then a was both It provide to terminal. manufacturer with braille work of only brand could one difficult readers so screen recently was until display that braille of brands communicate different to with able braille readers refreshable screen a Designing or display. screen system the synthesis speech on a displayed using readers information screen the called deliver programs which special of help the with Introduction 1. Br- of included. implementation is used lAPI widely device. a the of to description The access about gets choices eventually application sensible which con- making device BrlAPI braille with one appli- currently, several device, with how communicate each show We can of necessary. cations specificities if so the do with can do but deal They to devices. need braille with not communicate to use may not is devices braille task. of trivial a panel wide a ap- access and readers plications screen output both to allowing however, ability ; the braille need then would users. much applications impaired be Such visually to would adapted braille it use specially own to cases, their feedback, provide easy such applications let not In to is useful more sighted view device. this braille as cases, a information some on in same But the users. them providing environments, by computer access to users vi- impaired allowing sually for devices braille drive can readers Screen e:+354 03 0Fx 3 00 669 66 00 40 5 +33 Fax: 40 35 00 40 5 +33 Tel: sal,vsal mardpol s computers use people impaired visually Usually, applications that layer abstraction an present We [email protected] 5 or el Lib la de cours 351 30 aec Cedex Talence 33405 aulThibault Samuel Abstract plcto-ee oto fBaleTerminals Braille of Control Application-level France LaBRI rAI ipe otbe Concurrent, Portable, Simple, BrlAPI: eration ´ e:+338 92 5Fx 3 32 319 83 27 83 3 +33 Fax: 35 20 59 83 3 +33 Tel: ie oecnldn eak n it e remain- 6 few questions. Section a open lists and ing and 5 BrlAPI, remarks Section concluding of some Finally, clients gives current 4. the Section presents in this de- presented of 3 implementation is Section an solution solution; display. BrlAPI braille the to scribes specific are that lems com- to programs device. such the for with either, directly way municate neglected a not offers are BrlAPI tools since transfer appli- file Device-specific like device. cations appli- braille several the lets share and commu- cations terminal to braille how each exactly with know nicate to applications the removes for both need that server client- a a to res- thanks proposes terminals the braille with it communicate towards applications Indeed, lets step that first approach server problem. a this as of seen olution be can paper this exclusively readers. used screen being by are there these with that since directly fact terminals, communicate braille the to in applications for lies way problem no this is of part screen-based One standard one. the to addition more alternative, in an interface provide suited mech- to standard application no an inter- is for same there anism since the people, exactly sighted use as im- to face visually forced are unsolved: people remains paired that problem deeper a terminals. braille of brands read- several screen recent support most ers the Only it. using reader screen nScin2w ics ngetrdti h prob- the detail greater in discuss we 2 Section In in introduced be will that framework BrlAPI The is there readers, newer these with even However, [email protected] 1 u uJri Botanique Jardin du rue 615 40 Villers-l 54600 Sébastien Hinderer LORIA France es-Nancy ` 2. Issues for Driving Braille Devices events be described in a more standard way, in spite of their potentially different nature ? The question of accessing a braille display within an application is not trivial, mainly because of the large 2.2. Concurrency variety of available models, and because of the problem of properly sharing one display between several appli- Another issue is that modern operating systems let cations. Protecting the access to braille displays must several applications run concurrently, their visual out- also be considered. put being usually sent to separate virtual terminals or windows. Some of these applications may want to use 2.1. Heterogeneity of Braille Devices the braille device for providing more appropriate braille output. A visually impaired user may also want to use Every braille terminal has a refreshable braille dis- several screen readers simultaneously, in order to get play divided into eight pin cells; however, the number of the best benefits from each of them. This results in a available cells and their layout vary from display to dis- concurrency issue: what should eventually be displayed play. Some terminals have a unique display, others have on the braille terminal? To which application should the two (the main one, generally used to display a small re- key events be directed? gion of the screen, and an auxiliary one, usually used to Moreover, one may want to write some applica- report status information like cursor position or current tions dedicated to a given brand of terminals, that would time). Some modern devices also let users adjust the take full control of the braille device. For instance, firmness of braille dots to either display hard and easy- note-takers can often exchange files with the computer, to-read dots when the display is connected to a power through a brand-specific protocol. So as to avoid cor- source, or softer dots when the display is unplugged, to ruption during the transfer, all other device operations save energy. should be suspended, but the applications that requested Perhaps more importantly, braille terminals come them should not have to be aware of that. with very different keyboards. Some terminals which are also used as note-takers include a keyboard to write 2.3. Protection characters (either in braille or through a more standard PC-like keyboard). In addition to these optional Usually, input/output devices such as keyboards keyboards, braille terminals have function keys whose and screens are controlled by the operating system’s number and layout is, again, terminal dependent. Man- kernel. Each user process has to interact with the kernel ufacturers of braille terminals have shown a rather vivid to perform input/output. Therefore, it is unlikely that a imagination regarding the kind of available function user process can access these devices in a way which keys: simple keys, keys associated with braille cells would render them unusable. This, however, is not true (usually called routing keys because one of their func- for braille devices. Indeed, they are generally handled tions is to bring the cursor to the character displayed by by a user-mode process, since this makes development their associated cell), joysticks, navigation bars that can and distribution easier. As a consequence, braille ter- be moved one or two steps in each of four directions, minals are much more vulnerable than more traditional wheels, etc. To sum up, braille keyboards can be much input/output devices to incorrect use by other processes. more complex than plain PC keyboards. In fact, they Given that incorrect use can damage a braille terminal often provide the functionality available both through a irreversibly, and that these devices are extremely expen- keyboard and a mouse. sive, it is important to ensure good protection of this re- Last but not least, there is no standard way to com- source against incorrect usage or malicious intent. This municate with a braille display. Each manufacturer need for good protection becomes even more critical in has designed its own communication protocol to let its the context of modern multi-user networked operating braille devices receive text to display or configuration systems. information and to send keyboard events or report status information. 3. Proposed Solution Hence, the first problem one encounters is that braille terminals form a completely heterogeneous class For tackling all the issues described in the previ- of devices. How should displays be accessed? How ous Section, we propose the BrlAPI framework, based should keyboard events be delivered? Should they be on a client-server approach. A BrlAPI server uses a rather driver specific (each application then has to inter- driver for controlling the braille terminal, and appli- pret them in a consistent manner), or should keyboard cations connect to the server for interacting with the braille device. This even allows an application running that its output is actually sent to the braille device, and on one machine to drive a braille display connected to braille key events are sent back to it. Output of each another machine, provided that the two machines are other application is stored and will be displayed later, interconnected by a network link. when they get the focus again. What “appropriate focus” may mean is not com- 3.1. Handling the Variety of Terminals pletely clear. One natural way of switching between applications is to just follow the already well-known key- The BrlAPI server has a series of drivers, each board and mouse focus. This is necessary when using of them knowing how to communicate with one given braille terminals which do not have an integrated key- family of braille devices. More precisely, the server board, since the regular keyboard then has to be used. is able to send text to display, and to receive key Nevertheless, even when using a braille terminal with events. Optionally, a driver can also provide lower- an integrated keyboard, it is a lot easier for a visually level, packet-based input/output functionality. In prac- impaired person to work with a sighted person if the tice, one running instance of the BrlAPI server commu- notion of focus is the same for both of them. It should nicates with one braille device. be noted that this is different from the situation of dis- When clients connect to a server, they can request patching speech synthesis [6]. The reason for this is that the dimensions of the corresponding display, and then speech naturally expresses time, while braille naturally perform write requests. expresses space. Moreover, clients can query the server for the name The solution we adopted is hence to require appli- of the braille device it is communicating with. This cations, before issuing write requests or starting to wait permits a client to decide how it wants to receive key for key events, to declare “where” they are running, this events. Key events can be delivered to a client applica- being expressed as an integer, and the BrlAPI server just tion in two different ways. needs to find out where the keyboard and mouse focus First, the server can deliver key events it gets from is. In the case of the Linux console for instance, this the device as-is. This presupposes that client applica- is the number of the Virtual Terminal (VT) in which tions know exactly how key events are returned by the the application is running, and the BrlAPI server just braille driver they are talking to, and lets developers use asks the kernel which VT is active. In the case of an X- the knowledge of some given keyboard when designing Window desktop, this is the ID of the window in which their applications. the application is running, and a dedicated X-window Second, key events can be converted to a standard application called xbrlapi sends to the BrlAPI server representation by the server before being delivered to a the ID of the active window. The user can hence just client. For instance, braille devices often have keys typ- switch between consoles and windows as usual, and the ically used for browsing the current line, going to the braille device will always display the appropriate appli- next or previous line, etc. Clients can then be written cation’s output. independently of the braille device, their use remain- The notion of focus is even nested by having ap- ing easy and intuitive. This also favours a semantically plications actually send a list of integers: an application consistent use of keys from one client application to an- running on VT 2 will send a 1-element list just holding other. 2, while an application running in a window of an X server that is running on VT 7 will actually send a 2- 3.2. Selecting the Proper Application element list holding 7 and the window ID. The BrlAPI server just has to first find out which VT is active, and Previous work like Libbraille [12] achieved results if this is VT 7, find out which windowof the X server is similar to those described in Section 3.1. However, the active. context of Libbraille was the TIM project [2], in which Finally, as was mentioned in Section 2.2, some spe- only one application needed to be run at a time. As a cialized applications like file transfer tools may have to result, Libbraille was intrinsically not designed to allow take full control of the braille device. With our client- several applications to control the braille terminal. server approach, we check that only one application at a In contrast, thanks to the client-server design, sev- time requests such control, and disable all braille driver eral applications can connect to the BrlAPI server. Each handling except the basic low-level device protocol op- application behaves like if it were the only one that has erations like splitting data coming from the device into access to the braille device, but the BrlAPI server actu- packets (if the device protocol is packet-oriented). The ally switches between applications as appropriate. Only application can then send raw data to the braille driver, one application at a time has the “focus”, which means which in turn sends it to the device. Symmetrically, data that the braille driver receives from the device is passed 4. Implementation back to the application. This way, applications can easily implement file transfer protocols for instance. BRLTTY [9] is a portable screen reader: it works Furthermore, applications can even switch to the on Linux, all BSD flavors (including MacOS X), So- “suspended mode”, in which the BrlAPI server keeps laris, OSF, HP-UX, QNX, Windows, Hurd, DOS. It its braille driver shut down, so that the application may also supports a wide range of braille terminals (around open the device itself and thus be free to totally control 50 models). The BrlAPI framework was hence im- it. plemented within BRLTTY. A BrlAPI server is im- The XVI [4] client-server design is similar to ours, plemented within the screen reader in an independent but though it permits several clients to connect concur- thread, and a library is provided so that applications can rently, it does not perform an automatic switching be- easily act as BrlAPI clients. Bindings for this library tween them. It also does not allow applications to take have been written for Python, Java, TCL and Com- full control of the braille device. mon lisp. Bindings for some other languages (Objective Caml, Perl and Haskell) are currently in development. 3.3. A Simple Interface for Programmers When both clients and server are running on the same machine, either a Unix local socket or a Win- In order to achieve the various actions explained dows named pipe is used to establish the connection. above, applications actually switch between the follow- In some cases (just like with X-window applications), ing modes. clients may have to run on another machine. In this Tty mode: The application may issue write requests case, a TCP/IP connection is used instead. A range of for displaying text on the braille display, and may ports has been reserved at IANA [5] for this purpose. In receive key events. order to reduce the risks inherent in text parsing and to keep the server simple, we use a binary protocol over Raw mode: The application directly communicates this connection. No encryption is performed, since just with the braille device through the BrlAPI connec- like X, it would be easy to encryptthe connectionsusing tion. external tools like ssh or IP-layer encryption. Suspended mode: The BrlAPI server keeps its device Finally, the communication protocol has been de- driver shut down, letting the application directly signed to minimize the number of packets to exchange open the device itself. between client and server in situations where perfor- mance matters. For instance, write requests emitted by For instance, the following simplified code snippet a client require no acknowledgement from the server. If connects to the server, writes a prompt and waits for a the write fails for some reason (e.g. inconsistent data key: are sent), an error will be reported asynchronously. brlapi_openConnection(); brlapi_enterTtyMode(); 5. BrlAPI Clients brlapi_writeText("Press any key"); brlapi_readKey(&code); The BrlAPI framework is already widely used brlapi_leaveTtyMode(); on Linux systems as a means for screen readers to brlapi_closeConnection(); drive braille devices. This includes X-window readers like Gnopernicus from Baum [3], Orca from Sun [11] 3.4. Protection and LSR from IBM [8], as well as the Emacs reader speechd-el [7]. This permits not only to avoid having to BrlAPI includes an authorisation mechanism used re-implement device drivers, but also to cooperate har- by the server to decide whether a client is allowed to moniously with the Linux text console: BrlAPI allows connect or not. The precise authorisation procedures switching between the BRLTTY reading of Linux text are left unspecified, so that various authorisation proce- consoles and the reading of X or Emacs Windows. The dures can be implemented according to what the operat- gnome-braille [10] braille translation library also uses ing system supports (SO_PEERCRED, getpeereid, BrlAPI as one of its output devices. etc.). When a client connects, the server advertises the Some file transfer applications have been devel- list of supported authorisation mechanisms. The client opedand are nowusedon a daily basis: vbtp for Visio- uses one of them, and if the authorisation procedure Braille devices, trf for EuroBraille devices, and soon succeeds, the connection is accepted. Otherwise, the htcom for HandyTech devices. client may try a different mechanism. Also, the bless client is a little braille-enabled replacement for the well-known less Unix command. It would also be useful to write a detachable mul- Currently, it lets visually impaired persons read docu- tiplexer. It would for instance be run along with the ments without having to scroll the screen, and allows screen program and allow the running of applications them to save their current position in a document in or- in a “detached” session and let the user reattach the ses- der to come back to it quickly later. sion, including the BrlAPI applications. Finally, it is worth mentioning that the Accessibil- Furthermore, we would like to let applications con- ity Free Standards Group [1] is even considering adopt- figure and request the properties of the BrlAPI server. ing BrlAPI as its standard braille I/O sharing compo- These would of course include the name of the currently nent. loaded braille driver, the size of its braille display and the current braille table (which defines the mapping be- 6. Conclusion tween characters and their braille representations), but also the cursor blinking rate and shape, the braille con- The large diversity of braille terminals makes it es- traction style, etc. pecially difficult to design a generic communication in- Yet another capability we may add to BrlAPI is terface which also allows the use of advanced capabil- support for graphical braille displays. Indeed, in Sec- ities provided by each terminal. Allowing several ap- tion 2.1, we asserted that braille displays are divided plications to access a braille display concurrently, and into eight-pin cells. However, this is not always true in a way that protects the display from incorrect use as some terminals with a matrix of pins have been de- are two other important problems. BrlAPI solves these signed, their goal being to display figures and graphical problems by relying on a client-server approach: each information. Although this kind of terminal is rare, the application connects to a server, which is responsible situation may change in the future, hence the need for for abstracting away the details of each device and de- BrlAPI to support them too. cides which application should, at any moment, be al- Eventually, we would like to integrate the BrlAPI lowed to communicate with the device. That way, the protocol directly into X and TTY streams. That would end user doesn’t have to care at all: she can just switch solve a lot of focus and authorisation issues. between consoles and windows as usual, the braille device will always display the appropriate information. An implementation of the BrlAPI server has been inte- References grated to the BRLTTY screen reader, and a library has been provided to help client applications communicate [1] Free Standards Group – Accessibility Workgroup. with braille devices through BrlAPI. Currently, BrlAPI http://www.a11y.org/. is used by several screen readers for graphical environ- [2] Dominique Archambault, Dominique Burger, and ments. Sébastien Sablé. The TIM project: Tactile interactive It has been suggested that instead of using a client- multimedia computer games for blind and visually im- server approach, we could instead define a TTY ter- paired children. In AAATE 2001, Ljubljana, September minfo entry, and applicationswould not have to be mod- 2001. http://www.baum.ro/ ified: they would simply notice that they are running [3] Baum. Gnopernicus. gnopernicus.html. in a very small terminal (compared to the usual 80x25 [4] BEAM. XVI. http://portal.beam.ltd.uk/ ones). However, since applications have only one stan- xvil/sbserver_api.html. dard output, that would also mean that sighted users [5] Internet Corporation for Assigned Names and Numbers would be restricted to this output. (ICANN). Internet assigned numbers authority (IANA). Although we are fully convinced that BrlAPI is a http://www.iana.org/assignments/ mature project, we also think that a number of things port-numbers. remain to be done. For a start, many users and devel- [6] Free(b)soft. Speech Dispatcher. http://www. opers would like to be able to spatially share a braille freebsoft.org/speechd. window. Instead of using the whole display as is the [7] Free(b)soft. Speechd-el. http://www. freebsoft.org/speechd-el case for now, applications would be given the control of . [8] IBM. Linux Screen Reader. http: only one region of the display, leaving the other regions //www.alphaworks.ibm.com/tech/lsr, available for other applications. This would not only be http://live.gnome.org/LSR. interesting for multi-line displays; it would also be use- [9] Dave Mielke. BRLTTY. http://mielke.cc/ ful on long single-line displays, to reserve the rightmost brltty/. part for displaying the current time on those braille ter- [10] The GNOME Project. Gnome Braille. http://cvs. minals that do not include status cells. gnome.org/viewcvs/gnome-braille/. [11] The GNOME Project. Orca. http://live.gnome. org/Orca. [12] Sébastien Sablé and Dominique Archambault. Lib- braille: A portable library to easily access braille displays. In International Conference on Computers Help- ing People with Special Needs (ICCHP), Linz, Austria, July 2002. Springer.