MAK MESSENGER: INSTANT MAKATON MESSAGING FOR THE HEARING-IMPAIRED

James Ohene-Djan, Robert Zimmer, James Bassett-Cross, Ben Cosh

Goldsmiths College, University of London New Cross, London SE14 6NW Tel: +44 20 7919 7862, Fax: +44 2079197853 Email: [email protected]

Abstract: Increasingly communications-based technologies such as Instant Messaging and chat systems, are being used to enhance traditional learning environments with tools that allow students to formulate and exchange ideas. In this paper we present Mak-Messenger, the first Instant messaging system specifically designed for hearing- impaired people. Mak-Messenger has been designed to assist in the teaching of the Makaton signed language vocabulary. By providing this example of an assistive technology we hope to demonstrate how innovative learning technologies can be tailored to address issues relating to the educational needs of hearing-impaired people and those who wish to communicate with them.

Keywords: Instant messaging and chat systems, Makaton, deaf, hearing-impaired

1. Introduction

Sign languages are combinations of hand and body movements together with facial expressions used to communicate without sound (Sutton-Spence and Woll, 1999). Such languages enable hearing-impaired people to communicate with other people, both hearing-impaired and hearing. These languages, which consist of intricate vocabularies and syntaxes, are the native languages for a large number of hearing-impaired people. In an educational context both spoken and printed information can be translated into a pictorial representations to enable hearing-impaired people to have equal access to it. Such pictorial representations of the vocabulary of sign languages are called sign writing systems (Smith, 1990). Two prominent examples of these are the Makaton vocabulary (Makaton, 2004) and Valerie Suttons’ SignWriting system (Sutton, 2004).

Real-time communications technologies such as Microsoft Messenger (Microsoft, 2004) and Yahoo Messenger (Yahoo Messenger, 2004) are increasingly being used to enable users to communicate and to share information in a non-intrusive manner. As learning technologies such tools may be deployed to allow users to formulate and exchange ideas in workgroups. In this paper we present, to the best of our knowledge, the first real-time, instant messaging system, Mak-Messenger, specifically designed for hearing-impaired people. Mak-Messenger uses the Makaton vocabulary, the most popular sign writing language and vocabulary for people with learning difficulties in the United Kingdom. By demonstrating how innovative technologies can be tailored to address the needs of hearing-impaired people, we hope to raise awareness among software developers and to demonstrate how such needs may be catered for. The research reported in this paper is therefore motivated by the view that, through such tailoring it may be possible to provide students with equal access to educational resources. Mak- Messenger is a fully developed software product available for download (Ohene-Djan, 2004). The software, developed using the Java programming language, has been tested using personal computers running Windows 2000 and XP operating systems.

1 2. Background

In order to understand how Mak-Messenger can benefit those whishing to learn sign language writing systems, it is useful to consider some of the reading, writing and teaching issues associated with the education of hearing-impaired and deaf people. Historically, deaf children were only taught to lip-read and speak (Fischer and Lane, 1993). As a result reading levels of such children are generally of a lower standard than those of their hearing counterparts (Schleper and Mahshie, 1997). In the context of education, both spoken and printed information is translated into sign language, in order to enable students to have equal access to it. A sign language is not a word-by-word translation of a spoken language, but is a language in its own right. Sign languages, such as American Sign Language (ASL) (Sternberg, 1996) and British Sign Language (BSL) (Miles, 1998), differ significantly in their syntax and semantics from American or British English.

The Makaton sign language writing system, derived from BSL, was proposed and developed by Margaret Walker, a speech therapist, in the 1970’s. Originally designed as a communication system for a group of hearing-impaired adults with severe learning difficulties it has since been expanded for a broader use in the community for both children and adults with varying difficulties in conventional communication methods, in particular, those whose hearing is impaired. The Makaton vocabulary has now gained widespread acceptance for use in schools, hospitals and in the home. A set of symbols have been developed to accompany the signs, providing a visual representation of the language, making expressive communication easier for children and adults with physical disabilities that make the formation of signs difficult. At present the Makaton Vocabulary is the most widely used sign writing system within the United Kingdom.

There are several significant differences between the Makaton and BSL. Makaton was primarily developed to assist hearing people with learning and communication difficulties. It was devised to be an aid to communication rather than a complete language in its self, as is the case with BSL. BSL is a complex language with a well-defined grammar. Furthermore, When developing Makaton it was assumed that those who would use it may have the ability to use speech alongside signing. As with spoken languages, there are regional variations of dialect and accent, there are also variations in the signs that are used throughout the UK. The signs from BSL matched to Makaton have been standardised to those used in the South East/ London region. This standardisation is to avoid confusion for Makaton users if they move around the UK.

It is widely acknowledged that Information Communication Technology (ICT) when used to supplement traditional teaching methods has a positive impact of the educational development of children (Passey et, al, 2003). Allowing children to implement their knowledge in an interactive manner through the use of computers can help to increase their motivation for learning and give them pride in their personal development. It is our belief that if the Mak-Messenger system were used in parallel to the traditional method for teaching Makaton, hearing-impaired people would grasp the concepts of the vocabulary quicker than without its use, and would be more likely to embrace the language.

3. The Mak-Messenger Instant Messaging Application

Mak-Messenger is an Instant Messaging application specifically designed for the teaching of the Makatron sign writing vocabulary through means of interactive communication. It allows users to form a connection with a remote server, and then post messages to the server which are then broadcast to all users currently connected, in a way akin to an Internet chat room. The messages take the form of a series of Makaton signs, which are inserted by clicking buttons and are displayed as complete messages. Although Mak-Messenger incorporates a clear, simple user interface with buttons to insert symbols. The decision to keep the interface simple was taken so as to make the system as user friendly and straightforward as possible.

There are many forms of sign writing vocabularies in different countries. The Mak-Messenger has therefore been designed to accommodate a wide variety of pictorial representations of sign languages. In order to achieve this, the system’s display interface is dynamically generated. Additional sign writing vocabularies such Valerie Sutton’s SignWriting symbols (Sutton, 2004) can easily be added to the system by placing them in the appropriate directory.

2 3.1. Architectural Framework of Mak-Messenger

Figure 1 depicts the chat system framework, upon which Mak-Messenger has been designed and implemented. The intricate network establishment and communications between the server and its clients are handled by the Java networking package (java.net) (Hughes et, al, 1999). Broadly speaking, once the server is initialised it makes a ServerSocket on a specific port which waits for connection requests from clients. The client knows the IP address and port number where the ServerSocket resides. This is often called the handshake point.

Figure 1. Mak Messenger system framework

In order to make a connection with the server, the client initialises its own socket. Once a connection is established, the ServerSocket returns a new socket on a different port to which the client socket is now bound, thus freeing the ServerSocket to listen for further connection requests. As this is a typical connection-oriented service the protocol used for transmission is TCP (Transmission Control Protocol). Information or rather messages are sent between the server and client via OutputStream and InputStream objects. An OutputStream sends the information from the server to the client and an InputStream from client to server.

3.2. A Mak-Messenger Session

Figure 2 shows the opening screen of the Mak-Messenger application.

Figure 2. Mak-Messenger main screen Figure 3. Mak-Messenger login prompt

At this point the user is presented with two options; Connect and Exit. In order to participate in a conversation the user must select the connect option. This takes the user through the login process where they can enter their

3 username and a socket connection is established with the server as described above. Figure 3 displays the Mak- Messenger login prompt. Once a successful connection has been established the user is presented with the chat terminal from where they participate in conversations. Figure 4 shows the chat terminal from where the user creates and views messages.

Figure 4. Mak-Messenger chat terminal

Messages are typed by selecting signs from the button panel. When a message is typed it is displayed in the preview pane, which allows the user the chance to review and change their message before transmitting it to the server. Figure 5 shows how a message is constructed and previewed before submission.

Figure 5. Message construction

Once received by the server the message is broadcast to all connected clients who can then see the message in the message area of their terminal. Figure 6 shows how a message is displayed on two client terminals. Once a user decides to disconnect they select the disconnect option from the options menu which ends their session with the server.

4 Figure 6. Client terminals displaying received messages

4. Related Work

This section briefly compares the work reported here with that of other sign language learning resources. It is in no way exhaustive, but aims to give the reader a sense of where future technological developments may lie. The vast majority of learning materials for hearing-impaired people are paper-based or videotaped (Allsop and Mason, 2002). Work is also been conducted into the development of WWW-based learning materials (Gaertner, 2003). Although these materials are valuable educational resources, they offer relatively low levels of support for interactive learning between students. At present there are two broad categories of sign language technology, documentation systems such sign writing systems and word processors, and signing avatars to translate text into 2D and 3D animation, and streaming video systems used to translate written texts into sign language video presentations.

Valerie Sutton (Sutton, 2004), a leading innovator in sign documentation systems developed the first dance writing system used to document dance movement. Since 1979 she has developed the Valerie Sutton SignWriting system and word processor to document ASL. This system is a extremely versatile application which is now used worldwide. Research in the area of sign writing systems is now being directed towards providing software-based sign language translators such as the SignWriting Markup Language, SWML (Da Rocha Costa, 2003). These translators allow for the automatic generation of electronic texts written in sign language.

The KSO (Ohene-Djan et, al, 2003) is a 2D E-Book for Video-based Sign Language Education. KOS utilises digital video clips to represent sign language in the digital domain. KOS uses digitised video to show signing tutorials and presentations by hearing-impaired children to hearing-impaired children. However, the signing tutorials are presented via video, they inherit the inherent weaknesses of the medium. Video being a uni- directional medium does not enable two-way communication between users, whether that is a teacher and a group of students, or a work group comprised of hearing-impaired and hearing students. Recent research efforts have aimed to incorporate 3D animations of signers in contrast to video. Prominent examples of such systems are Vsign (Vsign, 2003) and VisiCast (Visicast, 2003). The Vsign allows users to build animations from a model of a human. Users may specify the complete range of body movements used in sign language. VisiCast is a project to develop a 3D model of a signing person to accompany subtitles and teletext information on digital television. Signs are digitally captured via advanced technologies, including a body suit and digital helmet.

5. Conclusions

Although Real-time communications technologies such as Microsoft Messenger and Yahoo Messenger are widely used within academic settings to enable students to formulate and exchange ideas in a non-intrusive

5 manner, scant research has been conducted into providing such facilities for the education of hearing-impaired people. The Mak-Messenger instant messaging system presented in this paper will enable the beginnings of such research to take place. The Mak-Messenger instant messaging system is currently being researched in educational settings to evaluate its usefulness in assisting the teaching of sign language. We believe, that for hearing-impaired people to have quality education throughout their lives information available on, and the tools used to communicate and learn via, the Internet must be tailored to address their accessibility needs. We aim, through future development, to test the effectiveness of Mak-Messenger in learning contexts.

References Allsop, L. and Mason, C., (2002) A two-hour introduction to British Sign Language presented by Lorna Allsop and Clive Mason, British Deaf Association Video Service, Classification BSL9 D Miles. (1998). British Sign Language: A Beginner's Guide. BBC Consumer Publishing (Books); ISBN: 0563211342. Da Rocha Costa, A. C., (2003), SWML SignWriting Markup Language http://swml.ucpel.tche.br/ Fischer, R. and Lane, H., (1993). Looking Back: A Reader on the History of Deaf Communities and Their Sign Languages (International Studies on Sign Language and Communication of the). ISBN: 3927731323. Gaertner, S., (2003), ASL Fingerspelling website, where.com/scott.net/asl/ Hughes, M., Shoffner, M and Hamner D (1999). Java Network Programming, 2nd edition Manning Publications Co ISBN 188477749X. Makaton (2004). Makaton Vocabulary Development Project http://www.makaton.org. Microsoft (2004) Microsoft (MSN) Messenger http://www.msnmessenger-download.com. Ohene-Djan (2004) The Mak- Messenger system is available at http://www.assist-tech.gold.ac.uk Ohene-Djan, J., Zimmer, R., Gorle, M., & Naqvi, S. (2003). A Personalisable Electronic Book for Video-based Sign Language Education. Educational Technology & Society, 6 (4), 86-99, Available at http://ifets.ieee.org/periodical/6_4/9.pdf Passey, D., Rogers, C., Machell, J., McHugh, G and Allaway D, (2003). The Motivational Effect of ICT on Pupils Emerging Findings1, ISBN 1 84478 136 4 DfES/0794/2003. Schleper, D. R. and Mahshie, S. N., (1997), Reading to Deaf Children: Learning from Deaf Adults (A Manual for Parents and Teachers). ISBN: 0880952121 Smith (1990) C Smith, Signs Make Sense, ISBN: 0285650831. Sternberg, M. L. (1996) Essential ASL : The Fun, Fast, and Simple Way to Learn American Sign Language. ISBN: 0062734288. Sutton (2004). Sutton, V. http://www.signwriting.org/ (2004) Sutton(1999). R Sutton-Spence & B Woll. The Linguistics of British Sign Language: An Introduction. ISBN: 0521631424. Visicast, (2003) Visicast available at http://www.visicast.co.uk Vsign, (2003) Vsign available at http://www.vsign.nl/ Yahoo Messenger (2004) Yahoo Messenger http://messenger.yahoo.com

6