Adapting the Portuguese System to Formal Semantics

Luís Filipe Cunha Faculdade de Letras da Universidade do Porto / Centro de Linguística da Universidade do Porto (Unidade FCT PEst-OE/LIN/UI0022/2011) Via Panorâmica, /, 4150-564, Porto, Portugal luisfi[email protected]

Abstract account for various aspects of meaning in nat- Since the seminal work of Richard Montague in the ural language. Although the initial task of for- 1970s, mathematical and logic tools have successfully mal semantics was to describe and resolve am- been used to model several aspects of the meaning of biguities regarding quantification in the nomi- natural language. However, visually impaired people continue to face serious difficulties in getting full ac- nal domain, it was rapidly expanded to cover cess to this important instrument. Our paper aims to other areas of interest to semanticists, from present a work in progress whose main goal is to pro- tense and aspect relations (.., Dowty, 1979; vide blind students and researchers with an adequate Bach, 1986; Moens, 1987; Parsons, 1990) method to deal with the different resources that are used to complex modal descriptions involving the in formal semantics. In particular, we intend to adapt the Portuguese Braille system in order to accommodate the interaction of time, situations, and possible most common symbols and formulas used in this kind worlds (cf. Portner, 2009 and the papers col- of approach and to develop pedagogical procedures to lected in Kratzer, 2012, for example). facilitate its learnability. By making this formalisation compatible with the Braille coding (either traditional The adoption of formal outlines in the context and electronic), we hope to help blind people to learn of the semantic descriptions for natural lan- and use this notation, essential to acquire a better under- guages has the advantage of giving rise to un- standing of a great number of semantic properties dis- ambiguous representations for the meaning of played by natural language. sentences, which facilitate the confirmation or rejection of the suggested hypotheses and pos- 1 Introduction tulates. In their attempt to describe and understand the To illustrate this point, let us consider a sim- multiple aspects that underlie the construction ple example, classically discussed in the liter- of sentential meaning in natural languages, ature. A sentence like “Every student read a philosophers and semanticists explored dif- book” is clearly ambiguous between two in- ferent conceptions and methodologies, which terpretations: it may mean that every student gave rise to a large number of theoretical ap- read a (non-specific) book, whatever it is; or it proaches. may mean that there is a (specific) book that A fruitful and promising way to tackle some every student read. The formal representations of the most complex semantic problems, based in (1) and (2) unambiguously spell out these on a formal notation, was initiated by Richard two readings: Montague in the decade of 1970 (see e.g., Montague, 1974). Formal approaches to se- mantics use logical and mathematical tools to 1) ∀ , student(x) → ∃ , book(y) & read(x, y)

7

Proceedings of the First Workshop on Language Technology for Equality, Diversity and Inclusion, pages 7–14 April 19, 2021. ©2020 Association for Computational Linguistics (For every x, if x is a student, then there is a y The variables that may be introduced in a for- which is a book so that x read y) mula correspond to entities of distinct kinds: they can be individuals, intervals of time, situ- 2) ∃ y, book(y) & ∀ x, student(x) → read(x, y) ations or (possible) worlds. Besides the pred- (There is a y which is a book and for every x, if x is a student, then x read y) icative relations, entities may interact with a range of operators that include quantifiers, re- lations between elements and sets of elements book) or between two sets of elements, or even prece- dence or overlapping relations in the temporal In other words, in the representation in (1), the domain. universal quantifier has wide scope over the ex- Despite its undeniable relevance regarding the istential one, which leads to the multiple book semantic studies on recent years, no attempt reading. Conversely, in (2), it is the existential has been done, at least to our knowledge, to quantifier that has wide scope over the univer- provide visually impaired people with ade- sal one, giving rise to the unique book reading. quate tools that enable them to learn and man- On the other hand, formal representations have age this kind of formalisation. been able to transcend the domain of simple The main goal of our project is, thus, to ex- sentences. In fact, Kamp & Reyle’s (1993) plore some ways that could make it possible to Discourse Representation Theory (DRT) ex- blind students and researchers to interact with tends the semantic formalisation to more com- the most common and widespread representa- plex discourses and Asher & Lascarides’ tions in formal semantics. To do so, we will (2003) Segmented Discourse Representation make use of the immense potential that is of- Theory (SDRT) even deals with pragmatic- fered to us by the Braille system. based notions such as those underlying Rhetor- 2 The Braille system: some preliminary re- ical Relations. marks Today, formal semantics supports a significant Braille is the most widespread and efficient number of theories that describe meaning in means for blind people to read and write texts. natural language. In fact, formal approaches It was created in France by in cover almost all areas that traditionally are as- 1824 in order to provide a tactile representa- cribed to semantic studies, from simple word tion for the French (cf. Mellor 2006). relations to the intricacies behind discourse Based on the encoding system structures. developed by , the Braille sys- Irrespective of the particularities that charac- tem consists of rectangular cells placed se- terise the diverse frameworks departing from quentially. Each cell corresponds to one char- formal approaches to meaning, there are, acter and it is traditionally constituted by two nonetheless, some important principles that columns each of which presents three “slots” . bring them together. In all cases, predicates are The different combinations of salient or raised related to their argument(s), i.e., entities that dots in each cell gives rise to the several Braille fulfil the predicative relation. Different kinds characters. The six dots that form a cell are of operators are introduced in order to model numbered, so that each character can be iden- the interactions that arise between the partici- tified by the raised dots that represent it. For pants in a sentence or even between indepen- instance, the character “” , in Braille, is iden- dent sentences in a discourse. tified by dots 1 and 2, the character “” by dots

8 2, 3, 4 and 5, and so on. There are 64 possi- and knowledge easily accessible: it allows ble combinations, including no dots at all for blind persons to access technology and to write spaces between words. and read emails, books, newspapers, and sci- As we have pointed out, the original purpose of entific articles on practically equal terms with the development of the Braille system was to respect to sighted people. provide a tactile transcription of the printed al- Although its use is declining in some parts of phabet, enabling blind people to read and write the world, due mainly to the growing influence texts. Since the languages of the world differ of audio media, such as speech synthesizers, considerably from each other, either concern- Braille remains essential for the learning pro- ing the that are employed, or regard- cess and for the general literacy of visually im- ing the number and type of graphic accents and paired people, facilitating their inclusion in the punctuation marks they use, Braille encoding labour market and in other important social ac- is language-specific, i.e., it varies according to tivities, especially when it is used in combi- the chosen language. nation with new technologies (see, e.g., Wia- Braille has proven to be an immensely power- zowski 2014). ful and flexible system in various ways. Of course, it was not our intention here to pro- First, it was fruitfully adapted to include vide an in-depth approach to the internal struc- a large number of languages and alphabets ture and functioning of the Braille system, hav- around the world. Moreover, it was extended ing limited ourselves only to leave a few brief to cover other notations. Music, mathemat- remarks on the aspects that seem most relevant ics, physics, chemistry, phonetics, or informat- and more useful to our purposes. For more de- ics are some of the multiple areas in which tails and information, see, among many others, Braille has been successfully employed to Hampshire (1981), Croisdale, Kamp & Werner make knowledge more accessible to visually (2012), Wiazowski (2014), and, for specific as- impaired people (cf. Herlein 1975; Thompson pects of the Portuguese Braille system, Lemos 2005; Schweikhardt et al. 2006; Englebretson & Cerqueira (1996), Baptista (2000), Reino 2009). (2000) or Pereira (2002). Second, Braille has adapted easily to the emer- 3 The project gence of new technologies, following its con- Given that our project is still at an exceedingly tinuous development. The ancient slate and early stage, it will not be possible to offer here stylus – a handwriting device – has been sub- any relevant results and conclusions. Never- stituted by Braille writers, and, more recently, theless, I will expose our main goals and the by refreshable Braille displays, notetakers and methodology that was chosen to achieve them. tablets (cf. Leonardis, Claudio & Frisoli 3.1 Main goals 2017). As we said earlier, although the Braille system In order to cover a greater range of symbols, has been successfully used in different math- as well as to be used with Braille displays ematical environments (cf. Annamalai et al. and notetakers, the six-dot Braille has been en- 2003; Alonso et al. 2006; Archambault et al. hanced with two additional dots at the bottom 2007; Asebriy, Raghay & Bencharef 2018), of the cell, giving rise to the so-called 8-dot there is no attempt, to our knowledge, that has braille system or 8-dot Braille code. been done to adapt the Braille system to ac- Nowadays, Braille is used with computers, count for semantic formalisation in a compre- tablets, and smartphones, making information hensive and systematic way. This is perfectly

9 reasonable, given, on the one hand, the great thank an anonymous reviewer of this paper for complexity, as well as the inherent difficulties pointing them to me). that such task involves, and, on the other, the The choice for the one-to-one matching be- reduced number of blind people that study or tween symbols used in formal semantics and work with formal frameworks in the semantic braille characters was preferred to the adop- field. tion of an audio output or to their simple sub- To fight against this state of affairs, we will stitution by the corresponding words in natu- take as our main goal the implementation of a ral language. Regarding the use of a speech first approach to the application of the Braille synthetiser to provide a translation of the for- system to formal semantics. malisation, the difficulty of retrieving and re- Bearing in mind that the existing Braille dis- taining information must be taken into account plays are typically constituted by a single line (Braille is much more flexible and reliable in of cells (cf. Leonardis, Claudio & Frisoli this respect). Concerning the translation into 2017), we decided to give primacy to lin- the corresponding words in natural language, it ear formal systems, having therefore excluded would create some difficulties when using au- proposals such as the Discourse Represen- tomatic translation of printed texts in Braille tation Theory developed by Kamp & Reyle displays or Braille printers: note that each (1993) or the Segmented Discourse Represen- “digital” character must be assigned to a well- tation Theory of Asher & Lascarides (2001) defined combination of dots in the Braille sys- that are based on more complicated two- tem in order to be appropriately read by these dimensional graphic representations. How- devices. Moreover, the system we propose ever, if Braille tablets with multiple lines is closer to the original spirit of formalisa- become standard, the inclusion of such ap- tion than the translation into natural language proaches will be easily feasible. words proposal. Besides ascribing the correspondences be- So, the use of Braille (in particular the 8- tween the symbols used in formal semantics dot Braille system) seems the most appropri- and Braille characters, we will try to provide ate way to provide blind students with valu- a simple description of the meaning and use of able tools to learn and manage semantic for- such tools, in order to facilitate the comprehen- malisation. Bear in mind that a 6-dot Braille sion of the internal coherence and functioning notation is also feasible, provided that we in- of formal systems by blind students. troduce compound characters, i.e., characters made of a prefix followed by the main symbol, Moreover, we will attempt to explore the better a practice that is current in many other Braille way to provide visually impaired students and notations. researchers with tools to interpret and produce 3.3 Brief description texts that include semantic formulae. In this subsection, we will provide a brief de- The ultimate goal of our project is, therefore, scription of the steps and tasks that we will un- to make semantic formalisation more friendly dertake to achieve the goals sketched in 3.1. and accessible to blind people. 3.2 A note on challenges and decisions 1) Mapping the characters Before describing the different stages that con- stitute our project, it seems important to ad- The first step to be performed in our dress a few challenges underlying our task (I project is the selection of the set of for-

10 mal symbols that will constitute the base us. However, with the relevant adapta- for mapping the Braille notation. tions, this matching procedure may be extended to cover other Braille codifica- tions. Given the quantity and the diversity of formal approaches to meaning, it will At the end of this stage, we hope to have not be possible to cover the full range a list of Braille characters (both the 6- of existing symbols in this area. In that dot characters and the 8-dot ones) that view, we will select only the most rel- directly correspond to each of the pre- evant and widespread symbols that ap- selected items, giving rise to a coher- pear in introductory textbooks dealing ent “translation” of formal symbols into with formal semantics, that is, those that Braille. constitute the core subjects of the for- malisation that students learn in the in- 2) Exemplification troductory courses of semantics. This does not mean, though, that our list will Since textbooks on formal semantics do be regarded as a closed one; on the con- not account for Braille notation, we con- trary, it will be considered as an open de- sider that it would be helpful to pro- parture point that may be continuously vide some explanation and exemplifica- enriched with new contributions. tion of some particular aspects regarding the use of our formal system with special focus on the consequences of its adapta- Having elected the basic set of sym- tion to the Braille code. bols, the next step will be to estab- lish a mapping between each of them Departing from some introductory texts and the corresponding Braille character. in this area (cf., e.g., Gamut, 1991; Since many of these symbols come from Cann, 1992; Portner & Partee, 2002; frameworks related to mathematics and Kearns, 2011; Winter, 2016), we will logic, we will try to attest the existence provide a simple explanation for the of their Braille correspondents in these meaning and use of each of the symbols systems. at hand. It is not our intention, of course, to offer a complete and detailed intro- duction to the semantic questions in- If the correspondence already exists, we volved, as this is better done by the liter- will incorporate it in our list; if not, we ature we have cited, but only to establish will try to propose a new Braille char- some bridges between the traditionally acter, always obeying the rules that reg- used symbology and the corresponding ulate the internal coherence of this sys- Braille characters, in order to facilitate tem. the understanding of specific issues by blind people. In other words, we want to As we have already pointed out, the make clear some aspects of the formal- Braille rules vary considerably accord- isation that could pose problems to vi- ing to the chosen language. We will take sually impaired students, due to the dif- as our working system the Portuguese ferences that exist between Braille and notation since it is the most familiar to paper-printed representations.

11 To achieve this goal, we will construct correspondences between symbols and Braille a brief presentation of each Braille sym- characters to be correctly displayed. bol, consisting of its Braille representa- Hence, our main task in this third stage of tion in 6 and 8-dot Braille, the explana- our project will be to build a Braille table that tion of its meaning and some examples includes the symbols used in formal seman- of its use in semantic formulae. Such in- tics so that blind students and researchers, em- formation may also be summarized in a ploying specific electronic equipment, can pro- table for easier and faster reference. duce and access texts in this field. Needless to say, the texts to be read have to be writ- 3) Braille tables ten in accordance with the standard symbology used in formal semantics. Unfortunately, texts The last stage of our project deals with the con- that are based on images or graphic representa- struction of Braille tables. tions cannot be properly accounted for by these To convert digital information associated with tools. each character into Braille, screen readers and 4 Conclusion Braille displays make use of Braille tables. A Braille table is designed to make a one-to- Our project aims to turn formal semantics one correspondence between ANSII (or Uni- more accessible to blind students and re- code) characters and their Braille correlates, searchers who want to learn and investigate enabling screen readers and Braille displays to different aspects of this important area of lin- represent the information through the correct guistic knowledge. With this work, we want to combination of dots. contribute in some extent to a better inclusion of this part of the population into graduate and Screen readers, such as JAWS for Windows, post-graduate programs, facilitating their par- Non-Visual Desktop Access (NVDA) or Win- ticipation in university advanced studies. dows Narrator, are a piece of software that turns the visual information that appears in the We believe that accessibility for all is the better computer’s screen into an audio or tactile out- way to spread knowledge and information, to put. The tactile output is typically conveyed improve social inclusion and to favour the em- by a Braille display or a Braille notetaker, powerment of impaired people, contributing to electronic-mechanical devices that present a a more equitable society. Our work hopes to tactile surface in which the refreshable Braille constitute a small step to create better condi- text is gradually exhibited. tions of access and to open new horizons to In the case of Braille displays, the text that is blind people. shown is controlled by an external device – a For future research, it would be interesting to personal computer or a smartphone, for exam- provide Optical Character Recognition (OCR) ple. In the case of Braille notetakers, text can software with the capacity of recognizing the be stored in different formats in their internal symbols used in formal semantics; this would memory, and, besides their usage as computer constitute an important step to turn printed terminals, thanks to their own firmware, these texts fully accessible to screen readers. How- devices can be operated autonomously. ever, the complexities that this task involves In any case, the contribution of a Braille ta- are completely beyond the scope of our present ble that allows the translation of the original work. text into Braille is essential since it permits the References

12 Alonso, Fernando, José Luis Fuertes, Ángel . Kamp, Hans & Uwe Reyle (1993) From Discourse to González & Loïc A. Martínez (2006) “SBT: A translator Logic. Introduction to Model-Theoretic Semantics of from Spanish mathematical Braille to MathML” . In In- Natural Language, Formal Logic and Discourse Rep- ternational Conference on Computers for Handicapped resentation Theory. Dordrecht: Kluwer Academic Pub- Persons: pp. 1207-1214. Berlin / Heidelberg: Springer. lishers. Annamalai, A., . Gopal, G. Gupta, . Guo & A. Kearns, Kate (2011) Semantics (2nd edition). London: Karshmer (2003) “INSIGHT: a comprehensive system Macmillan International Higher Education. for converting Braille based mathematical documents to Kratzer, Angelika (2012) Modals and Conditionals: Latex” . Universal Access in HCI – Inclusive Design in New and Revised Perspectives. Oxford: Oxford Uni- the Information Society, 4: pp. 1245-1249. versity Press. Archambault, Dominique, Bernhard Stöger, Mario Batusic, Claudia Fahrengruber & Klaus Miesen- Lemos, Edison Ribeiro & Jonir Bechara Cerqueira Benjamin Con- berger (2007) “A software model to support col- (1996) “ sistema Braille no Brasil” . stant laborative mathematical work between Braille and , 2. sighted users” . In Proceedings of the 9th in- Leonardis, Daniele, Loconsole Claudio & Antonio ternational ACM SIGACCESS conference on Com- Frisoli (2017) “A survey on innovative refreshable puters and accessibility: pp. 115-122. Doi: braille display technologies” . In International Confer- https://doi.org/10.1145/1296843.1296864 ence on Applied Human Factors and Ergonomics: pp. Asebriy, Zahra, Said Raghay & Omar Bencharef 488-498. Cham: Springer. (2018) “An assistive technology for Braille users Mellor, . Michael. (2006) Louis Braille: A touch of to support mathematical learning: A semantic re- genius. Boston: National Braille Press. trieval system” . Symmetry, 10 (11): 547. Doi: Moens, Marc (1987) Tense, Aspect and Temporal Refer- https://doi.org/10.3390/sym10110547 ence. PhD Dissertation, Edinburgh: Edinburgh Univer- Asher, Nicholas & Lascarides, Alex (2003) Logics of sity. Conversation. Cambridge: Cambridge University Press. Montague, Richard (1974) Formal Philosophy. Selected Bach, Emmon (1986) “The algebra of events” . Linguis- Papers of Richard Montague. . Thomason (ed.), New tics and Philosophy, 9 (1), pp. 5-16. Haven: Yale University Press. Baptista, José António (2000) A Invenção do Braille e Parsons, Terence (1990) Events in the Semantics of En- a sua Importância na Vida dos Cegos. Lisboa: Gráfica, glish: a Study in Subatomic Semantics. Cambridge 2000 (9). Mass.: MIT Press. Cann, Ronnie (1992) Formal Semantics: an Introduc- tion. Cambridge, Cambridge University Press. Pereira, Maria de Lourdes (2002) A Deficiência Visual e a Literacia Braille. Aspectos da Grafia Braille da Lín- Croisdale, D. ., H. Kamp & H. Werner (eds.) (2012) gua Portuguesa. Doctoral dissertation, Lisboa: Facul- Computerised Braille Production: Today and Tomor- dade de Motricidade Humana da Universidade Técnica row. Springer Science & Business Media. de Lisboa. Dowty, David (1979) Word Meaning and Montague Grammar: The Semantics of Verbs and Times and Portner, Paul (2009) Modality. Oxford: Oxford Univer- Generative Semantics and Montague’s PTQ. Dordrecht: sity Press. Reidel. Portner, Paul & Barbara H. Partee (eds.) (2002) Formal Englebretson, Robert (2009) “An overview of IPA Semantics: The Essential Readings. Oxford: Blackwell Braille: An updated tactile representation of the Interna- Publishing Company. tional Phonetic Alphabet” . Journal of the International Reino, Vítor (2000) 170 Anos Depois: Algumas Con- Phonetic Association 39 (1): pp. 67-86. siderações de Ordem Histórica, Sociológica e Psicope- Gamut, L. . T. (1991) Logic, Language, and Meaning. dagógica sobre o Sistema Braille. Lisboa: Biblioteca Volume 1: Introduction to Logic. Chicago: The Univer- Nacional de Portugal. sity of Chicago Press. Schweikhardt, Waltraud, C. Bernareggi, N. Jessel, B. Hampshire, Barry (1981) Working with Braille. Paris: Encelle & . Gut (2006) “LAMBDA: A European Unesco. system to access mathematics with Braille and audio Herlein, Doris G. (1975) “Music reading for the sight- synthesis” . In International Conference on Comput- less: Braille notation". Music Educators Journal 62 (1): ers for Handicapped Persons: pp. 1223-1230. Berlin / pp. 42-45. Heidelberg: Springer.

13 Thompson, David M. (2005) “LaTeX2Tri: Physics and Assistive Technology, 26 (4): pp. 227–230. mathematics for the blind or visually impaired” . In Pro- Wiazowski, Jaroslaw (2014) “Can Braille be revived? ceedings from the 20th Conference on Technology and A possible impact of high-end Braille and mainstream Persons with Disabilities (CSUN). Available online at technology on the revival of tactile literacy medium” . http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.90.1991&Assistive Technology 26 (4): pp. 227-230. rep=rep1& type=pdf Winter, Yoad (2016) Elements of Formal Semantics: An Wiazowski, Jaroslaw (2014) “Can Braille be revived? Introduction to the Mathematical Theory of Meaning in A possible impact of high-end Braille and mainstream Natural Language. Edinburgh: Edinburgh University technology on the revival of tactile literacy medium” . Press.

14