Animated Phonetic Script – Exploring Temporality in Visual Speech
Total Page：16
File Type：pdf, Size：1020Kb
Animated Phonetic Script – Exploring Temporality in Visual Speech Sean Isaacs S0937900 Master of Science – Design and Digital Media August 2016 Word Count: 6138 1 Contents History of Written Language ............................................................................................................... 3 International Phonetic Alphabet ......................................................................................................... 5 Regular Featural Phonetic Scripts ....................................................................................................... 6 Inspiration ........................................................................................................................................... 7 APS Design Context ............................................................................................................................. 9 APS Design ........................................................................................................................................ 10 Consonants ....................................................................................................................................... 10 Manner of articulation (Appendix 4) ............................................................................................ 11 Place of articulation (Appendix 5) ................................................................................................. 11 Voicing (Appendix 6) ..................................................................................................................... 11 Vowels ............................................................................................................................................... 11 Prosodic elements ............................................................................................................................. 13 Methodology ..................................................................................................................................... 13 Evaluation ......................................................................................................................................... 15 Obstacles ........................................................................................................................................... 15 Applications of APS ........................................................................................................................... 17 Acknowledgements ....................................................................................................................... 19 References .......................................................................................................................................... 20 Appendix .............................................................................................................................................. 22 2 Records of the use of written language date back to at least 3,500 B.C. The dominant media on which text is written have changed over the years from clay, to papyrus, to parchment, then to paper, which has been the globally dominant wriding medium since the mid-15th century. Static written text has been directly used in new media, and whilst moving text (such as in Kinetic typography) is not unheard of, it is not a preferred medium of written language communication. This dissertation aims to explore how a dynamic, temporal dimension could be added to the writing of text, by deconstructing how writing has historically been used, and by combining this with the study of the physiology of human speech. Could such an implementation of visual language be the next logical technological advancement? Does it have a place in cutting-edge technologies such as virtual reality displays? History of Written Language Early cuneiform scripts were recorded in impressions created by blunt wedge- In broad terms, language is a system of shaped reed styli in clay – a system which communication of ideas and emotions, of evolved out of the use of symbolic and which speech is the phonic medium of pictographic clay tablets and tokens, transmission (Crystal, 2008). Written predominantly for accounting purposes. language refers to the use of visible signs Papyrus use emerged in Ancient Egypt (or tactile signs in the case of systems such around 3,000 B.C. Owing to the flexibility as braille) to systematically represent units and lightness of papyrus, long texts could of language in order to allow them to conveniently be written, stored and subsequently be retrieved by individuals transported in the form of scrolls and familiar with the encoding of the language codices (which were developed later, in in question (Coulmas, 1999, p. 560). 200-300 A.D.) (UTexas-Austin, n.d.). In Written language is thought to have 200-100 B.C., parchment – a more emerged independently at least twice – in expensive and more durable material made 3,300 B.C. Mesopotamia and 9th century B.C. Mesoamerica (Wilford, 1999; Lo, 2012). The means by which language is transcribed have changed over time as media used to record writing developed. 3 of animal hide, superceded papyrus as the writing material of choice (EncyclopaediaBrittanica, 2016), before being obsoleted by paper, which was invented in China around 100 B.C (GeorgiaTech, 2006). The use of paper spread throughout Asia and into Northern Africa and Europe over the course of 1,000 to 1,300 years, but did not gain significant Figure 1: Kinetic Typography by Gery Greyhound favour in Europe until the rise of the printing https://commons.wikimedia.org/wiki/File:140815_AlvinKyneKonflikt_1.gif press in the mid 15th century (GeorgiaTech, 2006). be applied to the words which appear in works of kinetic typography. For nearly 600 years, paper has remained the dominant medium for written language, Whilst tools such as Adobe Flash/Animate whether for handwriting, printing, or typing. and Apple motion facilitate the animation of In the late 19th century, film added the typography, considerable user input and capacity for movement to visual media. time is still needed. It is therefore frequently Film was used to display text temporally an artistic decision, rather than a favoured whilst still a young medium, with the means of conveying a message, to use earliest known use in 1899 by George kinetic typography. Melies (Bellantoni & Woolman, 1999). This is the first example of kinetic typography Pieces of kinetic typography largely tend to (Figure 1), or moving text, a technique use single words as the smallest unit of which has since seen widespread use the typography to be animated. Experienced forms of animated film and television titles readers are generally thought to interpret and visualised monologues. words holistically to some degree, rather than strictly letter-by-letter (Reicher, 1969), The temporal dimension in kinetic although there are studies which refute this typography affords it with several qualities (Pelli, et al., 2003). Spoken words are absent from static type. The motion of primarily recognised syllable-by-syllable, words appearing and disappearing from and have also been shown to be view can mimic the pace of conversational interpreted phoneme-by-phoneme by speech. Established animation principles individuals with even moderate literacy that give cartoon characters emotional and levels (Mehler, et al., 1981). The visceral qualities (Lodigiani, 2014) can also contrasting means by which whole written words and segmented spoken words are 4 processed means that kinetic typography ordinary Latin characters as possible, cannot capture all of the the fine-level whilst minimising the addition of new temporal detail of speech. characters. Written language also does not capture many prosodic elements of speech. Some, Figure 2: Egyptian Hieroglyps for Pharaoh such as intonation and pitch, can be = ”pr”, logograph for “house”, = ”aa”, logograph for “great” gleaned from learned word pronounciation rules, such as stress-timing in the English The forms of the characters of the Latin language, and punctuation (Crystal, 2008). alphabet have roots in Egyptian This is howver not always the case; the hieroglyphs (~3,500 B.C.), which were sentence “Did he steal my wallet?” can initially logograms of Ancient Egyptian have five different meanings depending on words that were subsequently assigned which word is emphasised. Other more phonemes associated with each logogram complex prosodic properties, such as (Loprieno, 2009). emotion or sarcasm are quite apparent in speech, but must often inferred The characters of the Latin script have contextually with some difficulty from evolved from their logographic roots, written text. tending towards simplicity and ease of writing. There is therefore no available logic In common practice, such prosodic to intuit a phoneme-grapheme relationship elements are not directly encoded into a priori given the characters and phonemes written text, although writing systems such alone. I.e., one could not take any quality of as the International Phonetic Alphabet a phoneme, whether auditory or (IPA), have methods in place to do so physiological, link it to the visual properties (2016). Precise changes in pitch can also of its corresponding grapheme, and then be recorded by combining musical notation apply the same logic to match another with written lyrics. phoneme to its corresponding grapheme. International Phonetic Alphabet In order to visually represent speech at the The International Phonetic Alphabet