SPeech Phonetization Alignment and Syllabification (SPPAS): a tool for the automatic analysis of speech prosody Brigitte Bigi, Daniel Hirst To cite this version: Brigitte Bigi, Daniel Hirst. SPeech Phonetization Alignment and Syllabification (SPPAS): a tool for the automatic analysis of speech prosody. Speech Prosody, May 2012, Shanghai, China. pp.19-22. hal-00983699 HAL Id: hal-00983699 https://hal.archives-ouvertes.fr/hal-00983699 Submitted on 25 Apr 2014 HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est archive for the deposit and dissemination of sci- destinée au dépôt et à la diffusion de documents entific research documents, whether they are pub- scientifiques de niveau recherche, publiés ou non, lished or not. The documents may come from émanant des établissements d’enseignement et de teaching and research institutions in France or recherche français ou étrangers, des laboratoires abroad, or from public or private research centers. publics ou privés. SPeech Phonetization Alignment and Syllabification (SPPAS): a tool for the automatic analysis of speech prosody. Brigitte Bigi, Daniel Hirst Laboratoire Parole et Langage, CNRS & Aix-Marseille Université [email protected], [email protected] Abstract length, pitch and loudness of the individual sounds which make up an utterance”. [7] SPASS, SPeech Phonetization Alignment and Syllabifi- cation, is a tool to automatically produce annotations Linguists need tools for the automatic analysis of at which include utterance, word, syllable and phoneme seg- least these three prosodic features of speech sounds. In mentations from a recorded speech sound and its tran- this paper we concentrate on the tool for the automatic scription. SPPAS is currently implemented for French, analysis of segmental duration. We return in the final English, Italian and Chinese and there is a very simple section to the question of interfacing the tool we describe procedure to add other languages. The tool is developed here with the analysis of other prosodic parameters. for Unix based platforms (Linux, MaxOS and Cygwin on The analysis of the prosodic structure of speech Windows) and is specifically designed to be used directly nearly always requires the alignment of the speech record- by linguists in conjunction with other tools for the auto- ing with a phonetic transcription of the speech, usually matic analysis of speech prosody. The tools will all be in some version of the International Phonetic Alphabet. distributed under a GPL license. This task is extremely labour-intensive - it may require Index Terms: phonetic, annotation, speech prosody, several hours for even an experienced phonetician to tran- intonation scribe and align a single minute of speech manually. It is consequently obvious that transcribing and aligning sev- eral hours of speech by hand is not generally something 1. Introduction which can be envisaged. The last ten years or so has seen an explosion in the A number of tools are currently available which can quantity of linguistic data which has become available be used to automate the task, including the HTK Toolkit as evidence for the nature of linguistic representations [8], Festival [9], Julius [10], the P2FA [11], EasyAlign [12], of speech. Whereas it was common some years ago to For most of these, our experience is that the tools formulate prosodic models on the basis of rather limited require a level of expertise in computer science which data, today it is becoming more and more expected for is beyond the reach of most linguists without the help linguists to take into account large quantities of empirical of an engineer. P2FA, for example, assumes that HTK data, often including several hours of recorded speech. is installed on the user’s computer and it is run via a Up until a few years ago it was difficult, if not impos- text-based command-line interface. The EasyAlign Praat sible, to manipulate such large quantities of data outside plugin [12] is currently the most “linguist-friendly” but of specialized laboratories using expensive main-frame it currently runs only on Windows and relies on compo- computers. Software for the analysis of acoustic data was nents which cannot all be distributed with a GPL license. often difficult to install and use, nearly always platform EasyAlign is currently available for French and English specific and, above all, very expensive. although the author claims that adding a new language Today the situation has changed radically and the is relatively easy and announces forthcoming versions for availability of cheaper and cheaper data storage has made German, Dutch and Hebrew. it possible to envisage data analysis involving several We are developing a GPL licensed set of tools for Unix hundreds of gigabytes of data. In the same period, a based platforms, including Linux, Mac-OSX and Win- number of software tools for the analysis of audio and/or dows (using Cygwin). The tool we describe here imple- video recordings of speech have become available such ments the automatic phonetization, alignment and syl- as Anvil [1], Elan [2], Praat [3], Transcriber [4] and labification of speech recordings and it is specifically de- WaveSurfer [5], to name just some of the most popu- signed to be used directly by linguists in conjunction with lar tools, all of which are both free and multi-platform. other tools for the automatic analysis of speech prosody. For an extensive list of speech analysis software see [6]. The biggest obstacle today facing linguists is not the 3. SPPAS description storage of data, nor its analysis, but its annotation. 3.1. Overview 2. Tools for prosody analysis SPPAS - SPeech Phonetization Alignment and Syllabi- fication, is a tool to automatically produce annotations The description of the prosodic structure of speech re- which include utterance, word, syllable and phoneme seg- quires: mentations from a speech recording and its transcrip- “the explicit characterization of the tion. SPPAS is currently implemented for French, En- glish, Italian and Chinese and there is a very simple pro- 3.3. Phonetization cedure to add other languages. Clearly, there are different ways to pronounce the same The whole process is a succession of 4 automatic utterance. Different speakers have different accents and steps. The resulting alignments are a set of TextGrid tend to speak at different rates. Phonetization is the pro- files, the native file format of the Praat software [3] which cess of representing sounds by phonetic symbols. There has become the most popular tool for phoneticians today. are two general ways to construct a phonetization pro- SPPAS generates separate TextGrid files for (i) utterance cess: rule based systems (with rules based on inference segmentation, (ii) word segmentation, (iii) syllable seg- approaches or proposed by expert linguists) and dictio- mentation and (iv) phoneme segmentation. The output nary based solutions which consist of storing a maximum is illustrated as a Screenshot in Figure 1, with TextGrid of phonological knowledge in a lexicon. SPPAS imple- files merged in Praat. For the Chinese example, only 3 ments both approaches for French, but currently only the tiers are illustrated with the tokens tier corresponding second for other languages since the rule-based approach to both syllables and morphemes since the corpus was is of course highly language dependent. transcribed in Pinyin. The program for the phonetization of the ortho- A important point for software which is intended to graphic transcription produces a phonetic transcription be widely distributed is its licensing conditions. SPPAS based on a phonetic dictionary. In the case of phonetic uses only resources, tools and scripts which can be dis- variants, no rules are applied: all possibilities are taken tributed under the terms of the GPL license. SPPAS into consideration. The phonetization contains all pos- tools and resources are currently available at the URL: sible grapheme-to-phoneme conversions of each word of the transcription. If a word cannot be found in the dic- http://www.lpl-aix.fr/∼bigi/sppas/. tionary, the phonetization is “UNK”. In this case, the ut- terance containing the missing word will not be aligned: Recently, the software was successfully applied during the the missing word will have to be added to the dictionary Evalita 2011 campaign, on Italian map-task dialogues. and SPPAS will need to be re-started. EVALITA is an initiative devoted to the evaluation of To perform the phonetization, an important step is to Natural Language Processing and Speech tools for Ital- build the pronunciation dictionary, where each word in ian1. Systems were required to align audio sequences of the vocabulary is expanded into its constituent phones. spoken dialogues to the provided relative transcriptions. The phonetization is the equivalent of a sequence of Final results report a correct phoneme alignment rate of dictionary-look-ups. It supposes that all words of the 88% (both phonetization and alignment errors), and a speech transcription are mentioned in the pronunciation correct word alignment rate of 97.6%. dictionary. Actually, some words can correspond to sev- There is a simple procedure to add new languages eral entries in the dictionary with various pronunciations. in SPPAS: it requires only putting a dictionary and an To deal with such a case, SPPAS decides on the correct acoustic model into the appropriate directory. pronunciation during the alignment step because the pro- nunciation can generally be inferred from the speech sig- 3.2. Inter-Pausal Unit Segmentation nal. For example, the following input sentence: “I never Inter-pausal unit segmentationn is a completely open re- get to sleep on the airplane” will produce the following search problem. It consists in aligning sentences of a phonetization: “ay n.eh.v.er g.eh.t|g.ih.t t.uw|t.ix|t.ax document with the corresponding sound. Our current s.l.iy.p aa.n|ao.n dh.ax|dh.ah|dh.iy eh.r.p.l.ey.n” using implementation of SPPAS includes an algorithm for this the Arpabet transcription alphabet.