ISSUES IN TEXT-TO-SPEECH FOR FRENCH
Evelyne Tzoukermann
AT&T Bell Laboratories 600 Mountain Avenue, Murray tlill, N.J. 07974 evelyne@rcsearch, art.corn
Abstract in the standard International Phonetic Alphabi,t; the second column ASCII shows the ascii correspon- This paper reports the progress of the French dence of these characters for the text-to-speech text-to-speech system being developed at AT&T system, and the third column shows art example Bell Laboratories as part of a larger project for of the phoneme in a French word. multilingual text-to-speech systems, including lan- guages such as Spanish, Italian, German, Rus- Consonant s Vowels sian, and Chinese. These systems, based on di- IPA ASCII WORD IPA ASCII WORD phone and triphone concatenation, follow the gen- p p paix i i vive eral framework of the Bell Laboratories English t t tout e e the TTS system [?], [?]. This paper provides a de- k k eas e g aisc scription of the approach, the current status of the b b bas a a table French text-to-speech project, and some problems iI d dos u a time particular to French. g g gai 3 > homme m m mais o o tgt n n liOn u U boue 1 Introduction .p N gagner y y tour l 1 livre n ellX In this paper, the new French text-to-sIieech sys- f f faux ce @ seul tem being developed at AT&T is presented; sev- s s si o & peser eral steps have been already achieved while others f S chanter I bain are still in progress. First we present a brief' de- v v vive ~t A bane scription of the phonetic inventory of French, with z z zero 5 O bon a discussion of the approach used to select and 3 Z jupe de I brun segment phonetic units for the system. Methods r r rare o A samedi for automatic segmentation, and for the choice of Semi-vowels diphone and triphone units are presented. Some comments on durational and prosodic issues fol- IPA ASCII WORD low. We conclude with some discnssions on direc- j j yeux tions for fllture improw.'ment, including morpho- w w oui logical analysis, part-of-speech tagging, and par- q W huit tial phrasal analysis for the purpose of phrasal Table 1: French Phonetic Phonemes grouping. For the French text-to-speech synthesis system Phonetic Description of we use 35 phonemes, consisting of 17 consonants, 15 vowels (and not 1{3 like in the n,a cohlmn), and French 3 semi-vowels. As shown in Table 1, the fourth nasal /de/ has been removed, /07,/ and /g/ being The French phonetic system consists of 36 represented by the single phoneme /g/. The rea- phonemes, including 17 consonants, 16 vowels, sons for this change are that (1) /de/ tends to be and 3 semi-vowels. Table 1 shows the different assimilated to the phoneme/g/, and (2) this nasal phonemes; the IPA column contains the phonemes vowel occurs in very few words in French. Thus,
976 iC eould be said thai, functionally the disi, inel, ion eouix~xl of ol hor phon
977 resulting in the devoiced sibilant. nouneiation fields are often not phonetically line- grained enough for acceptable speech output (see for a discussion on machine-readable dictionar- 2.1 Diphone Structure and Selec- [?] ies in text-to-speech systems). Finally, due to the tion of Carrier Word lack of explicit inflectional information for nouns 2.1.1 Structure of Diphones and adjectives, only the non-inflected forms of the entries were extracted during dictionary lookup. This section discusses the nature, of the diphone set Sirnilarly for verbs, only the infinil;iwd forms were and the manner in which diphones were collected. used since the dictionary does not list the intleeted Diphones are structured as [bllows: forms as headwords. A program was written to search through the dictionary pronunciation field *V, *C, g*, C*, CV, VC, CO, gV and select the longest word where the phonenm where * is a silence, C a consonant, and V a pairs would be in mid-syllable position in order to vowel. Semi vowels were treated in the same fash- avoid the extraction of' phonemes occuring at the ion as consonants. Diphones were recorded fol- beginning or end of words. In this way, l,he influ- lowing two (lif[erent strategies: the first one con- ence of lexicM stress was reduced. The orthog- sisted of picking existing words from a dictionary raphy/prommciation pair [headword_orth, head- list. The second consisted of deciding on a neu- wordq~)hon] was extracted and headword_orth was tral phonetic context in using logatornes or non- placed in a carrier sentence for recording. Out existing words. Logatomes are phonotactiically of 1225 original phonemic pairs, 874 words wet'{'. well-formed strings, which do not exist as words found with at least one occurence of the pair. Be in the current French language. cause 1225 is the number of all phonemic pairs in French whether they are allowed or not, it is inter- esting to notice that only 874 pairs occur within 2.1.2 Selection of existing words from real words in the Robert dictionary. machine-readable dictionary
A word list was extracted from a subset of the 2.1.3 Selection of logatomes Robert French dictionary [?] and the pronuncia- tion fields were extracted. The dictionary contains For the logato.tes, two phonen,es /a/ attd /t/ were almost 89,000 entries, of which 85,796 entries con- used to encompass the selected diphone, since they Lain a headword, a phonemic transcription, and a appear to be fairly stable from a phonetic-acousLic part of speech. The remaining entries are prefixes standpoint. In order to balance the alternation of and suffixes. The first task consisted of convert- vowel and consonant, the words were constructed ing and mapping the dictionary phonemic symbols as follows: to the ones adopted in our system (shown in ta- ble 1). This was not straightforward since there Logatome position Structure Example was not always a one-to-one mapping between the initial vow. *V-ta ota two sets. For handling symbol mapping, a pro- initial cons. *C-ata bata gram was written that converts any set of charac- final vow. at-V* at() ters to any other set of characters I. The program final cons. ta-C* Lab is developed so that characters coded in octal or COILS. VOW. ata-CV-ta atabota decimal code not only can he translated in either VOWel (;ons. at-VC-aba atit}ata code, but also can be input in ascii format for be- cons.1 cons.2 ata-C,C-ata at.akrata ing converted 2 vow.1 vow.2 at-VV-ta atoata Quite often, there was more than one pronunci- ation in the phonetic field and the. pattern match- "Fable 2: Phonotactic structure of logatomes ing program chose the pronunciation correspond- ing to the one required. Moreover, dictionary pro- All strings were generated in this way, ewm if 11 am very grateflfl to Mike Tanenblatt who wrote this they were not phonotaeticMly well-formed for iso- program and made a succession of changes until complete lated words in the language. Nonetheless, these flexibility of character conversion was obtained. R)rms were generated and used since they were 2This tool allowed the conversion of databases originally necessary for interword phenomena. Approxi- written on Macintosh, PC, or Unix. Additionally, we used it to convert all the French textual databases into latin1 8 mately 1225 words were constructed following the bit encoding format. ab ow~' model.
978 l{.esearchers disagree as to whether to use representing all the stops, the phoneme./v/ repre- logatomes or real words for synthesis. The ar- seutmg all the fficatiw~s) be picked to rel)resent a gmnent for using logatomcs is that it is t)etter class or sit(mid all the phonemes belonging to the to collect non-real words so that the diphone ix class he sekwted? Resenreh is underway in this recorded as neutrally as possible and does uot un- a,~a using a phone,,, clustering approach [':], [':] dergo any real word stress. Those against argue that allows the sehx:tion of segnwaltal units fi'om a thai; the ([iphone is ov(~r-articulated in a logatome database of I)honemes containing several instances environment and that it reduces l, he naturahmss of of the same phoneme. Tim extraction is made at the synthesized sl)eech. The choice is more corn a spectral point common to the pho,wmes. Fi- plex in the sense that it greatly depends on the nally, he.cause the nnml)er of selected units atfects speaker, the articulation, and the comfort in read results, the choi('e of polyphones must be Ilia(h! ing the two diff('.rent sets. Given the controversy, with care. 'l'aking illto accotlrlt the size limita in the present system, we decided to record the lion, one has to balanc(~ out the choice of the poly l)houemie t)airs in bot;h environments, so thai we phones considering its frequency in ~he. language. (:ould choose the best ones, This brings in the additional complexity of cort)us selection (its language properties, dialects, socio linguisl,ic tyl)e of language, topic, and size). 2.2 The other polyphonic units [?] applies a series of rules on phoneum coln- I)ue to the variability of liquids and semi-vowels, bination to exclude inter-word concatenation that synthesis based only on (liphones will uot give would not occur in French. For example, one can- good results. Indeed, such systems have provcu not lind a glide, in I'~rench that ix not in the left to be insut[icient. Researchers [?] argue l;ha| di- or right cont;ext of a vowel; therefore, the combi- ])holle COllcat(?ll~ttion alolle is llOt a(l(2(ltlate or sl/f- nation consonant-glide-consonant is excluded. An ticient, particularly for complex transitions. [?] optimal set of polyphone combinations is com- claims that "Meal diphones with perfect (;oncatc- puted that re.aches a tmmber of 7725 units. Calcu- nation would giw~ imperf(~ct results". Complex lated from texts, statistics are then run on these polypho,~es are not equivalent to concatenated all- illlits to (teterlllille the most freqllellt oc(;iH'elH;es phones. Therefore, louger concatcnativ(~ units are in French, and the numbex of units is lowered to necessary. Polyphones are defined by ['/] as be- 3000, It remains to be seen whether this al)proacll ing a segmental unit where the initial and linal is successfidl iu a workiug system. phoneme are not subject to variability, thus, ex- cluding liquids and semi-vowels. 2.3 Construction of the corpus The strategy chosen in the Fre.nch system re- A carrier sentence "C'est CAI~I~ItgI~_WOnl) qlle JC lies on some phonetic ge.neralities to build a set of dis" was selected to fulfill the following require tril)honcs. It was decided a to form a (:lass of tri- i'qeill.s: phoues, based on the following transition: I'VC'~ , where 1) is a phoneme, V a vowel, and Cc a conso • short sentelice~ to record, naut rel)resenl,ative of the ~trticulatory locations, i.e. one velar, one dental, and one nasal. The • ability t() surrourid the' carrier word to avoid set consisted then of 35 phones x 14 vowels x 3 selfl, ential accent and effects, consonants = t47() triphon(:s. The same method- ology used for building the set of (liptlones was • phonetically neutral environment. used for the triphon(~'s. These were inchMe(I in a carrier word for the logatomes and extracted from 2.4 Choice of a Speaker the dictionary for the real words. l?ive male natiw: speakers of Continental French Researchers disagree on which criteria are best were interviewed for selc'cting tile voice of the for the selection of triphones; should the selectiou lq'eneh synthesizer. A sample of text represent- rely on phonectic-a<:oustic 979 along with a measure corresponding to the cov- 2.6 Transcription of recording lna- erage of the graphemic triphone. Once tile sample terial was recorded by tire 5 speakers, the natural voices were run through LPC analysis and re-synthesize.d For the recording, all sentences were transcribed in order to judge the resistance of tile voice to from the phonetic alphabet to an orthographic synthesis. Five subjects were asked to give their Ibrrnat. This was done to allow tile speaker judgcrnent on the following criteria: to utter sent(;nees with more naturalness. Once the recording was dorlc'~ th(" sentences were setni- clear articulation: tile voice was carefully automatically re-transcribed into phonc%c form. listened to evaluate tire articulation of the For some~ utterances, the phon('tic transcription speaker. Subjective perceptual judgements was manually adjusted to the idiosynerasi(;s of the were lnade. speaker. For example, it often happened that confusion arises between open and closed vow- 2. neutral French accent: the candidate was els, such in the ~ord '~zoologique" zoological that asked about tile areas of Franc(: where he can be pronomtced either/zooloaik/or/zaalosik/. grew up. The central area of France "l'Ile de In case the output was /zooloaik/ instead of the France" is known for its neutral accent and is expe(%ed /zaalosik/, the transcription was read- regarded as being a well-received accent. Ad- justed. ditionally, for French native speakers resid- ing in the USA, particular attention was paid 2.7 Segmentation to the influence of English in tire prommcia- lion of French, especially for English borrow- Segmentation is presently in progress; efforts are ings, such as for example, the company name being pursued to adapt an automatic segmentor AT&T to be pronounced/a te re/(the French for English to French and other languages. In the way) and not; /el t n t/ as in English. meantime, rnannal segmentation is being done as a pilot experiment in order to cheek the accuracy regularity: special attention was given to en- of automatic segmentation. Beyond the scope of sure that the speaker would have a reason- this paper are many complex issues raised in seg- able degree of regularity in uttering French menting French, such as the segmentation of semi- phonemes. voweds (/j/, /w/, and /q/) and liquids (/l/ and /r/), each of these phonemes being quite unstable ph:asantness of the voic(.': the subjects doing f¥om a phonetic-~eoustic standpoint. These issues the evaluation were asked to give their opinion will be addressed in hmm; work. on the pleasantness of the voice, in particular the timber, the level of nasality, and the into- 2.8 Integration of an orthographic nation. Of course, this is a highly subjective transcriber matter but a critical one for success. A grapheme-to-phoneme transcriber [?] was ac- quired to convert French orthography to a phone.- 2.5 Recording Conditions mie representation. The software performs some syntactic and partial semantic analysis of the sen- The recording was done on four non-consecutive tence in order to disambiguate the input string. days under the following conditions. Thc sen Once performed, spellings at0. converted in a se- tences were recorded directly onto the com- ries of steps into a phonernic representation. puter through a 1)AT (Digital audio 'rape) tape recorder, using interactive software allowing easy reading and repetition of the sentences Lo be 3 Issues in Text Analysis recorded. Additional time was devoted to the recording of triphones as well as the re-recording of We have t)nrsued work in the text analysis of sentences that were improperly uttered. The same French in order to obtain linguistic data for in- carrier sentence and a regular prosodic context tonation and prosody; additionally, the output of was carefully maintained so that there was mini- the work will be used in the translation project. real suprasegmental variation. Once the recording This aspect of the work has entailed several points: was done, the 48 kHz digitized acoustic signal was • acquisition of a large French dictionary: downsized to 12 kllz. lt.obcrt Encyclopedic dictionary (containing 980 over 851¢ ent,rie.s, 80k articles, 160k ei- T,,tah 9 : ,:,,tu tatious, analogical terms (synonyms, ho- MONIDE ~7737: inltond,r~ble. I,a c,.te changcait d' hculc Mf)NI)E 1 t,~656: d,)pause l~ cole 2 1111o n|onlylns, el,(;), and conjugatiou tables for MONI)Iq 33162~H att-,lessus (le la c¢,le 3D~], et 1~ MONI)I'; ~1:I5288: ({Qll]¢lltfellSC, ,'ga cllie de l)llplllglit~ illOSt l?rerl(;[l verl)s), MON[31,} (;88281: ~lols que la cote <1, VM,~ry (ilsc~r,l MONDF) 7{11355: par nnc ,:ot~ de poplll~itg. • collectiol~ of French corpora: Table 3: Some concordances of the word "cot<'" in i,'rcneh news from LI'; M()NI)I'; [?] the databas(, "l,e Monde" I/'retlch news daily <:ompih'.d by the French embassy in Washington DC the word "cote" ill the thre.e Fr(mch cor[)ora. (24657K byt;es arc now en<:oded, and a Moreover, the " i" option allows the retrieval monthly update is being done.). Tim of all instances of a word with or without ac-. data are in ascii and aeeeltts were re= cent, therefore {,he three l/'rench words "cote", stored using oue of the features of the "trite", and"c6t6". For more information on gral>hetue:l,o: pholmlne software. An tlw use of the concordance tools, refer to [?], other [)rogl?aliq was writl, en to alltOtll&t M^tch: ,:,,t(, ieally cletm aim norntalize these e-mail 'l'l,t al: 9 [or|nat d ;-d.a. 9 : ,,,,re MONDH 2673: pied ~ul 1^ e6le qu' [Is serMcllt extraction of some of the, I.Lot>crt di(> MONDI; 3835: i,r~vu : l& 1:611~ dll C:Mvados. MONDE ,ID811 de (cux de la 1:61e :~u,4 de tionary datal>as<~s: the 160,000 citations MONI)Iq 41MIII~: unlvc;~it&. ,le i~ ,:file .xtl~ntlque. AI"P 257O: 8avel~t. (:6t6 t r~vailliste tTrom literary Fren 981 we also intend to build a TTS system for Canadian [13] Joe P, ()live. Rule synthesis of speech fi'om French. dyadic units. In Proceedings of the IEEE- ICAS"SP, pages 569 570, 1977. References [14] Joe P. Olive. A new Mgorithm for a eoncate- native speech synthesis system using an aug- [1] Veronique Aul)ergd. La synth~se de la parole: mented acoustic inventory of speech sounds. 'des r@les aux lezique'. I'M) thesis, (lniver- In Gdrard Bailly arid Christian Benoit, edi- sitd de Grenoble, Grenoble, France, 1991. tors, Proceedings of the ESCA Work:shop on Speech Synthesis, 1990. [2] Jared 11ernstein. Speech synthesis: System design and applications, pages 39 42. Na- [15] Joe P. ()live and Mark Y. Liherman. A set of tional Computer Conference, 1987. concatenative ,,nits for speech synthesis. In In J. ,I. Wolf and I). t1. Klatt, editors, Speech [3] Fr~,ddric Bimbot. Synth~se de la Parole: Des Communication Papers Presenled at the 97th. Segments aux r@les, avec utilisalion de la Meeting of the Acoustical Society of America, ddcomposition lemporelle. PhD thesis, Tele- pages 515 518, New York: American Insti- corn Paris 88 EOl9, Paris, France, 1988. tute of Physics, 1!)79. [16] G. E. Peterson, W.S.Y. Wang, and E. Siver- [4] M. Chafeouloff. Les propridtds acoustiques stem Segmentations techniques in speech syn- de [j, y, 1, r] en franeais, volume 6, pages thesis. Journal of the Acoustical Society of 10 24. Travaux de l'Institut de Phondtique America, 30:8:739 749, 1958. d'Aix, 1979. [17] Scott Rosenherg and Joseph 1'. Olive. Ex- [5] N.R. Dixon and H. D. Maxey. Terminal arm- pediting the selection and clipping of nmlti- log synthesis of continuous st,eech using the phone sequences for acoustic inventory. diphone method of segment assembly. In In 11222-930830-I2TM, Murray Hill, N,a., [EEE 7)'ansactions on Audio and Electroa- USA, 1993. Technical Memorandum, AT& cousties A U16, pages 40 50, 1968. Bell Laboratories. [6] Fxancoise Emerard. Synth~se par diphones el [18] ,lira Rowley. Phoneme clustering tools. traitement de la prosodic. PhD thesis, Univer- In 11222-931123-25TM, Murray flill, N.J., sitd de Grenoble Ill, Grenoble, France, 1977. USA, 1993. Technical Memorandum, AT& Bell Laboratories. [7] Alain Duval et al. Robert Encyclopedic Dic- tionary (CD-ROM). Hachette, Paris, 1992. [19] R. Schwartz, J. Klovst,ad, J. Makhoul, D. Klatt, and V. Zue. l)iphone synthesis [8] Judith Klavans and Evelyn(; Tzoukermann. for phonetic vocoding. In Proceedings of the The use of machine-readable dictionaries in LEI3E-ICASSP, page 891, 1979. text to speech, under review, 1994. [20] Evelyne Tzoukermann and Mark Y. Liber- [9] P. Laferrib.re, G. Chollet, L. Miclet, and J.P. man. A finite-state morphological proces- Tubaeh. Segmentation d'une base de donndes sor for spanish. In Proceedings of Colin990 , de 'polyson', application 5, la synth;;se de pa- Helsinki, Finland, 1990. International Con['er- role. In JEP: 1~, pages t07 .110, 1985. (.'tic(; on Computational Linguistics. [10] L'histoire au jour le jour 194:4-1991. In ver- [21] Church Kenneth W. Concordancc's for par- sion 199~. CD-ROM, 1992. allel text. Oxford, England, 1991. Seventh Annual Conference of the UW Centre for the. [11[ F, Marty. Trois systbmes informatiques de New ()El) and Text Research. transcription phondtique et graphdmique. Le [22] David Yarowsky. Cone: Tools for text cor Franeais Moderne, LX, 2:179 197, 1992, pora. In 1.1222-921222-29TM, Murray ltill, [12[ L, Miclet. Enregistrernent d'une base N.J., USA, t992. Technical Memorandum, de donndes vocales. In LAA/TSS/RCP AT& Bell Laboratories. E.N.S.T. CNET, 1984. 982