Proceedings of ICAD 05-Eleventh Meeting of the International Conference on Auditory Display, Limerick, Ireland, July 6-9, 2005 Lexical Semantics and Auditory Presentation in Virtual Storytelling Research paper for the ICAD05 workshop "Combining Speech and Sound in the User Interface" Minhua Ma and Paul Mc Kevitt School of Computing & Intelligent Systems, Faculty of Engineering, University of Ulster, Magee Campus Derry/Londonderry, BT48 7JL Northern Ireland. {m.ma,
[email protected]} ABSTRACT Audio presentation is an important modality in virtual 2. BACKGROUND AND PREVIOUS WORK storytelling. In this paper we present our work on audio presentation in our intelligent multimodal storytelling system, We are developing an intelligent multimedia storytelling CONFUCIUS, which automatically generates 3D animation interpretation and presentation system called CONFUCIUS. It speech, and non-speech audio from natural language automatically generates 3D animation and speech from natural sentences. We provide an overview of the system and describe language sentences as shown in Figure 1. The input of speech and non-speech audio in virtual storytelling by using CONFUCIUS is sentences taken from children’s stories like linguistic approaches. We discuss several issues in auditory "Alice in Wonderland" or play scripts for children. display, such as its relation to verb and adjective ontology, CONFUCIUS’ multimodal output include 3D animation with concepts and modalities, and media allocation. Finally we speech and nonspeech audio, and a presentation agent—Merlin conclude that introducing linguistic knowledge provides more the narrator. Our work on virtual storytelling so far focussed on intelligent virtual storytelling, especially in audio presentation. generating virtual human animation and speech with particular emphasis on how to use visual and audio presentation to cover more verb classes.