• Abstract and Figures
• Public Full-text

barelyMusician: An Adaptive Music Engine For Interactive Systems by Alper Gungormusler, B.Sc. Dissertation Presented to the University of Dublin, Trinity College in fulfillment of the requirements for the Degree of Master of Science in Computer Science (Interactive Entertainment Technology) University of Dublin, Trinity College September 2014 Declaration I, the undersigned, declare that this work has not previously been submitted as an exercise for a degree at this, or any other University, and that unless otherwise stated, is my own work. Alper Gungormusler September 1, 2014 Permission to Lend and/or Copy I, the undersigned, agree that Trinity College Library may lend or copy this thesis upon request. Alper Gungormusler September 1, 2014 Acknowledgments First and foremost, I would like to thank my supervisors Dr. Mads Haahr and Natasa Paterson-Paulberg for their guidance and support in the completion of this project. I would like to specially thank Altug Guner, Talha & Tarik Kaya (Kayabros) for the ridiculous amount of playtesting they had to suffer throughout the process. Finally, I would like to thank my family and friends for all their encouragement and support. Alper Gungormusler University of Dublin, Trinity College September 2014 iv barelyMusician: An Adaptive Music Engine For Interactive Systems Alper Gungormusler University of Dublin, Trinity College, 2014 Supervisors: Mads Haahr, Natasa Paterson-Paulberg Aural feedback plays a crucial part in the field of interactive entertainment when delivering the desired experience to the audience particularly in video games. It is, however, not yet fully explored in the industry, specifically in terms of interactivity of musical elements. Therefore, an adaptive music engine, barelyMusician, is proposed in this dissertation in order to address this potential need. barelyMusician is a compre- hensive music composition tool which is capable of real-time musical piece generation and transformation in an interactive manner, providing a bridge between the low-level properties of a musical sound and the high-level abstractions of a musical composi- tion which are significant to the user. The engine features a fully-functional software framework alongside a graphical user interface to enable an intuitive interaction for the end-user. v Contents Acknowledgments iv Abstract v List of Tables ix List of Figures x Chapter 1 Introduction 1 1.1 Motivation . 1 1.2 Objectives . 3 1.3 Dissertation Roadmap . 3 Chapter 2 State of the art 4 2.1 Background . 4 2.1.1 Tonal Music Theory . 4 2.1.2 Adaptiveness and Virtual Environments . 7 2.1.3 A Brief History of Algorithmic Composition . 8 2.2 Related Work . 10 2.2.1 Offline Music Generation . 11 2.2.2 Real-Time Music Generation . 12 2.2.3 Affective Music Transformation . 13 Chapter 3 Design 16 3.1 Main Architecture . 17 3.2 Hierarchical Music Composition . 18 vi 3.3 Rule-based Note Transformation . 20 3.4 Audio Output Generation . 22 3.5 Limitations . 23 Chapter 4 Implementation 25 4.1 Technology Overview . 26 4.2 Main Components . 27 4.2.1 Sequencer . 27 4.2.2 Instrument . 29 4.2.3 Generators . 31 4.2.4 Ensemble . 32 4.2.5 Conductor . 33 4.2.6 Musician . 35 4.3 API Properties . 36 4.3.1 User Interface . 36 4.3.2 Supplementary Features . 39 Chapter 5 Demonstration 41 5.1 Sample Presets . 41 5.2 Proof-of-concept Applications . 45 5.2.1 Demo Scene . 45 5.2.2 Game Prototype . 47 Chapter 6 Evaluation 49 6.1 Level of Interactivity . 50 6.2 Quality of Generated Audio . 50 6.3 General Discussion . 52 Chapter 7 Conclusion 53 7.1 Contributions . 53 7.2 Future Work . 54 7.2.1 Community Support . 54 7.2.2 Licensing & Release . 55 vii Appendices 56 Bibliography 59 viii List of Tables 4.1 Mapping between the musical properties and the mood. 34 4.2 High-level abstractions and the corresponding mood values. 35 5.1 Sample ruleset and an arbitrary iteration of the generative grammar algorithm. 42 5.2 Demo scene settings. 46 5.3 Game prototype settings. 48 ix List of Figures 3.1 Main architecture dependencies diagram. 17 3.2 Generators structural diagram (example iterations are given on the right). 19 3.3 Note transformation diagram. 20 3.4 Instrument dependencies diagram. 22 4.1 Sound envelope example (horizontal axis: time, vertical axis: amplitude). 30 4.2 Sample screenshot of the custom editor for the main component. 37 4.3 Sample screenshot of the custom editor for performer creation. 38 5.1 Sample iteration and note mapping of a 2D cellular automaton. 43 5.2 Sample screenshot of the demo scene. 45 5.3 Sample screenshot of the game prototype. 47 6.1 Some interesting comments from the participants. 49 x Chapter 1 Introduction Aural feedback plays a crucial part in the field of interactive entertainment when deliv- ering the desired experience to the audience particularly in video games. It is, however, not yet fully explored in the industry, specifically in terms of interactivity of musical elements. Therefore, an extensive approach for development of an adaptive music com- position engine is proposed in this dissertation research project in order to address this potential need. The motivation behind the choice of this topic is presented in greater detail in Sec- tion 1.1. An overview of the main objectives of the proposed approach is provided in Section 1.2. The organization of the rest of this report is presented in Section 1.3. 1.1 Motivation There has been a tremendous developmental curve with regards to the technological aspects in the interactive entertainment field in the last two decades { specifically in the video game industry. While the main focus was mostly towards the visual aspects of the experience, with the maturation of graphics technologies in the field, developers recently tend to focus more on the other areas|such as game AI|in order to take one step further in the field. 1 That being said, one of the most important aspects which dramatically affects the immersion|hence, the quality of the experience|in the field is the use of audio ele- ments [1]. Until recently, the term interactive, was often discarded in such applications for game audio, especially in terms of musical elements, even in high-budget main- stream products [2]. The main reason why is arguably due to the misinterpretation of the problem which led the industry to treat the musical pieces as film scores. Con- sequently, even though many rich and sophisticated scores have been composed, they still more or less lack the main feature required; interactivity. Having said that, unlike movies, video games as interactive applications rarely follow a linear path, and more importantly they offer a great range of repeatability which makes dynamism|hence, adaptability|a must to have in terms of the audio content, as it is in other parts of such applications. While a few examples could be found in the industry which try to achieve such adaptability to a certain extent (see Section 2.2), the issue has not been solved yet to an acceptable level particularly considering the practical aspects. Fortunately, there is recently an increasing interest and demand in the field in order to achieve adaptive and responsive aural feedback [3]. Nevertheless, while significant progress has been made for interactive sound synthesis in academic research as well as in practical applications in the industry [4, 5], achieving adaptive music compositions in real-time still remains an unsolved problem. More specifically, existing technologies in the field, labeled as dynamic, generally rely on the vertical approach [6] { layering and branching the composition into segments. Thus, the resulting output can be varied in an automated manner by certain event triggering approaches. The idea is, however, still based on pre-recorded audio loops which do not allow sufficient dynamic adapt- ability and flexibility after the offline creation process [7]. Moreover, such approaches require considerable amount of resources and dependencies mainly due to the manual authoring of the outcome. As a matter of fact, there have been several attempts in the industry to take this approach one step further by introducing certain generative algorithms in order to pro- duce the musical elements procedurally. However, those remained as specific solutions exclusive only for those specific projects. Having considered this, there is no generic approach, i.e. all purpose solution with such capabilities in the field. 2 1.2 Objectives To address the issues and the potential need stated in the previous section, this re- search project proposes a practical approach for development of a middleware tool, an adaptive music engine, which is capable of autonomously generating and manipulating musical structures in real-time to be used by not only developers but also designers. The proposed approach treats the problem in an interactive manner, providing a bridge between the low-level properties of a musical sound and the high-level abstractions of a musical composition which are meant to be significant to the end-user. Moreover, it features a fully-functional software framework alongside a graphical user interface to enable an intuitive interaction. Even though the main target is chosen to be the video games industry, the approach is also applicable to any other areas in the field of interactive digital entertainment if desired. 1.3 Dissertation Roadmap State-of-the-art in the field is presented in Chapter 2 introducing some essential back- ground research and relevant work in academia as well as the industry in order to develop an understanding and an analysis of the stated problem. It is followed by the methodology of the proposed approach and the implementation phase in full de- tail in Chapter 3 and 4 respectively.

Load more

  73

Copy Link